~www_lesswrong_com | Bookmarks (712)

Published on April 17, 2025 5:12 PM GMTLast fall the Roots of Progress Institute hosted the first annual Progress Conference. 200 people excited about human progress gathered for two days in Berkeley, California, to share ideas in deep conversation, catalyze new projects, and get energized and inspired. Several attendees even said it was the best conference they had ever attended. We shared more about...

Published on April 17, 2025 3:10 PM GMT I do a lot of work on EC2, where I ssh into a few instances I use for specific purposes. Each time I did this I'd get a prompt like: $ ssh_ec2nf The authenticity of host 'ec2-54-224-39-217.compute-1.amazonaws.com (54.224.39.217)' can't be established. ED25519 key fingerprint is SHA256:... This host key is known by the following other names/addresses:...

Published on April 17, 2025 3:10 PM GMTOpenAI has upgraded its entire suite of models. By all reports, they are back in the game for more than images. GPT-4.1 and especially GPT-4.1-mini are their new API non-reasoning models. All reports are that GPT-4.1-mini especially is very good. o3 is the new top of the line ChatGPT reasoning model, with o3-pro coming in a few...

Published on April 17, 2025 2:14 PM GMTThe US is experiencing a great decline in trust. According to the US General Social Survey, people who agreed with the statement "most people can be trusted" went from 49% to 25% between 1984 and 2022. Trust in institutions is also falling. Over that same period trust in the government fell from 42% to 20%. For civil services it...

Published on April 17, 2025 12:31 PM GMTIt seems to me the question of consciousness of LLMs is a bit of a red herring. Instead the salient point is that they are sequence learning systems similar to our cortex (+ hippocampus). Therefore we should expect them to be able to learn sequences. What we should not expect is that they feel pain. In humans that is...

Published on April 17, 2025 11:36 AM GMTWhat is this about?With some parameters we have thus defined probabilistic skills, one can formulate questions for special moments in life to help organize events, routines, and tasks. This approach approaches the atomic model where, even if we cannot precisely pinpoint an electron’s layer, through specific questions we can predict which area our focus lies in.It also...

Published on April 17, 2025 11:04 AM GMTWhat is this about?We often experience complex things (like learning) that can’t be understood just by looking at their pieces. This framework uses ideas from information theory (how systems handle uncertainty) to give us five simple questions for examining our own thinking and behavior.1. The Mega Vector: Information EfficiencyThe “Mega Vector”: Humans as information machines. We organize, transform,...

Published on April 17, 2025 1:04 AM GMTCrossposting from my substack.Note for LW: This post is about the apathy and nonchalance I've been seeing in many peers regarding AGI. There seems to be massive cognitive dissonance (and it isn't straightforward to overcome) – how can we on the one hand talk about things like AIs becoming smarter than humans across all domains, and on...

Published on April 16, 2025 9:47 PM GMTSUMMARY:  ALLFED is making an emergency appeal here due to a serious funding shortfall. Without new support, ALLFED will be forced to cut half our budget in the coming months, drastically reducing our capacity to help build global food system resilience for catastrophic scenarios like nuclear winter, a severe pandemic, or infrastructure breakdown. ALLFED is seeking $800,000 over...

Published on April 16, 2025 9:30 PM GMTMidjourneyThanks to Renaissance Philanthropy for their support of my research on this topic.Why Prodromes?A lot of chronic diseases, particularly autoimmune and neurodegenerative disorders, develop over the course of years, as the patient transitions from “healthy” to “sick”. This may involve the loss of neurons (as in neurodegenerative diseases like Alzheimer’s or Parkinson’s) or the loss of other...

Published on April 15, 2025 6:14 PM GMTOne key hope for mitigating risk from misalignment is inspecting the AI's behavior, noticing that it did something egregiously bad, converting this into legible evidence the AI is seriously misaligned, and then this triggering some strong and useful response (like spending relatively more resources on safety or undeploying this misaligned AI). You might hope that (fancy) internals-based...

Published on April 15, 2025 4:01 PM GMTWelcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.In this newsletter, we cover the launch of AI Frontiers, a new forum for expert commentary on the future of AI. We also discuss AI 2027, a detailed scenario describing how artificial superintelligence might...

Published on April 15, 2025 3:56 PM GMTIntroductionWriting this post puts me in a weird epistemic position. I simultaneously believe that:The reasoning failures that I'll discuss are strong evidence that current LLM- or, more generally, transformer-based approaches won't get us AGIAs soon as major AI labs read about the specific reasoning failures described here, they might fix themBut future versions of GPT, Claude etc....

Published on April 15, 2025 3:30 PM GMTThree big OpenAI news items this week were the FT article describing the cutting of corners on safety testing, the OpenAI former employee amicus brief, and Altman’s very good TED Interview. The FT detailed OpenAI’s recent dramatic cutting back on the time and resources allocated to safety testing of its models. In the interview, Chris Anderson made...

Published on April 15, 2025 1:02 PM GMTHi, I thought it might be interesting to some here to take note of the 3M subscriber YouTube channel "Channel 5 with Andrew Callaghan" posting an hour-long news room style podcast/documentary/report about the rationalist movement (in the context of Ziz, but covering it much more generally). Full disclaimer that I am a personal fan of Andrew for his...

Published on April 15, 2025 1:44 PM GMTBasically, the linkpost argues that the broad reason why benchmark progress is disconnected from broader progress is because AI progressed so fast that they didn't realize their benchmarks would be obsolete soon.More is in the link above.Discuss!Relatedly, another part of the story may be that researchers underestimated the rate of AI progress. For instance, researchers working on...

Published on April 15, 2025 1:04 PM GMTThe original Map of AI Existential Safety became a popular reference tool within the community after its launch in 2023. Based on user feedback, we decided that it was both useful enough and had enough room for improvement that it was worth creating a v2 with better organization, usability, and visual design. Today we’re excited to announce...

Published on April 15, 2025 12:33 PM GMTSummary We conducted a small investigation into using SAE features to recover sandbagged capabilities.We used the Goodfire API to pick out 15 SAE features, 5 related to sandbagging, 5 related to general expertise, and 5 unrelated features.We used these features to steer a Llama 3.3 70b Instruct that has been prompted to sandbag We observed that steering can recover...

Published on April 15, 2025 11:10 AM GMT The ideal seat height for foot percussion is significantly higher than a typical chair. I've tried a few things over the years: Stacking chairs: works great, until you come to a venue without any stackable chairs. Drum stool: good (unless you buy a very cheap wobbly one like I did) but heavy and not good for...

Published on April 13, 2025 5:40 PM GMTWe live in a world where numerous agents, ranging from individuals to organisations, constantly interact. Without the ability to model multi-agent systems, making meaningful predictions about the world becomes extremely challenging. This is why game theory is a necessary prerequisite for much of economic theory.Most relevant agents today are either human (such as you, me, or Mira...

Published on April 13, 2025 3:40 PM GMTTL;DR: I claim that many reasoning patterns that appear in chains-of-thought are not actually used by the model to come to its answer, and can be more accurately thought of as historical artifacts of training. This can be true even for CoTs that are apparently "faithful" to the true reasons for the model's answer.Epistemic status: I'm pretty...

Published on April 13, 2025 11:22 AM GMTEpistemic status: a model I find helpful to make sense of disagreements and, sometimes, resolve them. I like to categorize disagreements using four buckets:Facts, values, strategy and labels They don’t represent a perfect partitioning of “disagreement space”, meaning there is some overlap between them and they may not capture all possible disagreements, but they tend to get me pretty far...

Published on April 13, 2025 6:37 AM GMTThanks to Linda Linsefors for encouraging me to write my story. Although it might not generalize to anyone else, I hope it will help some AI safety newcomers get a clearer picture of how to get into the field.In Summer 2023, I thought that all jobs in AI safety were super competitive and needed qualifications I didn’t have....