~www_lesswrong_com | Bookmarks (702)

Published on April 13, 2025 1:09 AM GMTIntroductionThis is a nuanced “I was wrong” post.Something I really like about AI safety and EA/rationalist circles is the ease and positivity in people’s approach to being criticised.[1] For all the blowups and stories of representative people in the communities not living up to the stated values, my experience so far has been that the desire to be truth-seeking...

Published on April 12, 2025 11:15 PM GMTWe published the MONA paper about three months ago. Since then we’ve had many conversations about the work, and want to share some of the main updates that people make after talking to us:The realism of our model organismsWhat does "approval" meanIsn't this just RLHF?Why should we care about reward hacking?We’ve also updated the paper with a new...

Published on April 12, 2025 10:35 PM GMTIn his famous 1644 treatise on freedom of speech, Areopagitica, John Milton uses the archaic term dividual to describe religion as something that can come and go, rather than being identified in a one-to-one relationship with a person.“… a man may say his religion is now no more within himself, but is become a dividual movable, and...

Published on April 12, 2025 10:22 PM GMTA vivid introduction to Commitment RacesWhy committed agents defeat uncommitted agentsImagine it's the year 2050. We have an aligned Artificial Superintelligence (ASI) working with us, and we've used self replicating nanobots to convert a lot of matter in the solar system into giant supercomputers and factories. We have a lovely utopia, we have all the resources we'll...

Published on April 12, 2025 7:32 PM GMTSomething that's painfully understudied is how experts are more efficientthan novices while achieving better results. I say understudied and notunstudied, because it's common knowledge that charging people for their timeresults in experts being paid less since they work faster, which is why expertscharge more for their time.This effect is understudied in the impact it has on novices...

Published on April 12, 2025 7:20 PM GMTDisclaimer: This is Kongo Landwalker's translation of lsusr's fiction Luna Lovegood and the Chamber of Secrets - Part 3 into russian. — Торжественно клянусь, что замышляю только шалость.Фред и Джордж с тех пор, как в прошлом году тролль напал на школу, всегда носили с собой спрятанные мётлы. Когда они услышали пророчество Гарри Поттера на финале по квиддичу, то...

Published on April 11, 2025 2:56 PM GMTThe interactions between a misaligned AI and mankind have often been compared to the interactions of Spanish conquerors and Native Americans or of European colonizers and local peoples. Fortunately, most civilised peoples have apparently stopped the destructive practices like confiscation of land in Algeria and condemned colonialism (however, postmodernists have apparently grossly expanded the latter notion to...

Published on April 11, 2025 4:33 PM GMTTLDR: We show that Qwen-2.5-3B-Instruct can learn to encode load-bearing reasoning tokens when RL incentives this in a simple reasoning task. However, it was difficult to achieve this result. Our attempts to learn encoded reasoning via RL were generally bottlenecked either by exploration issues or hacking of the CoT monitor.IntroductionMotivation.Reasoning LLMs rely on natural language chains of thought (CoT)...

Published on April 11, 2025 4:31 PM GMTI wake up as usual by immediately going on my phone. I reply to some of my messages - yes dad, the solar installation you have at the villa could be redundant soon, but again, it can still benefit them (even if slightly) if you comply and let them integrate it into the new grid. A grid...

Published on April 11, 2025 3:05 PM GMTCross-posted from Substack.AI job displacement will affect young people first, disrupting the usual succession of power and locking the next generation out of our institutions. I’m coining the term youth lockout to describe this phenomenon.Youth LockoutWe are on track to build AI agents that can independently perform valuable intellectual labour. These agents will directly compete with human workers...

Published on April 11, 2025 1:27 PM GMTSummaryOpenAI recently released the Responses API. Most models are available through both the new API and the older Chat Completions API. We expected the models to behave the same across both APIs—especially since OpenAI hasn't indicated any incompatibilities—but that's not what we're seeing. In fact, in some cases, the differences are substantial. We suspect this issue is...

Published on April 11, 2025 1:09 PM GMTEpistemic status: I'm pretty sure the problem is somewhat interesting, because it temporarily confused several smart people. I'm not at all sure that it is very original; probably somebody has already thought about something similar. I'm not at all sure that I have actually found a flaw in UDT, but I somewhat expect that a discussion of...

Published on April 11, 2025 12:51 PM GMTGoogle Lays Out Its Safety Plans I want to start off by reiterating kudos to Google for actually laying out its safety plan. No matter how good the plan, it’s much better to write down and share the plan than it is to not share the plan, which in turn is much better than not having a...

Published on April 11, 2025 12:42 PM GMTDisclaimer: This is Kongo Landwalker's translation of lsusr's fiction Luna Lovegood and the Chamber of Secrets - Part 2 into russian. — Интересно, что тянет лодки? — сказала Луна.— Должно быть то же самое, что тянет и повозки без лошадей, — сказала первогодка.— Но... но... Можно ВИДЕТЬ лошадей тянущих повозки, но НЕ ВИДНО ничего, что тянуло бы лодки,...

Published on April 11, 2025 12:20 PM GMTPaper is good. Somehow, a blank page and a pen makes the universe open up before you. Why paper has this unique power is a mystery to me, but I think we should all stop trying to resist this reality and just accept it.Also, the world needs way more mundane blogging.So let me offer a few observations...

Published on April 11, 2025 9:41 AM GMTI am really not sure of why neuro-symbolic systems are considered as alternatives to the current black-box ones? A concrete example I have found (and currently studying) is HOUDINI (https://arxiv.org/pdf/1804.00218). Essentially, it implements neural networks using higher order combinators (map, fold etc.) that were found via enumeration/genetic programming searches. When the programs are found, the higher order combinators...

Published on April 10, 2025 5:39 PM GMTWhen I was a really small kid, one of my favorite activities was to try and dam up the creek in my backyard. I would carefully move rocks into high walls, pile up leaves, or try patching the holes with sand. The goal was just to see how high I could get the lake, knowing that if...

Published on April 10, 2025 5:15 PM GMTThis is cross-posted from my blog.We need more people on board for life extension in order to hit longevity escape velocity in our lifetimes. But most people have never heard of life extension, and even those who have often follow the same predictable arguments. “What if it doesn't work?” “What if bad people live forever?” “What if...

Published on April 10, 2025 5:13 PM GMTEpistemic status: Briefer and more to the point than my model of what is going on with LLMs, but also lower effort. Here is the paper. The main reaction I am talking about is AI 2027, but also basically every lesswrong take on AI 2027. A lot of people believe in very short AI timelines, say <2030. They want...

Published on April 10, 2025 4:31 PM GMTThis is part of the MIRI Single Author Series. Pieces in this series represent the beliefs and opinions of their named authors, and do not claim to speak for all of MIRI.Most human endeavors have bounded results. A construction project may result in a functional bridge or a deadly collapse, but even catastrophic failure will not kill...

Published on April 10, 2025 2:43 PM GMTEssentially all solutions in life are conditional: you apply them in the right context, in the right conditions to achieve a good outcome. Obviously banging a hammer on your deskdoes probably no good, while banging a hammer on a nail you want to use to hang that nice painting on the wall may be a great idea...

Published on April 10, 2025 2:37 PM GMTWhen complex systems fail, it is often because they have succumbed to what we call "disempowerment spirals" — self-reinforcing feedback loops where an initial threat progressively undermines the system's capacity to respond, leading to accelerating vulnerability and potential collapse.Consider a city gradually falling under the control of organized crime. The criminal organization doesn't simply overpower existing institutions...

Published on April 10, 2025 2:09 PM GMTPartially inspired by AI 2027, I've put to paper a day in one of the more optimistic scenarios I envision which are realistic to me.Discuss