~www_lesswrong_com | Bookmarks (703)

Published on June 1, 2025 8:18 AM GMTIn a previous article I detailed FSU Law professor Nadia Batenka's proposed "Inverted Sliding Scale Framework" approach to the question of legal personhood for digital intelligences. In this article, I am going to examine another paper approaching the issue which was first authored by Claudio Novelli, Giorgio Bongiovanni, and Giovanni Sartor. Since its original writing it has been...

Published on May 31, 2025 10:04 PM GMTI study Economics and Data Science at the University of Pennsylvania. I used o1-pro, o3, and Gemini Deep Research to expand on my ideas with examples, but have read and edited the paper to highlight my understanding improved on by AI. I. The AI Labor Debate: Beyond "Robots Taking Our Jobs"The Prevailing Narrative: Supply-Side AutomationThe discourse surrounding artificial...

Published on May 31, 2025 9:17 PM GMTAdapted from this twitter thread. See this as a quick take.Mitigation StrategiesHow to mitigate Scheming?Architectural choices: ex-ante mitigationControl systems: post-hoc containmentWhite box techniques: post-hoc detectionBlack box techniquesAvoiding sandbaggingWe can combine all of those mitigation via defense-in-depth system (like the Swiss Cheese model below)I think that applying all of those strategies should divide the risk by at least...

Published on May 31, 2025 6:20 PM GMTThere have recently been various proposals for mitigations to "the intelligence curse" or "gradual disempowerment"—concerns that most humans would end up disempowered (or even dying) because their labor is no longer valuable. I'm currently skeptical that the typically highlighted prioritization and interventions are best and I have some alternative proposals for relatively targeted/differential interventions which I think would be more...

Published on May 31, 2025 4:30 PM GMTThere’s a story — I'm at a conference and cannot access my library, but I believe it comes from Structural Anthropology by Claude Lévi-Strauss — about an anthropologist who took a member of an Amazonian tribe to New York.The man was overwhelmed by the city, but there were two things that particularly interested him.One was the bearded...

Published on May 31, 2025 4:18 PM GMTRational Animations takes a look at Tom Davidson's Takeoff Speeds model (https://takeoffspeeds.com). The model uses formulas from economics to answer two questions: how long do we have until AI automates 100% of human cognitive labor, and how fast will that transition happen? The primary scriptwriter was Allen Liu (the first author of this post), with feedback from...

Published on May 30, 2025 4:57 PM GMTThese are the most significant pieces of life advice/wisdom that have benefited me, that others often don't think about and I end up sharing very frequently. I wanted to share this here for a few reasons: First and most important, I think many people here will find this interesting and beneficial. I am very proud of this post...

Published on May 30, 2025 2:11 PM GMTStatus: musings. I wanted to write up a more fleshed-out and rigorous version of this, but realistically wasn't likely to every get around to it, so here's the half-baked version. Related posts: Firming Up Honesty Around Its Edge-Cases, Deep honesty What I mean by 'honesty' There are nuances to this, but I think a good summary is...

Published on May 30, 2025 1:46 PM GMTI envision a future more chaotic than portrayed in AI 2027. My scenario, the Rogue Replication Timeline (RRT), branches off mid-2026. If you haven’t already read the AI 2027 scenario, I recommend doing so before continuing.You can read the full version on my blog, but I have copied the summary and first half of the scenario below.This...

Published on May 30, 2025 10:50 AM GMTLetting kids be kids seems more and more important to me over time. Our safetyism and paranoia about children is catastrophic on way more levels than most people realize. I believe all these effects are very large: It raises the time, money and experiential costs of having children so much that many choose not to have children,...

Published on May 30, 2025 1:21 AM GMTThe Geometry of LLM Logits (an analytical outer bound) 1 Preliminaries Symbol Meaning d.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px...

Published on May 30, 2025 12:23 AM GMTFrom June 2-6, the Center for Applied Rationality will be running an experimental mini-workshop at Arbor Summer Camp. This is an in-person event held in Berkeley, CA at the Lighthaven campus and requires paid admission -- read on for more details.This workshop is "mini"This workshop will last for about three days (two half-days on the edges, two...

Published on May 29, 2025 10:02 PM GMTHello from the Center for Applied Rationality!Some of you may have attended our classic applied rationality workshops in the past; others of you may have wanted to attend a workshop but not yet had a chance to. It's been a while since we've last run public-facing workshops, but we wanted to say: We're not dead! (More on that...

Published on May 29, 2025 9:42 PM GMTIn previous posts in this sequence, I laid out a case for why most AI governance research is too academic and too abstract to have much influence over the future. Politics is noisy and contested, so we can’t expect that good AI governance ideas will spread on their own – we need a large team of people...

Published on May 29, 2025 6:55 PM GMTThis post benefitted greatly from comments, suggestions, and ongoing discussions with David Duvenaud, David Krueger, and Jan Kulveit. All errors are my own.A few months ago, I and my coauthors published Gradual Disempowerment (GD hereafter). It was mostly about how things might go wrong, but naturally a lot of the resulting interest has been about solutions. We have some...

Published on May 29, 2025 6:49 PM GMTEveryone around me has a notable lack of system prompt. And when they do have a system prompt, it’s either the eigenprompt or some half-assed 3-paragraph attempt at telling the AI to “include less bullshit”.I see no systematic attempts at making a good one anywhere.[1](For clarity, a system prompt is a bit of text—that's a subcategory of "preset"...

Published on May 28, 2025 6:30 PM GMTSince Claude 4 Opus things have been refreshingly quiet. Video break! The First Good AI Videos First up we have Prompt Theory, made with Veo 3, which I am considering the first legitimately good AI-generated video I’ve seen. It perfectly combining form and function. Makes you think. Here’s a variant, to up the stakes a bit, then...

Published on May 28, 2025 4:19 PM GMTIntroductionI have long been very interested in the limitations of LLMs because understanding them seems to be the most important step to getting timelines right. Right now there seems to be great uncertainty about timelines, with very short timelines becoming plausible, but also staying hotly contested. This led me to revisit LLM limitations and I think I noticed a...

Published on May 28, 2025 4:03 PM GMTOkay, I think it's time to stop polishing this playlist & post it. This is music about the moods of people I met at Manifest. I'm pretty delighted with it! Discuss

Published on May 28, 2025 4:00 PM GMTWelcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.In this edition: Google released a frontier video generation model at its annual developer conference; Anthropic’s Claude Opus 4 demonstrates the danger of relying on voluntary governance.Listen to the AI Safety Newsletter for free...

Published on May 28, 2025 12:57 PM GMTAn interim research reportSummaryWe introduce a novel methodology for quantitatively evaluating metacognitive abilities in LLMsWe present evidence that some frontier LLMs introduced since early 2024 - but not older or smaller ones - show some metacognitive abilitiesThe metacognitive abilities that current LLMs do show are relatively weak, and manifest in a context-dependent manner; the models often prefer...

Published on May 28, 2025 11:42 AM GMTDuring a visit to a Hong Kong children’s welfare home, I met a 12-year-old girl I'll call Kylie. She had suffered a severe illness that left her blind, deaf, non-verbal, and nearly immobile, yet no identified damage was done to her brain.The staff described her, without hesitation, as “always cheerful,” and indeed she smiled the entire time...

Published on May 28, 2025 10:50 AM GMTThe following analogy is intended to illustrate a novel proof-theoretic concept. It is metaphorical in nature and should not be interpreted literally.The Vanishing SkyConsider the following analogy:In a universe undergoing accelerating expansion, distant galaxies gradually slip beyond our horizon, until the point at which their light can no longer reach us. These galaxies do not disappear in...

Published on May 28, 2025 9:30 AM GMTContext: This is a linkpost for https://aisafety.info/questions/NM3I/6:-AI%E2%80%99s-goals-may-not-match-ours This is an article in the new intro to AI safety series from AISafety.info. We'd appreciate any feedback. The most up-to-date version of this article is on our website. Making AI goals match our intentions is called the alignment problem.There’s some ambiguity in the term “alignment”. For example, when people talk about...