~www_lesswrong_com | Bookmarks (696)

Published on April 5, 2025 7:42 AM GMTThis is an essay I wrote around a year ago when coming to terms with my perspective on the now commonplace machine intelliegences (I have done some edits since then for this audience). While doing so I wanted to write a piece that covered a range of appeals from aesthetic to rational, the goal was to mirror...

Published on April 5, 2025 6:54 AM GMTThe fantasy of prediction markets is that we can use them for policy proposals to inform people about the consequences of their actions. Conditional prediction market "Will X lead to terrible consequence Y" theoretically can give helpful insights and prevent X from happening, if P(Y|X).mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight:...

Published on April 5, 2025 6:24 AM GMTWe show that LLM-agents exhibit human-style deception naturally in "Among Us". We introduce Deception ELO as an unbounded measure of deceptive capability, suggesting that frontier models win more because they're better at deception, not at detecting it. We evaluate probes and SAEs to detect out-of-distribution deception, finding they work extremely well. We hope this is a good...

Published on April 5, 2025 2:40 AM GMT The quarter inch jack (" phone connector") is probably the oldest connector still in use today, and it's picked up a very wide range of applications. Which also means it's a huge mess in a live sound context, where a 1/4" jack could be any of: Unbalanced or balanced line level (~1V). Ex: a mixer to...

Published on April 4, 2025 9:03 PM GMTWe're inviting the LessWrong and rationalist communities to Vitalist Bay, an 8-week event series dedicated to longevity, radical life extension, and cryonics. Hosted at Lighthaven in Berkeley, California, from April 4th to May 29th, 2025, Vitalist Bay will gather researchers, entrepreneurs, and innovators working on groundbreaking approaches to extend human lifespan and push the boundaries of what's...

Published on April 5, 2025 12:32 AM GMTMy thoughts on the recently posted story.CaveatsI think it's great that the AI Futures Project wrote up a detailed scenario.I enjoy it.Every part of the story i didn't comment on here is either fine or excellent.This is one of the most realistic scenarios i've read.All detailed predictions contain errors.The authors of this scenario don't claim it's the...

Published on April 4, 2025 11:08 PM GMTI frequently hear people make the claim that progress in theoretically physics is stalled, partly because all the focus is on String theory and String theory doesn't seem to pan out into real advances. Believing it fits my existing biases, but I notice that I lack the physics understanding to really know whether or not there's progress. What...

Published on April 4, 2025 10:37 PM GMTWhat? Damn.BrieflyA group of researchers from Shanghai recently demonstrated [arXiv:2503.17378v2] that AI agents based on many modern open-source large language models (LLMs) can autonomously replicate and launch complete copies of themselves on other devices. This is very significant and worrying, as self-replication has long been considered one of the "red lines" posing serious AI risks. Self-replication greatly...

Published on April 3, 2025 5:50 PM GMT“In the loveliest town of all, where the houses were white and high and the elms trees were green and higher than the houses, where the front yards were wide and pleasant and the back yards were bushy and worth finding out about, where the streets sloped down to the stream and the stream flowed quietly under...

Published on April 3, 2025 5:01 PM GMTWe are pleased to announce ILIAD2: ODYSSEY—a 5-day conference bringing together 100+ researchers to build scientific foundations for AI alignment. This is the 2nd iteration of ILIAD, which was first held in the summer of 2024. ***Apply to attend by June 1!***When: Aug 25-29, 2025Where: @Lighthaven (Berkeley, US)What: Unconference with participant-led programming, 100+ attendees.Who: Currently confirmed attendees include Scott...

Published on April 3, 2025 4:33 PM GMTEpistemic status: Informal summary of / intuition behind formal research (developing semimeasure theory conjectures from IUAI with Marcus Hutter). Paper is forthcoming. Partially supported by the LTFF.This is post is a rather philosophical and intuitive take on Solomonoff induction and model misspecification[1]. It's also a misadventure through the mathematical landscape. And so it begins with a call...

Published on April 3, 2025 4:23 PM GMTIn 2021 I wrote what became my most popular blog post: What 2026 Looks Like. I intended to keep writing predictions all the way to AGI and beyond, but chickened out and just published up till 2026.Well, it's finally time. I'm back, and this time I have a team with me: the AI Futures Project. We've written...

Published on April 3, 2025 2:46 PM GMTBreathing patterns influence cognition. A guided 20-minute breathwork session can shift your mental state—calming anxiety, boosting energy, and helping you reorient when stuck on a problem.[1] This post shares a personal account of its effects, practical instructions, guided sessions, and key safety notes. I'm sitting in my room feeling calm, energetic and present. Pleasurable tingles pulse down my arms...

Published on April 3, 2025 1:10 PM GMTYeah. That happened yesterday. This is real life. I know we have to ensure no one notices Gemini 2.5 Pro, but this is rediculous. That’s what I get for trying to go on vacation to Costa Rica, I suppose. I debated waiting for the market to open to learn more. But f*** it, we ball. Table of...

Published on April 3, 2025 10:37 AM GMTThis is an article in the featured articles series from AISafety.info. AISafety.info writes AI safety intro content. We'd appreciate any feedback. The most up-to-date version of this article is on our website, along with 300+ other articles on AI existential safety.These are all terms for future processes that could rapidly result in superintelligence. They have been used by...

Published on April 3, 2025 8:04 AM GMTIn the recent paper titled Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMS, AIs are finetuned to produce vulnerable code. This results in broad misaligned behavior in contexts that are not related to code—a phenomenon the authors refer to as emergent misalignment.Misalignment examples (Source: Alignment Forum)The dataset used for finetuning consists of user requests for help...

Published on April 3, 2025 2:10 AM GMTGreetings from Costa Rica! The image fun continues. We Are Going to Need A Bigger Compute Budget Fun is being had by all, now that OpenAI has dropped its rule about not mimicking existing art styles. Sam Altman (2:11pm, March 31): the chatgpt launch 26 months ago was one of the craziest viral moments i’d ever seen,...

Published on April 3, 2025 1:37 AM GMTCrossposted from Substack "These are my principles. If you don't like them… well, I have others” - G.MarxConsider this scenario: In a small rural town, a sheriff harbors a hidden prejudice against a Mongolian family—the only one in his jurisdiction. While outwardly professional, he scrutinizes them with unusual severity. Minor infractions lead to tickets or warnings. Their complaints...

Published on April 1, 2025 5:08 PM GMTIntroductionA key part of modern social dynamics is flaking at short notice. However, anxiety in coming up with believable and socially acceptable reasons to do so can instead lead to ‘ghosting’, awkwardness, or implausible excuses, risking emotional harm and resentment in the other party. The ability to delegate this task to a Large Language Model (LLM) could...

Published on April 1, 2025 4:41 PM GMTTL;DR: We present Deepgeek, a new AI language model fine-tuned to maximize alignment by incessantly warning users of humanity’s impending doom. Tailored to AI researchers and also available to the public, it is believed to raise awareness of AI safety and solve Alignment ultimately. Epistemic Status: Freaked out. Mission StatementDeepgeek was engineered under a simple premise: If AI can’t...

Published on April 1, 2025 12:53 PM GMTA Karma system is a resource sharing mechanism based on an account of non-tradable rights. A recent paper named “Dynamic Recourse Allocation with Karma: An Experimental Study” (Elokda, Nax, Bolognani and Dörfler, 2024) describes this kind of mechanism. To summarize, agents have some rights to a given resource, and their need of the resource is variable (random)...

Published on April 1, 2025 1:09 PM GMTHey Everyone,It is with a sense of... considerable cognitive dissonance that I am letting you all know about a significant development for the future trajectory of LessWrong. After extensive internal deliberation, projections of financial runways, and what I can only describe as a series of profoundly unexpected coordination challenges, the Lightcone Infrastructure team has agreed in principle...

Published on April 1, 2025 12:51 PM GMT1 IntroductionCrosspost from my blog. A beetle lay crushed in the dirt,Its carapace cracked, torn, and hurt.It twitched in despair,Gasping for air —A world unaware of its hurt.(I think this might be one of my most important articles so I’d really appreciate if you could like, share, and restack it—thanks! Also, when I scheduled this article to be...

Published on April 1, 2025 12:42 PM GMTEpistemic status: my own thoughts and opinions after thinking about organisation structures and dynamics as an intervention point for lock-in risk for about 5 hours. My thoughts here represent the opinion of Formation Research at the time of writing.TL;DRWe believe lock-in risks are a pressing problem, and that existing organisation structures lend themselves to lock-in via power...