~www_lesswrong_com | Bookmarks (706)

Published on April 3, 2025 10:37 AM GMTThis is an article in the featured articles series from AISafety.info. AISafety.info writes AI safety intro content. We'd appreciate any feedback. The most up-to-date version of this article is on our website, along with 300+ other articles on AI existential safety.These are all terms for future processes that could rapidly result in superintelligence. They have been used by...

Published on April 3, 2025 8:04 AM GMTIn the recent paper titled Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMS, AIs are finetuned to produce vulnerable code. This results in broad misaligned behavior in contexts that are not related to code—a phenomenon the authors refer to as emergent misalignment.Misalignment examples (Source: Alignment Forum)The dataset used for finetuning consists of user requests for help...

Published on April 3, 2025 2:10 AM GMTGreetings from Costa Rica! The image fun continues. We Are Going to Need A Bigger Compute Budget Fun is being had by all, now that OpenAI has dropped its rule about not mimicking existing art styles. Sam Altman (2:11pm, March 31): the chatgpt launch 26 months ago was one of the craziest viral moments i’d ever seen,...

Published on April 3, 2025 1:37 AM GMTCrossposted from Substack "These are my principles. If you don't like them… well, I have others” - G.MarxConsider this scenario: In a small rural town, a sheriff harbors a hidden prejudice against a Mongolian family—the only one in his jurisdiction. While outwardly professional, he scrutinizes them with unusual severity. Minor infractions lead to tickets or warnings. Their complaints...

Published on April 1, 2025 5:08 PM GMTIntroductionA key part of modern social dynamics is flaking at short notice. However, anxiety in coming up with believable and socially acceptable reasons to do so can instead lead to ‘ghosting’, awkwardness, or implausible excuses, risking emotional harm and resentment in the other party. The ability to delegate this task to a Large Language Model (LLM) could...

Published on April 1, 2025 4:41 PM GMTTL;DR: We present Deepgeek, a new AI language model fine-tuned to maximize alignment by incessantly warning users of humanity’s impending doom. Tailored to AI researchers and also available to the public, it is believed to raise awareness of AI safety and solve Alignment ultimately. Epistemic Status: Freaked out. Mission StatementDeepgeek was engineered under a simple premise: If AI can’t...

Published on April 1, 2025 12:53 PM GMTA Karma system is a resource sharing mechanism based on an account of non-tradable rights. A recent paper named “Dynamic Recourse Allocation with Karma: An Experimental Study” (Elokda, Nax, Bolognani and Dörfler, 2024) describes this kind of mechanism. To summarize, agents have some rights to a given resource, and their need of the resource is variable (random)...

Published on April 1, 2025 1:09 PM GMTHey Everyone,It is with a sense of... considerable cognitive dissonance that I am letting you all know about a significant development for the future trajectory of LessWrong. After extensive internal deliberation, projections of financial runways, and what I can only describe as a series of profoundly unexpected coordination challenges, the Lightcone Infrastructure team has agreed in principle...

Published on April 1, 2025 12:51 PM GMT1 IntroductionCrosspost from my blog. A beetle lay crushed in the dirt,Its carapace cracked, torn, and hurt.It twitched in despair,Gasping for air —A world unaware of its hurt.(I think this might be one of my most important articles so I’d really appreciate if you could like, share, and restack it—thanks! Also, when I scheduled this article to be...

Published on April 1, 2025 12:42 PM GMTEpistemic status: my own thoughts and opinions after thinking about organisation structures and dynamics as an intervention point for lock-in risk for about 5 hours. My thoughts here represent the opinion of Formation Research at the time of writing.TL;DRWe believe lock-in risks are a pressing problem, and that existing organisation structures lend themselves to lock-in via power...

Published on April 1, 2025 10:39 AM GMT(Not an April fools' joke)If anyone wants to return to one of Keltham's lectures, here's a list with links to all the lectures. In principle, one could also read these without having read the rest of project lawful, I am not sure how easy it would be to follow though. I included short excerpts, so you can...

Published on April 1, 2025 10:32 AM GMTOut of all the research rabbit holes I've ever gone down, this one is by far my favorite, as to most people at first glance it's so unthinkable an idea.Years ago, I would have been right there with you in disbelief, but the reasons why turned out to be a kind of perfect combination of counterfactuals that...

Published on April 1, 2025 8:22 AM GMTFor more than five years, I've posted an average of more than 1× per week on Less Wrong. I've learned a lot from you nerds. I've made friends and found my community. Thank you for pointing out all the different ways I've been wrong. Less Wrong has changed my life for the better. But it's time to...

Published on April 1, 2025 7:12 AM GMTEpistemic status - statistically verified. I'm writing this post to draw peoples' attention to a new cause area proposal - haircuts for alignment researchers.Aside from the obvious benefits implied by the graph (i.e. haircuts having the potential to directly improve organizational stability), this would represent possibly the only pre-singularity opportunity we'll have to found an org named "Clippy".I...

Published on March 30, 2025 6:06 PM GMTtl;dr:From my current understanding, one of the following two things should be happening and I would like to understand why it doesn’t:EitherEveryone in AI Safety who thinks slowing down AI is currently broadly a good idea should publicly support PauseAI.OrIf pausing AI is much more popular than the organization PauseAI, that is a problem that should be...

Published on March 30, 2025 4:58 PM GMTIntroductionResearch on Sparse Autoencoders (SAEs) has identified "known entity" features in language models - features that activate when the model processes entities it "knows." If we can find the circuit models use to recognise that they know an entity, then by computing which inputs would trigger the circuit, we can extract a list of "known entities".The aim...

Published on March 30, 2025 1:16 PM GMTNote: I created an entry highlighting the formal attack on qualia presented in: I, G(Zombie)Introducing Eliminative Nominalism (EN), a novel position in the philosophy of mind that extends and critiques Eliminative Materialism by rejecting not only mental states, but also the ontological assumptions embedded in both physicalist and emergentist accounts of consciousness. Drawing from proof theory, cybernetics, and...

Published on March 30, 2025 7:16 AM GMTIn neuroscience, we learned about foveated vision — our eyes focus sharply at the center and blur the edges to conserve energy.I was curious how LLMs handle that kind of input, so I wrote a quick script to apply a foveated blur to 30 CAPTCHA images of cars, with the car in different regions each time.At first,...

Published on March 30, 2025 6:54 AM GMTNow and then, at work, we’ll have a CEO on from the megacorp that owns the company I work at. It’s a Zoom meeting with like 300 people, the guy is usually giving a speech that is harmless and nice (if a bit banal), and I’ll turn on my camera and ask a question about something that...

Published on March 30, 2025 1:24 AM GMTThere is no such thing as philosophy-free science; there is only science whose philosophical baggage is taken on board without examination.— Daniel Dennett, Darwin's Dangerous Idea (1995) INTRODUCTIONPrefaceThis document is shared publicly as an initial draft to invite constructive feedback, insights, and reflections.AbstractIntroducing Eliminative Nominalism (EN), a novel position in the philosophy of mind that extends and critiques Eliminative Materialism...

Published on March 29, 2025 7:08 PM GMTOr: Why thinking about blue tentacle arms is not always a waste of time.0. IntroductionAs I work through the Sequences, I find myself disagreeing—slightly—with a point Eliezer Yudkowsky makes in A Technical Explanation of Technical Explanation:Imagine that you wake up one morning and your left arm has been replaced by a blue tentacle. The blue tentacle obeys...

Published on March 29, 2025 8:37 PM GMTA better anything can be achieved with simple tests at low costs.BackgroundPeople often settle for "good enough" and "if it ain't broke don't fix it" in their personal lives, opting not to make any effort to improve said things because either:Time, money, and/or effort can be spent elsewhere for a higher expected valueThey think it can't be...

Published on March 29, 2025 7:38 PM GMTFor the purposes of FOOM, I'm defining it as a situation in which once an AI is capable enough to automate away all AI R&D, progress starts exploding hyper-exponentially for a period because the returns to better software is larger than 1, meaning AI labor quality is improving faster than the problem of finding new algorithms gets...

Published on March 29, 2025 6:11 PM GMTSuppose you’re an AI researcher trying to make AIs which are conscious and reliably moral, so they’re trustworthy and safe for release into the real world, in whatever capacity you intend.You can’t, or don’t want to manually create them; it’s more economical, and the only way to ensure they’re conscious, if you procedurally generate them along with...