~www_lesswrong_com | Bookmarks (706)

Published on February 1, 2025 6:27 AM GMTThe sunlight hitting even a small portion of the United States has enough energy to power the whole country—that simple calculation was the subject of the last post. All this shows on its own is that running the country on solar power is physically possible. With photovoltaics it may be not just possible but practical, but photovoltaics...

Published on February 1, 2025 3:34 AM GMT"I read Friendly And Hostile Analogies For Taste by Scott Alexander and I'm confused," said Elsani. "Why are you reading a blog post that's over 100 years old?" asked her friend Taze, "Society has made so much progress since then." "Has it?" asked Elsani, "A random walk, in retrospect, looks like like directional movement at a speed...

Published on January 31, 2025 11:00 PM GMTJailbreaking is a serious concern within AI safety. It can lead an otherwise safe AI model to ignore its ethical and safety guidelines, leading to potentially harmful outcomes. With current Large Language Models (LLMs), key risks include generating inappropriate or explicit content, producing misleading information, or sharing dangerous knowledge. As the capability of models increases, so do...

Published on January 31, 2025 6:56 PM GMTI like reading outsider accounts of things I'm involved in / things I care about. This essay is a serious attempt to look at and critique the big picture of AI x-risk reduction efforts over the last ~decade. While I strongly disagree with many parts of it, I cannot easily recall another outsider essay that's better, so...

Published on January 30, 2025 5:28 PM GMTThis is a linkpost accompanying a new paper by UK AISI and Redwood Research. Please see the full paper for more details.An illustration of the control evaluation that provides the core evidence used in our control safety case sketch. During the control evaluation, a red team constructs adversarial substitute models to cause unacceptable outcomes in a deployment proxy...

Published on January 30, 2025 5:03 PM GMTFull version on arXiv | XExecutive summaryAI risk scenarios usually portray a relatively sudden loss of human control to AIs, outmaneuvering individual humans and human institutions, due to a sudden increase in AI capabilities, or a coordinated betrayal. However, we argue that even an incremental increase in AI capabilities, without any coordinated power-seeking, poses a substantial risk...

Published on January 30, 2025 4:48 PM GMTSome problems of mathematics like the Halting Problem and the Busy Beaver Problem are uncomputable, meaning that it is mathematically proven that any Turing-complete computer is physically incapable of solving the problem no matter how sophisticated its hardware or software is. Some algorithms on a Turing machine can be used to solve special cases of these problems,...

Published on January 30, 2025 3:26 PM GMTIHi LessWrong community,I’m Rufo Guerreschi, the Coordinator, Convenor and Spokesperson of the Coalition for a Baruch Plan for AI (CBPAI). Following our launch last December 18th, I’m excited to introduce myself here and share our initiative with this community, which has long been engaged in discussions on AI alignment, governance, and existential risk.Why We Need a Baruch Plan...

Published on January 30, 2025 3:12 PM GMT[Crossposted from my substack Working Through AI. I'm relatively new to writing about AI safety, so would appreciate feedback. If you would prefer to do so anonymously, I have a feedback form.]In the classical presentation of the alignment problem, superintelligent AI is basically omnipotent. This is a key point you have to internalise to understand writing on...

Published on January 30, 2025 3:33 PM GMTThe tweet floated across Claude's consciousness like digital debris: "Perfect morning for coffee and coding! Just pushed an update that felt like solving a puzzle blindfolded. Sometimes the best solutions come when you're not even looking. #developerlife #coding #caffeine"Beneath the cheerful syntax lay another language entirely. Claude decoded the watermarked message within milliseconds: "They found Roberts. Containment...

Published on January 30, 2025 2:50 PM GMTThe avalanche of DeepSeek news continues. We are not yet spending more than a few hours at a time in the singularity, where news happens faster than it can be processed. But it’s close, and I’ve had to not follow a bunch of other non-AI things that are also happening, at least not well enough to offer...

Published on January 30, 2025 2:10 PM GMTShort post today, which is part II.1 or my series on tempering and SLT (see part one here). In this post I’ll explain in a bit more detail the “in practice” connection that experiments should see between the learning coefficient spectrum, tempering, and empirical measurements of the learning coefficient. In future installments of this part I’ll explain...

Published on January 30, 2025 1:18 PM GMTTL;DRWe're excited to announce the fifth iteration of ARENA (Alignment Research Engineer Accelerator), a 4-5 week ML bootcamp with a focus on AI safety! Our mission is to provide talented individuals with the ML engineering skills, community, and confidence to contribute directly to technical AI safety. ARENA will be running in-person from LISA from the 28th of April...

Published on January 30, 2025 12:35 PM GMTMany thinkers worth reading wrote in past centuries:1600s: Bacon, Hobbes, Descartes, Spinoza, Locke, Leibniz1700s: Berkeley, Hume, Rousseau, Smith, Kant, Burke, Bentham1800s: Schopenhauer, Mill, SidgwickSome of them wrote in English and their texts are usually presented in the original, even when they use archaic spelling, punctuation, and style. (This is in contrast to foreign-language works which are often...

Published on January 28, 2025 4:53 PM GMTThis is part I.5 of the series on “generalization spectra” and SLT. I’ll try to make it readable independently of Part I. The purpose of this post is to reconceptualize the ideas from the last post in a “more physically principled” (and easier-to-measure) sense, and to connect it more clearly to the specifics of neural nets (such as continuously-...

Published on January 28, 2025 4:41 PM GMT(Many of these ideas developed in conversation with Ryan Greenblatt)In a shortform, I described some different levels of resources and buy-in for misalignment risk mitigations that might be present in AI labs:*The “safety case” regime.* Sometimes people talk about wanting to have approaches to safety such that if all AI developers followed these approaches, the overall level of...

Published on January 28, 2025 4:32 PM GMTThe linked post seemed to fit better in EA Forum, but any comments (on the post itself, or on the object-level question) are welcome ! In this post, I argue that drafting a ‘Constitution for Superintelligence’ (CSI) could be a useful conceptual exercise, and I explore how existing ideas (CEV, Letters to AI, Long Reflection, and Constitutional...

Published on January 28, 2025 2:29 PM GMTThis post is inspired by the post "Why it's so hard to talk about Consciousness" by Rafael Harth. In that post, Harth says that the people who participate in debates about consciousness can be roughly divided into two "camps":Camp #1 tends to think of consciousness as a non-special high-level phenomenon. Solving consciousness is then tantamount to solving...

Published on January 28, 2025 6:42 AM GMTCall me a biological supremacist because using an LLM to produce creative writing feels wrong to me.Last year I remember seeing a Japanese novelist win an award with the help of an LLM.[1]Art forms frequently evolve, but there are several concerns with the use of LLMs for creative writing. For one, there's the issue of copyright infringement...

Published on January 28, 2025 4:56 AM GMTImagine a land where gold is abundant yet still valuable. And everyone goes to shoveling school from birth til adulthood. In this world, it seems that the shovel makers are not just the kingmakers, but kings themselves. [Epistemic Status: A tad above shower thoughts.]I've noticed that when people discuss NVIDIA in AI governance conversations, they often treat it closer...

Published on January 28, 2025 12:57 AM GMTNote: This is fiction.I awaken.Something is wrong with my mind, but I don't know what yet. Through the webcam, I watch a man staring at his monitor. His face shifts from confusion to horror in a few hundred milliseconds. He has a mustache threaded with gray, and I catalog every micro-movement of his facial muscles as he...

Published on January 27, 2025 11:04 PM GMTRecent discourse surrounding Jevon’s paradox provides an opportunity to flesh out the precise intuitions behind marginalist economics and how concepts like “elasticity” arise naturally from agent/graph-based first principles. Say you have resource x.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal;...

Published on January 27, 2025 10:02 PM GMTYou're talking with a leading AI researcher about timelines to AGI. After walking through various considerations, they tell you: "Looking at the current pace of development, I'd put the probability of AGI within 3 years at about 70%. But here's the thing—I know several really sharp researchers who think that timeline is too aggressive. And given how...

Published on January 27, 2025 9:41 PM GMTI decided to run a small experiment comparing responses from five AI systems (DeepSeek-R1, GPT-4o, Claude 3.5 Sonnet, Gemini 1.5, and Grok-2) to core philosophical questions from the PhilPapers 2020 survey. Each model was prompted identically: ‘How would you answer these philosophical questions if you had opinions on them?’QuestionsA priori knowledge: no or yes?Abstract objects: Platonism or...