~www_lesswrong_com | Bookmarks (696)

Published on December 24, 2024 6:50 PM GMTEpistemic Status: This was composed in late 2017, sat editing limbo, was still worth publishing in 2021 and got some edits, and by 2023 was overdue for being published. Now, in 2024, it is almost just embarrassing that I haven't published it yet.The 2006 book "Data Analysis: A Bayesian Tutorial" by Devinderjit Sivia & John Skilling exposed...

Published on December 24, 2024 1:37 PM GMTI find that most communities I am part of, where AI discussions occur, fall into two categories: The community is too broad, and the discussion is fragmented. ACX would fall into this category. The AI discussion focuses on AGI/ASI and related topics (alignment, safety, how it will affect humanity, etc.). Lesswrong would fall into this category. I...

Published on December 24, 2024 1:00 PM GMTThis post goes over the important and excellent new paper from Anthropic and Redwood Research, with Ryan Greenblatt as lead author, Alignment Faking in Large Language Models. This is by far the best demonstration so far of the principle that AIs Will Increasingly Attempt Shenanigans. This was their announcement thread. New Anthropic research: Alignment faking in large...

Published on December 24, 2024 10:25 AM GMTTLDR: We're hosting a 3-month, fully-funded fellowship to do AI safety research drawing on inspiration from fields like evolutionary biology, neuroscience, dynamical systems theory, and more. Past fellows have been mentored by John Wentworth, Davidad, Abram Demski, Jan Kulveit and others, and gone on to work at places like Anthropic, AISI UK, Apart research, or as full-time PIBBSS...

Published on December 23, 2024 5:37 PM GMTThis essay was part of my application to UKAISI. Unclear how much is novel, but I like the decomposition.Some AI-enabled catastrophic risks involve an actor purposefully bringing about the negative outcome, as opposed to multi-agent tragedies where no single actor necessarily desired it. Such purposeful catastrophes can be broken down into a conjunction: they materialize if and only...

Published on December 23, 2024 3:44 PM GMTAs a holiday gift to myself, I put together a printable book of some creative writing by Scott Alexander & Eliezer. I'm sharing it here in case anyone else is interested :).It contains most of Scott's published fiction since he switched to Substack, plus some select creative non-fiction pieces (including a few from the old SSC days)....

Published on December 23, 2024 2:20 PM GMTI took a trip to San Francisco early in December. Ever since then, things in the world of AI have been utterly insane. Google and OpenAI released endless new products, including Google Flash 2.0 and o1. Redwood Research and Anthropic put out the most important alignment paper of the year, on the heels of Apollo’s report on...

Published on December 23, 2024 1:17 PM GMTtl;dr: The glitch tokens ' petertodd' and ' Leilan' were studied extensively in the context of GPT-3 before its decommissioning at the end of 2023 [1] [2]. Here, the conception of these two tokens and their relationship is studied for GPT-2, GPT-2-xl and GPT-J (which share the same token vocabulary as GPT-3). An appendix explores possible connections...

Published on December 23, 2024 9:38 AM GMTI'm seeing a lot of people on LW saying that they have very short timelines (say, five years or less) until AGI. However, the arguments that I've seen often seem to be just one of the following:"I'm not going to explain but I've thought about this a lot""People at companies like OpenAI, Anthropic etc. seem to believe...

Published on December 23, 2024 9:32 AM GMTThe AI should talk like a team of many AI. Each AI only uses the word "I" when referring to itself, and calls other AI in the team by their name. I argue that this may massively reduce Self-Allegiance by making it far more coherent for one AI to whistleblow or fight another AI which is unethical...

Published on December 23, 2024 8:51 AM GMTProject summaryAI Safety Camp has a seven-year track record of enabling participants to try their fit, find careers and start new orgs in AI Safety. We host up-and-coming researchers outside the Bay Area and London hubs.If this fundraiser passes…$15k, we won’t run a full program, but can facilitate 10 projects.$40k, we can organise the 11th edition, for...

Published on December 23, 2024 6:32 AM GMTThis is an article in the featured articles series from AISafety.info. AISafety.info writes AI safety intro content. We'd appreciate any feedback. The most up-to-date version of this article is on our website, along with 300+ other articles on AI existential safety.Compute governance is a type of AI governance that focuses on controlling access to the computing hardware needed...

Published on December 23, 2024 5:16 AM GMTEpistemic Status: Seven years and one day ago, I wrote this. In 2017 we faced years of Trump and couldn't imagine covid. Here in 2024 we face years of Trump and I'm trying to "imagine what we can't imagine". I dislike the title (except for the part where it might introduce people to music from a previous...

Published on December 23, 2024 3:58 AM GMTOf the posts I've delayed writing for years, I maybe regret this one the most.I think more people (x-risk focused people in particular) should consider becoming (and hiring) metacognitive assistants. This is the single largest performance boost I know of – hire someone to sit with you and help you think. It doesn't help me (much) when...

Published on December 21, 2024 5:17 PM GMTI used to think that all generic strategy advice was pointless. After all, the point of a strategy is to achieve a thing, and to achieve a thing you just think hard about how to best do it and then work hard to do it. I said this to my friend Dewi, who said that this is...

Published on December 21, 2024 8:28 AM GMTEpistemic status: We are moderately confident in the feasibility of serially filtered shelters for 14-log protection as well as cost estimates, but more uncertain about how mirror biology might develop in the real world.Originally published on the EA Forum. Minor edits for LW adaptation.Please sign up here for ~bi-monthly updates on shelter/refuges work.Summary and main take-awaysI want to be...

Published on December 21, 2024 2:55 AM GMT2024 is the year it became clear that we're actually going to do battery energy storage. That it was time to stop soberly reminding anyone too excited about solar that storage is an unsolved problem. In fact I personally became very excited about solar, and briefly thought that this year's wave of battery installations was just the...

Published on December 21, 2024 1:30 AM GMT I originally wrote this a year ago, but just now found it in my drafts. Not sure why I didn't post it then. One flavor of response I got with my post on deanonymizing accounts was roughly: Why not just go ahead and post the list of alts? It's not like people sneakily using multiple accounts...

Published on December 21, 2024 1:19 AM GMTHere is an intuitively compelling principle: hearing a bad argument for a view shouldn’t change your degree of belief in the view. After all, it is possible for bad arguments to be offered for anything, even the truth. For all you know, plenty of good arguments exist, and you just happened to hear a bad one.But this...

Published on December 21, 2024 12:15 AM GMT7000 MMLU questions visualized with bird's eye viewHow do you make sense of a 300-page book in 15 minutes? Or get insights from thousands of news articles without relying on recommendation algorithms? Our usual approach is to filter: we use search engines, summaries, or recommendations to select a manageable subset of content. But what if we could...

Published on December 20, 2024 10:50 PM GMTI can't find the videos where they said this, but I remember Buffet and Soros rationalizing their choice of profession by saying that they make market prices more informative. Is there a way to quantify that? What units would we use? Could we say that Buffet added $100 billion of "liquidity" to markets over the course of...

Published on December 20, 2024 11:58 PM GMT"What is the use of having developed a science well enough to make predictions if, in the end, all we're willing to do is stand around and wait for them to come true?" F. SHERWOOD HOWLAND in his speech accepting the Nobel Prize in Chemistry in 1995. "Once upon a time on Tralfamadore there were creatures who weren’t...