~www_lesswrong_com | Bookmarks (687)

Published on October 25, 2024 7:37 PM GMTTL;DRI spent a few hours going through Open Philanthropy (OP)'s grant database. The main findings were:Open Philanthropy has made $28 million grants for Technical AI Safety (TAIS) in 202468% of these are focused on evaluations / benchmarking. The rest is split between interpretability, robustness, value alignment, forecasting, field building and other approaches.OP funding for TAIS has fallen...

Published on October 25, 2024 6:00 PM GMTWhat labs should doThe table/list in Towards best practices in AGI safety and governance: A survey of expert opinion (GovAI: Schuett et al. 2023): a collection of many briefly-described safety practicesResponsible Scaling Policies and Key Components of an RSP (METR 2023)Model evals for dangerous capabilities (Stein-Perlman 2024): just the summarySafety cases (arguments that an AI system is safe to...

Published on October 24, 2024 9:08 PM GMTSentience is the capacity to experience anything – the fact that when your brain is thinking or processing visual information, you actually feel what it’s like to think and to see.The following thought experiment demonstrates, step by step, why conscious experience must be a product of functional patterns, not any specific physical material or structure.The Inevitable Self-Report...

Published on October 25, 2024 11:06 AM GMTThe EU and UK's Network and Information Systems (NIS) Regulations aim to improve the cybersecurity of essential services and important digital providers. They set requirements around network security, information system security, physical security, incident handling, business continuity and security auditing.I think OpenAI needs to comply with these regulations. This article sets out why, and what the implications...

Published on October 25, 2024 4:37 AM GMT Memorandum on Advancing the United States’ Leadership in Artificial Intelligence; Harnessing Artificial Intelligence to Fulfill National Security Objectives; and Fostering the Safety, Security, and Trustworthiness of Artificial Intelligence This memorandum fulfills the directive set forth in subsection 4.8 of Executive Order 14110 of October 30, 2023 (Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence)....

Published on October 24, 2024 11:10 PM GMTI am often tempted to give advice, and find it difficult to say nothing while friends and colleagues make decisions that seem (to me) obviously wrong.But advice is by nature bad, and it is not hard to see why. We are different people living under different circumstances. Giving advice is not only unhelpful, it is often harmful....

Published on October 25, 2024 12:10 AM GMT A few weeks ago I posted about how I was thinking about a pedalboard. I've now made one! Before: After: It folds up slightly and fits in a standard rolly suitcase: Not pictured: I later added padding on all six sides. Here's the pedalboard by itself: You can see the bundle of wires to the left...

Published on October 24, 2024 8:50 PM GMTHuman capital trades more like real estate than equities. While millions in S&P 500 stocks can exchange hands with a single click, hiring employees or finding life partners always requires a final offline, high-touch evaluation - much like inspecting a house before closing the deal.In the job market, some predictive attributes can be translated into text -...

Published on October 24, 2024 6:36 PM GMT[Linkpost. This presents a way to talk to people about AI that hold overly-romanticized views built on fictional evidence.]MRSA (Methicillin-resistant Staphylococcus aureus) is a type of staph bacteria that’s become resistant to many of the antibiotics we use. The original staph bacteria that is the ancestor of MRSA was first identified in 1880, but has likely existed...

Published on October 23, 2024 6:10 PM GMTWe are launching an independent research lab that builds open, scalable technology for understanding AI systems and steering them in the public interest.Transluce means to shine light through something to reveal its structure. Today’s complex AI systems are difficult to understand—not even experts can reliably predict their behavior once deployed. At the same time, AI is being...

Published on October 23, 2024 3:50 PM GMTThis post includes a "flattened version" of an interactive diagram that cannot be displayed on this site. I recommend reading the original version of the post with the interactive diagram, which can be found here. Over the last few months, ARC has released a number of pieces of research. While some of these can be independently motivated,...

Published on October 23, 2024 2:31 PM GMTAI is a very controversial topic these days, but I can't help but ask, what's the deal with OI[1], does it have any powerful effects on the future, and can it compete with AI?^Organoid Intelligence is, I quote "an emerging multidisciplinary field working to develop biological computing using 3D cultures of human brain cells (brain organoids) and...

Published on October 23, 2024 12:18 PM GMTTwo important themes in many discussions of the future of AI are:AI will automate research, and thus accelerate technological progressThere are serious risks from misaligned AI systems (that justify serious investments in safety)How do these two themes interact? Especially: how should we expect the safety tax requirements to play out as progress accelerates and we see an...

Published on October 23, 2024 6:01 AM GMTHey LessWrong,After sharing early results at Less Online this summer (thanks for the feedback during the strength training demos), I'm recruiting for a LessWrong team in the 2025 Unaging System Challenge. What started as helping one friend who got winded climbing stairs has grown into a complete system that could add two decades to our lives. The...

Published on October 23, 2024 5:39 AM GMTI've written a lot of words—hundreds of blog posts, thousands of comments, tens of thousands of emails, and hundreds of thousands of short messages. I'm even written most of a book! In total, between personal and professional writing, I estimate to have put north of 3 million words into text. So it may come as a surprise...

Published on October 22, 2024 9:57 PM GMTRecently the whole "if your p(doom) is high, you should short the market" thing has been going around. Let's say the market prices in a 0.1% chance of extinction per century, and you think there's a 50% chance we're dead in the next 25 years. This is about 12 bits of evidence. How much profit is it...

Published on October 22, 2024 8:57 PM GMTThanks to Holden Karnofsky, David Duvenaud, and Kate Woolverton for useful discussions and feedback.Following up on our recent “Sabotage Evaluations for Frontier Models” paper, I wanted to share more of my personal thoughts on why I think catastrophic sabotage is important and why I care about it as a threat model. Note that this isn’t in any...

Published on October 22, 2024 6:20 PM GMT~5 months I formally quit EA (formally here means “I made an announcement on Facebook”). My friend Timothy was very curious as to why; I felt my reasons applied to him as well. This disagreement eventually led to a podcast episode, where he and I try convince each other to change sides on Effective Altruism- he tries...

Published on October 21, 2024 5:56 PM GMTHere I define autonomy as not having your insides controlled by your outsides. Complete autonomy is impossible, of course. This is the lim→∞.For example, imagine a bacterium. It has insides: its guts. It has outsides: its environment.And in between, it has a membrane. The membrane is the thing that attempts to control what comes in and what goes out. For example, pores can guide nutrients...

Published on October 21, 2024 5:10 PM GMTI'm an advocate of something known as sortition. The premise is simple. Choose people at random, to serve a finite term, in some decision making capacity. Pay them to be there. Sounds ridiculous, right?  How could we possibly trust ignorant, stupid, normal people to make good decisions? What would this look like? Why would this be better than...