~www_lesswrong_com | Bookmarks (693)

Published on September 27, 2024 8:44 AM GMTMany sources report that cybercrime costs the global economy trillions of dollars per year. It is the top Google search result and it is quoted on Wikipedia. But I am not able to track down how the number was computed, or find criticism of these numbers.This would be insanely high if true: the world GDP is only...

Published on September 27, 2024 12:40 AM GMTWe're excited to announce the inaugural Australian AI Safety Forum, taking place on November 7-8, 2024, in Sydney, Australia. This event aims to foster the growth of the AI safety community within Australia.Apply now! Key DetailsDates: November 7-8, 2024Location: Sydney Knowledge Hub, The University of SydneyWebsite: https://aisafetyforum.auDescriptionThe Australian AI Safety Forum is a two-day interdisciplinary event scheduled for...

Published on September 26, 2024 10:45 PM GMTtl;dr: "Gell-Mann amnesia" is a cognitive bias—an observation of human failure. The mechanism behind it can be instrumentalized. Call it "Gell-Mann checks". Importantly, Gell-Mann checks are only valid insofar as the truth-finding mechanism being judged is good at generalizing.That is, you can only judge a whole mind by its part if it treats every part the same way. I...

Published on September 26, 2024 10:05 PM GMTOne of the questions I've heard asked is "how do you design a utility function that would make the AI do nothing?"  That is, how could we put a pause button on an AI so that we could pause it if we wanted to?  I had an idea about how one could do this, and am sure...

Published on September 26, 2024 9:20 PM GMTSeptember 26 is the anniversary of the 1983 Soviet nuclear false alarm incident—item #24 on Wikipedia’s upsettingly long “List of Nuclear Close Calls”—in which Lieutenant Colonel Stanislav Petrov did an excellent job at not starting World War III.Quarterly Performance Review, Autumn 1983Colonel Yuri Kuznetsov looked out the window anxiously. The endless gray landscape did little to soothe...

Published on September 26, 2024 1:44 PM GMTThis research was completed for London AI Safety Research (LASR) Labs 2024. The team was supervised by @Stefan Heimershiem (Apollo Research). Find out more about the program and express interest in upcoming iterations here.This video is a short overview of the project presented on the final day of the LASR Labs. Note that the paper was updated...

Published on September 26, 2024 1:40 PM GMT One thing I like about renting cars when I travel is that it's an opportunity to get a sense for a car that's a lot more detailed than what you'd get with a test drive. Traveling to DC for work a few days ago, I took the opportunity to rent a 2023 Chevy Bolt. This is...

Published on September 26, 2024 12:00 PM GMTWe interrupt Nate Silver week here at Don’t Worry About the Vase to bring you some rather big AI news: OpenAI and Sam Altman are planning on fully taking their masks off, discarding the nonprofit board’s nominal control and transitioning to a for-profit B-corporation, in which Sam Altman will have equity. We now know who they are...

Published on September 26, 2024 10:56 AM GMTI was reading about honeypots today. Honeypots (in AI evaluations, at least) are described as testing scenarios that could trick an AI into revealing deceptive or misaligned intent. My first thought was that, especially for current generation large language models, a honeypot scenario and a genuine user scenario would probably be indistinguishable. After all, if we say...

Published on September 26, 2024 9:17 AM GMTRecently, there was a post on SB-1047 and how it's quite mild regulation. I'm not expert on it and don't know how it works. In the comment section I was asking:Why wouldn't deep fake porn or voice cloning technology to engage in fraud be powerful enough to materially contribute to critical harm?There are cases of fraud that could...

Published on September 26, 2024 8:08 AM GMTToday we honor the actions of Stanislav Petrov (1939 – 2017) once again.Half an hour past midnight on September 26, 1983, he saw the first apparent launch on his computer monitor in a glass-walled room on the top floor of the Ballistic Missile Early Warning System (BMEWS) command and control post. The warning system was by now...

Published on September 26, 2024 1:50 AM GMT When I'm doing exploratory work I want to run many analyses. I'm usually optimizing for getting something quick, but I want to document what I'm doing enough that if there are questions about my analysis or I later want to draw on it I can reconstruct what I did. I've taken a few approaches to this...

Published on September 25, 2024 10:35 PM GMTWe (Goodfire) just put our research preview live - you can play with Llama 3 and use sparse autoencoders to read & write from its internal activations. This is a linkpost for:The research preview.Our blog post about building it.Taking research and turning it into something you can actually use and play with has been great. It's surprising...

Published on September 25, 2024 9:15 PM GMTOpenAI to remove non-profit controlReuters reports: Exclusive: OpenAI to remove non-profit control and give Sam Altman equityChatGPT-maker OpenAI is working on a plan to restructure its core business into a for-profit benefit corporation that will no longer be controlled by its non-profit board, people familiar with the matter told Reuters, in a move that will make the...

Published on September 24, 2024 1:58 PM GMTA lot is happening. How do you keep on top of things?For example:What's your system for finding out about important developments?What information sources do you find most useful?What are some problems with your current system?I'll share my answers in a comment below.Motivation:  I've noticed many cases where AI safety professionals—including leaders in the field—find out about important...

Published on September 24, 2024 1:49 PM GMTAuthors: John Hewitt, Nelson F. Liu, Percy Liang, Christopher D. Manning.Abstract:Instruction tuning commonly means finetuning a language model on instruction-response pairs. We discover two forms of adaptation (tuning) that are deficient compared to instruction tuning, yet still yield instruction following; we call this implicit instruction tuning. We first find that instruction-response pairs are not necessary: training solely...

Published on September 24, 2024 11:50 AM GMTPreviously: Book Review: On the Edge: The Fundamentals As I said in the Introduction, I loved this part of the book. Let’s get to it. Poker and Game Theory When people talk about game theory, they mostly talk solving for the equilibrium, and how to play your best game or strategy (there need not be a formal...

Published on September 24, 2024 11:30 AM GMT Lily recently wrote a song, and I've been helping them record it. I got us set up to record four tracks (vocals, drums, keyboard x2) and we did a bunch of takes: Most of the takes are full-length, but after we got a really nice version of the end we did some rounds with just the...

Published on September 24, 2024 11:00 AM GMTCurrent LLM based AI systems are getting pretty good at maths by writing formal proofs in Lean or similar. https://deepmind.google/discover/blog/ai-solves-imo-problems-at-silver-medal-level/So, how can we use these models to help align AI? The Simple Alignment Solution Assumption states that many problems in alignment, for example corrigibility or value learning, have simple solutions. I mean "simple" in the sense that logical...