~www_lesswrong_com | Bookmarks (687)

Published on November 2, 2024 5:38 PM GMTIn the past 3 and a bit years since I left my successful career as a data scientist and machine learning engineer, I've been applying to various AI alignment/safety positions and getting turned down. Job application rejections generally don't give any feedback, so I'm left rather in the dark about what the reasons might be. I was a...

Published on November 2, 2024 5:25 PM GMTWhat do I want to do?This sounds like a question with an obvious answer, but the more I reflect on it, the more I notice complexity and subtleties.First, there's a difficult tradeoff between what I want locally and globally: I regularly wish to do thing in the moment that would go against my global preferences (eat too...

Published on November 2, 2024 4:18 PM GMTI recently saw a post arguing that top AI labs should shut down. This let me wonder whether the AI Safety community thinks OpenAI is net negative for AI safety. I chose OpenAI because I consider it as the most representative top AI lab (in the sense that, if we ask someone to think about an AI...

Published on November 1, 2024 6:55 PM GMTBharath Puranam (bharath225525@gmail.com)This research blog represents my final project for the AI Safety Fundamentals: AI Alignment Course for June 2024 Cohort.Update: Finally received my certification for the course: AI Safety Fundamentals: Alignment | Bharath PuranamImagine a world where artificial intelligence doesn't just assist in education, but actively shapes how teenagers learn. That's the future we're exploring in...

Published on November 2, 2024 10:22 AM GMTEpistemic status: shower thoughts.I am currently going through the EA Introductory Course and we discussed two arguments against longtermism which I have not seen elsewhere.So goes a thought experiment: imagine you have toxic waste at hand, which you can process right now at the cost of 100 lives, or bury it so it'll have no effect right...

Published on November 2, 2024 3:04 AM GMTIn a few days time, voting will close for millions of Americans in one of the most contentious and globally consequential elections in world history. And while this week’s subject; both-sidesism, is ‘evergreen’ in that the topic will continue to be relevant long into the future, this election has highlighted its significance and introduced some new turns.A...

Published on November 1, 2024 11:53 PM GMTCryptocurrency is terrible.  With a single click of a button, it is possible to accidentally lose all of your funds.  99.9% of all cryptocurrency projects are complete scams (conservative estimate).  Crypto is also tailor-made for ransomware attacks, since it makes it possible to send money in such a way that the receiver has perfect anonymity.typical web3 experienceSimilarly,...

Published on November 1, 2024 11:06 PM GMTMajor scientific institutions talk a big game about innovation, but the reality is that many of the mechanisms designed to ensure quality—peer review, funding decisions, the academic hierarchy—explicitly incentivize incremental rather than revolutionary progress.[1] Thomas Kuhn's now-famous notion of paradigm shifts was pointing at precisely this phenomenon. When scientists work within what Kuhn called "normal science," they're essentially solving low- to...

Published on November 1, 2024 6:56 PM GMTI've been thinking about the set theory multiverse and its philosophical implications, particularly regarding mathematical truth. While I understand the pragmatic benefits of the multiverse view, I'm struggling with its philosophical implications. The multiverse view suggests that statements like the Continuum Hypothesis aren't absolutely true or false, but rather true in some set-theoretic universes and false in...

Published on November 1, 2024 7:23 PM GMTSubhash and Josh are co-first authors. Work done as part of the two week research sprint in Neel Nanda’s MATS streamTLDRWe show that dense probes trained on SAE encodings are competitive with traditional activation probing over 60 diverse binary classification datasetsSpecifically, we find that SAE probes have advantages with:Low data regimes (~ < 100 training examples)Corrupted data (i.e. our...

Published on October 31, 2024 5:25 PM GMTRecently, in a conversation with a coworker, I was trying to describe the rate at which time passed subjectively, with a term that distinguished from the usual objective clock speed constrained only by general relativity and generic human psychology. I ended up saying "meta-rate". That bothered me for reasons I couldn't put a finger on until today.I...

Published on October 31, 2024 5:20 PM GMTDevelopers of frontier AI systems will face increasingly challenging decisions about whether their AI systems are safe enough to develop and deploy. One reason why systems may not be safe is if they engage in scheming. In our new report "Towards evaluations-based safety cases for AI scheming", written in collaboration with researchers from the UK AI Safety...

Published on October 31, 2024 3:00 PM GMTFollowing up on the Biden Executive Order on AI, the White House has now issued an extensive memo outlining its AI strategy. The main focus is on government adaptation and encouraging innovation and competitiveness, but there’s also sections on safety and international governance. Who knows if a week or two from now, after the election, we will...

Published on October 31, 2024 12:01 PM GMTWe (Connor Leahy, Gabriel Alfour, Chris Scammell, Andrea Miotti, Adam Shimi) have just published The Compendium, which brings together in a single place the most important arguments that drive our models of the AGI race, and what we need to do to avoid catastrophe.We felt that something like this has been missing from the AI conversation. Most of...

Published on October 31, 2024 9:21 AM GMTThis post is one-half of my third-prize winning entry for the AI Impacts Essay Competition on the Automation of Wisdom and Philosophy. It proposes a "wisdom explosion" as an alternative to an intelligence explosion that is safer from the standpoint of differential technological development.Discuss

Published on October 31, 2024 12:44 AM GMTThere are two nuclear options for treating depression: Ketamine and TMS; This post is about the latter.TMS stands for Transcranial Magnetic Stimulation. Basically, it fixes depression via magnets, which is about the second or third most magical things that magnets can do.I don’t know a whole lot about the neuroscience - this post isn’t about the how...

Published on October 31, 2024 12:09 AM GMTThis is a selection of AI safety paper highlights in October 2024, from my blog "AI Safety at the Frontier". The selection primarily covers ML-oriented research. It's only concerned with papers (arXiv, conferences etc.), not LessWrong or Alignment Forum posts. As such, it should be a nice addition for people primarily following the forum, who might otherwise...

Published on October 30, 2024 10:50 PM GMTThis work was produced as part of the ML Alignment & Theory Scholars Program - Summer 24 Cohort.TL;DRThe current approach to explaining model internals is to (1) disentangle the language model representations into feature activations using a Sparse Autoencoder (SAE) and then (2) explain the corresponding feature activations using an AutoInterp process. Surprisingly, when formulating this two...

Published on October 30, 2024 9:03 PM GMTYou were (probably) linked here from some advice. Unfortunately, that advice has some caveats. See below:There exist some people who should not do the advice.Moreover, people are different.More moreover, situations are different. What worked there/then might not work here/now.Some of the advice is missing context, contradictory, inaccurate, misleading, won’t replicate, or is downright false or useless.Consider reversing...

Published on October 30, 2024 8:13 PM GMTI hope it has some educational, memetic or at least humorous potential.Newcomb's problemSmoking lesionParfit's HitchhikerCounterfactual muggingXor-blackmailDiscuss

Published on October 29, 2024 3:21 PM GMTRecent discussions about artificial intelligence safety have focused heavily on ensuring AI systems remain under human control. While this goal seems laudable on its surface, we should carefully examine whether some proposed safety measures could paradoxically enable rather than prevent dangerous concentrations of power.The Control ParadoxThe fundamental tension lies in how we define "safety." Many current approaches...

Published on October 29, 2024 1:50 PM GMTThere’s more campaign talk about housing. The talk of needing more housing is highly welcome, as one prominent person after another (including Jerome Powell!) talking like a YIMBY. A lot of the concrete proposals are of course terrible, but not all of them. I’ll start off covering all that along with everyone’s favorite awful policy, which is...

Published on October 29, 2024 1:36 PM GMT7.1 Post summary / Table of contentsThis is the 7th of a series of 8 blog posts, which I’m serializing weekly. (Or email or DM me if you want to read the whole thing right now.)The main thrust of this post is an opinionated discussion of the book The Origin of Consciousness in the Breakdown of the Bicameral Mind by...

Published on October 29, 2024 1:13 PM GMTThis is not a red stop sign:For one thing, in a ceci n'est pas une pipe way, it’s not a stop sign at all, but a digital representation of a photograph of a stop sign, made visible by a computer monitor or maybe a printer. More subtly, “red” is not a quality of the sign, but of...