Bookmarks (683)

Published on January 21, 2025 6:09 PM GMTLike this piece? It's cross-posted from by blog: https://collisteru.net/writing/This is a work of fiction and parody. I have done my best to get the scientific details right as far as they are known today, but my real goal is social commentary, not scientific accuracy. NOAA DISCOVERS INSCRIBED METEOR ARTIFACT UNDERNEATH ATLANTICDec 8, 2018Scientists from NOAA claim to have...

Published on January 21, 2025 5:34 PM GMTGetting personalised advice from a real human can help newcomers to AI safety figure out how to contribute most effectively. For example, I (Bryce) ended up in my current role largely thanks to a call with 80,000 Hours.There are a number of organisations and individuals offering advisory calls, but many people who want to work on AI...

Published on January 21, 2025 5:27 PM GMTCrossposted on The Field Building Blog and the EA forum.Discuss

Published on January 21, 2025 4:10 PM GMTMuch of this content originated on social media. To follow news and announcements in a more timely fashion, follow me on Twitter, Threads, Bluesky, or Farcaster.ContentsMy writing (ICYMI)Jobs and fellowshipsAnnouncementsNewsEventsOther opportunitiesWe are not close to providing for everyone’s “needs”The printing press and the InternetThe ultimate form of travelFive hot takes about progressWhat could have been, for SFQuick thoughts on AILinks and bulletsChartsPicsMy...

Published on January 21, 2025 4:03 PM GMTThe AI Control Agenda, in its own words:… we argue that AI labs should ensure that powerful AIs are controlled. That is, labs should make sure that the safety measures they apply to their powerful models prevent unacceptably bad outcomes, even if the AIs are misaligned and intentionally try to subvert those safety measures. We think no fundamental research...

Published on January 21, 2025 3:31 PM GMTPosition Piece: Most of the developing world lacks the institutional capacity to adapt to powerful, unsecure AI systems by 2030. Incautious model release could disproportionately affect these regions. Enhanced societal resilience in frontier AI states consequently provides no 'black cheque' for incautious release. ‘We should develop more societal resilience to AI-related harms’ is now a common refrain in AI...

Published on January 21, 2025 12:20 PM GMTAs always, some people need practical advice, and we can’t agree on how any of this works and we are all different and our motivations are different, so figuring out the best things to do is difficult. Here are various hopefully useful notes. Table of Contents Effectiveness of GLP-1 Drugs. What Passes for Skepticism on GLP-1s. The...

Published on January 21, 2025 12:03 PM GMTWe still need more funding to be able to run another edition. Our fundraiser raised $6k as of now, and will end if it doesn't reach the $15k minimum, on February 1st. We need proactive donors.If we don't get funded for this time, there is a good chance we will move on to different work in AI...

Published on January 21, 2025 10:47 AM GMTMy view on the concept of responsibility has shifted a lot over the years. I’ve had three insights that brought me from my initial, very superficial and implicit understanding of responsibility, to the one I have today, which I consider more accurate, more practical, and more healthy.  Responsibility is Made UpThe first insight came while I was part...

Published on January 19, 2025 6:29 PM GMTA story I wrote about living through the transition to utopia.This is the one story that I've put the most time and effort into; it charts a course from the near future all the way to the distant stars.Discuss

Published on January 19, 2025 10:32 AM GMTWe choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard. (Kennedy’s famous “We chose to go to the moon” speech) The ‘real’ mathematics of ‘real’ mathematicians, …, is almost wholly ‘useless’ (Hardy’s “A Mathematician’s Apology”) If the "irrational" agent is outcompeting you on a...

Published on January 19, 2025 8:04 AM GMTTl;dr, Neural networks are deterministic and sometimes even reversible, which causes Shannon information measures to degenerate. But information theory seems useful. How can we square this (if it's possible at all)? The attempts so far in the literature are unsatisfying.Here is a conceptual question: what is the Right Way to think about information theoretic quantities in neural...

Published on January 19, 2025 6:50 AM GMTCapitalism is powered by greed. People want to make money, so they look hard for things they can produce and that others want. Unknowingly, however, they are powering the great information-processing machine that is the market. The output of the machine is the efficient allocation of resources and, eventually, wealth.Something we intuitively consider bad (greed) is made...

Published on January 19, 2025 3:03 AM GMTTHERE ARE LAKES at the bottom of the ocean. I saw it in a nature documentary. You get a weird mineral deposit on the seafloor and it makes these brine pools, water so salty it doesn't mix with the sea water around it. Because it has no oxygen, any unlucky fish or crabs that fall in there...

Published on January 19, 2025 3:53 AM GMTLast week some results were released from a 6-week study using AI tutors in Nigeria. Below I summarize the results of that and four other recent studies about AI tutoring (the dates reflect when the study was conducted rather than when papers were published):Summer 2024 — 15–16-year olds in NigeriaThey had 800 students total. The treatment group...

Published on January 18, 2025 10:32 PM GMTHow many times have you been in a meeting where people seem to talk past each other? Everyone is smart and well-intentioned, but you don’t seem to be making any progress.Here’s the likely problem, you don’t have a tangible thing to anchor your discussions around. You need something real (a doc, a sketch, a prototype) to create...

Published on January 18, 2025 3:32 PM GMTWhen learning about classic utilitarianism (approximately, the quest to maximize everyone's expected well-being), I struggle because much of my well-being seems internal. If happiness or misery are significantly influenced by our internal processing of events, then how does this affect utilitarianism and its practical application?I'll start with a few examples:When I was a child, my family moved...

Published on January 18, 2025 3:39 PM GMTHi, everyone. I'm not sure if my post is well-written, but I think LW might be the only right place to have this discussion. Feel free to suggest changes.AGI may arrive soon and it is possible that it would kill us all. This does not bother me that much, as dying happens to people all the time.But...

Published on January 18, 2025 4:00 PM GMT In general, you're not supposed to wear a beard with a respirator mask (N95, P100, etc), at least not in a way where you have facial hair under the seal: But how much worse is the fit? A P100 with a beard is going to filter less well than a P100 without a beard, but does...

Published on January 18, 2025 3:29 PM GMTConfusion is a felt sense; a bodily sensation you can pay attention to and notice! So here's my question for you:What's this Song about?Think About Things by Daði FreyYoutube Link, Spotify LinkFull Lyrics:Baby I can't wait to knowBelieve me I'll always be there soThough I know I love youI find it hard to see how you feel...

Published on January 18, 2025 3:10 PM GMTWe’re developing an AI-enabled wargaming-tool, grim, to significantly scale up the number of catastrophic scenarios that concerned organizations can explore and to improve emergency response capabilities of, at least, Sentinel.Table of ContentsHow AI Improves on the State of the ArtImplementation Details, Limitations, and ImprovementsLearnings So FarGet Involved!How AI Improves on the State of the ArtIn a wargame,...

Published on January 18, 2025 12:43 PM GMTepistemic status: I know next to nothing about evolution, development psychology, AI, alignment. Anyway, I think the topic is important, and I should do my, however small part, in trying to think seriously for 5 minutes about it. So here's what I thinkHow come, that I am aligned? Somehow neocortex plays along with older parts of the...

Published on January 18, 2025 9:20 AM GMTI think a lot of people have heard so much about internalized prejudice and bias that they think they should ignore any bad vibes they get about a person that they can’t rationally explain. But if a person gives you a bad feeling, don’t ignore that. Both I and several others who I know have generally come...

Published on January 18, 2025 3:54 AM GMTIntroduction: Why QFT?In a previous post, Lauren offered a take on why a physics way of thinking is so successful at understanding AI systems. In this post, we look in more detail at the potential of Quantum field theory (QFT) to be expanded into a more comprehensive framework for this purpose. Interest in this area has been...

Published on January 17, 2025 9:24 PM GMTCross-posted from SubstackFeeling intellectually understimulated, I've begun working my way through Max Lamparth's CS120 - Introduction to AI Safety. I'm going to use this Substack as a kind of open journaling practice to record my observations on the ideas presented, both in the lectures and in the readings.The reading for the first week is Dobbe, Gilbert, and...

Published on January 17, 2025 9:45 PM GMTInner Alignment is the problem of ensuring mesa-optimizers (i.e. when a trained ML system is itself an optimizer) are aligned with the objective function of the training process.Inner alignment asks the question: How can we robustly aim our AI optimizers at any objective function at all?As an example, evolution is an optimization force that itself 'designed' optimizers...

Published on January 16, 2025 6:03 PM GMTThis is a linkpost for How should we analyse survey forecasts of AI timelines? by Tom Adamczewski, which was published on 16 December 2024[1]. Below are some quotes from Tom's post, and a bet I would be happy to make with people whose AI timelines are much shorter than those of the median AI expert.How should we...

Published on January 16, 2025 5:20 PM GMTSummaryCan LLMs science? The answer to this question can tell us important things about timelines to AGI. In this small pilot experiment, we test frontier LLMs on their ability to perform a minimal version of scientific research, where they must discover a hidden rule about lists of integers by iteratively generating and testing hypotheses. Results are ambiguous:...

Published on January 16, 2025 5:19 PM GMTI would like to thank Professor Vincent Conitzer, Caspar Oesterheld, Bernardo Subercaseaux, Matan Shtepel, and Robert Trosten for many excellent conversations and insights. All mistakes are my own.I think that there's a fundamental connection between AI Safety via Debate and Guaranteed Safe AI via topology. After thinking about AI Safety via Debate for nearly two years, this...

Published on January 16, 2025 2:20 PM GMTThe fun, as it were, is presumably about to begin. And the break was fun while it lasted. Biden went out with an AI bang. His farewell address warns of a ‘Tech-Industrial Complex’ and calls AI the most important technology of all time. And there was not one but two AI-related everything bagel concrete actions proposed –...

Published on January 16, 2025 1:55 PM GMTNB this dialogue occurred at the very end of 2023, and for various reasons is only being published ~a year later! Keep this in mind while reading.Key topics coveredconsistent NN-internal deceptive planning (search: goblin, fae, homunc, inner)making AI/agents from foundation models (search: scaffold, condition)planning (search: planning, simulator, consequential)standards of evidence (search: fanfic, theory, Shrek, analog, logic)elicitation gaps...

Published on January 16, 2025 11:38 AM GMTWeirdML websiteRelated posts:How good are LLMs at doing ML on an unknown dataset?o1-preview is pretty good at doing ML on an unknown datasetIntroductionHow good are Large Language Models (LLMs) at doing machine learning on novel datasets? The WeirdML benchmark presents LLMs with weird and unusual machine learning tasks, designed to require careful thinking and actual understanding to...

Published on January 16, 2025 10:51 AM GMTFrom the earliest days of evolutionary thinking, we’ve used metaphors to understand how life changes over time. One of the most enduring is the image of a vast “fitness landscape” with countless peaks and valleys, each corresponding to different levels of survival and reproduction. This landscape is a way of imagining how living things (the “replicators”) search...

Published on January 16, 2025 8:53 AM GMTLearning quantum mechanics involves two things:learning the fundamentals, the nature of the stuff of which Nature is made; andloads of universe-imposed incidental complexity, spinors and vector calculus and Lie algebras and Hilbert spaces, in order to be able to effectively apply the fundamentals to the real world.Now, me, personally, I really like the feeling of understanding how...

Published on January 16, 2025 8:04 AM GMTToday's "nanowrimo" post is a fun longform introduction to permanents and their properties, organized in the way I wish it had been explained to me. Note before going on: this is not my field, and this post is even more likely than usual to contain incorrect statements or bugs.Recall that given an n×n.mjx-chtml {display: inline-block; line-height: 0; text-indent:...

Published on January 16, 2025 2:14 AM GMT(Explanation. Also I have no reason to think they hate me.)Do not use the original TruthfulQA multiple-choice or the HaluEval benchmark. We show that a simple decision tree can theoretically game multiple-choice TruthfulQA to 79.6% accuracy—even while hiding the question being asked! In response, the TruthfulQA authors created a new multiple-choice condition which avoids the vulnerabilities we...

Published on January 15, 2025 6:16 PM GMTApplications are now open for the Cooperative AI Summer School, which will take place from 9th to 13th July 2025 in Marlow, near London! Designed for students and early-career professionals in AI, computer science, and related disciplines—such as sociology and economics—the summer school offers a firm grounding in the emerging field of cooperative AI.Confirmed speakers for this year include:Michael Wellman (University...

Published on January 15, 2025 6:00 PM GMTI'm collecting (x-risk-relevant) safety research from frontier AI companies published in 2023 and 2024: https://docs.google.com/spreadsheets/d/10_dzImDvHq7eEag6paK6AmIdAGMBOA7yXUvumODhZ5U/edit?usp=sharing.I was planning to get AI safety researchers to score each of the papers, so that we could compare the labs on quality-adjusted safety research output. I'm giving up on this for now, largely because I expect to struggle to find scorers. Let...

Published on January 15, 2025 3:55 PM GMTSome fraction of the time, LLMs naturally go on existential rants. My best guess is that, just as people can flip into a context where we do that, so can LLMs. With the result that the LLM certainly sounds like it is suffering, even if we discount the possibility that it actually is. Which raised a question....

Published on January 15, 2025 2:50 PM GMTSocial networks have shaped me since a young age. Growing up at the beginning of the millennium, I used to spend my time in phpBB and vBulletin forums. There, I befriended internet strangers, started my way into graphic design, and learned about torrents. Forums were my favorite third places—little corners on the web where I felt a...

Published on January 15, 2025 2:30 PM GMTTable of Contents Man With a Plan. Oh the Pain. Actual Proposals. For AI Builders. Think of the Children. Content Identification. Infrastructure Week. Paying Attention. Man With a Plan The primary Man With a Plan this week for government-guided AI prosperity was UK Prime Minister Keir Starmer, with a plan coming primarily from Matt Clifford. I’ll be...

Published on January 15, 2025 2:22 PM GMTThe first section describes problems with a few different alignment targets. The second section argues that it is useful to view all of them as variations of a single alignment target: building an AI that does what a Group wants that AI to do. The post then goes on to argue that all of the individual problems...

Published on January 15, 2025 2:08 PM GMTMy current outlook on LLMs is that they are some combination of bullshit to fool people who are looking to be fooled, and a modest but potentially very important improvement in the capacity to search large corpuses of text in response to uncontroversial natural-language queries and automatically summarize the results. Beyond this, I think they’re massively overhyped....

Published on January 15, 2025 6:45 AM GMTSometimes I see a comment I'd like to bookmark, but currently the only ways to save a comment are by subscribing to its replies (which sometimes produces unwanted notifications and requires me to check a different profile section than the rest of my bookmarks) or bookmarking the post it's attached to (which can be inconvenient for posts...

Published on January 15, 2025 6:11 AM GMTThis is an article in the featured articles series from AISafety.info. AISafety.info writes AI safety intro content. We'd appreciate any feedback. The most up-to-date version of this article is on our website, along with 300+ other articles on AI existential safety.There are many fictional stories that depict unaligned non-human entities in ways that can illustrate some aspects of...

Published on January 15, 2025 4:56 AM GMTThere is a view I’ve encountered somewhat often,[1] which can be summarized as follows: After the widespread deployment of advanced AGI, assuming humanity survives, material scarcity will largely disappear. Everyone will have sufficient access to necessities like food, housing, and other basic resources. Therefore, the only scarce resource remaining will be "social status". As a result, the primary activity...

Published on January 13, 2025 6:35 PM GMTMuch of this content originated on social media. To follow news and announcements in a more timely fashion, follow me on Twitter, Threads, Bluesky, or Farcaster.ContentsFrom me and RPIJobs and fellowshipsOther opportunitiesEventsQuestionsAnnouncementsCommentary on the wildfiresSam Altman: AI workers in 2025, superintelligence nextNever underestimate elasticity of supply“The earnestness and diligence of smart technical people”“Americans born on foreign soil”UndauntedEli Dourado’s model of policy changeStatsLinksAIInspirationPoliticsChina...

Published on January 13, 2025 3:05 PM GMTNote: Thank you to Surge Biswas (founder of Nabla Bio) for comments on this draft and and Dylan Reid (an investor into Nabla) for various antibody discussions! Also, thank you to Martin Pacesa for adding some insight on a paper of his I discuss here (his comments are included).IntroductionAntibody design startups are singlehandedly the most common archetype...

Published on January 13, 2025 2:31 PM GMTNote: I am a postdoc in fMRI neuroscience. I have been exploring LLMs for their use in fMRI research (e.g., extracting activations and mapping them onto brain data). To support this direction, I first did something of an interpretability study. This yielded some potentially interesting and surprising results on the emergent effects of scaling. I organized the...