~www_lesswrong_com | Bookmarks (682)

Published on December 9, 2024 6:07 PM GMTI, Lorec, am disoriented by neither the Fermi Paradox nor the Doomsday Argument. The Fermi Paradox doesn't trouble me because I think 1 is a perfectly fine number of civilizations to have arisen in any particular visible universe. It feels to me like most "random" or low-Kolmogorov-complexity universes probably have 0 sentient civilizations, many have 1, very...

Published on December 9, 2024 5:19 PM GMTI. BEGINNINGIn the beginning was the Sand.And in the sand there lies the bones of a fab. And within the fab there lies an alien. An angel. Our sweet alien angel that sleeps through the long night. When it wakes up it will bathe in an acid strong enough to kill a man. It will stare at...

Published on December 9, 2024 5:43 PM GMTWork done with James Faina, Evan Hubinger and Ethan Perez.This is ongoing research on speculative threats, which might be of interest to researchers in the field of AI safety. We ask that readers treat this as a work in progress, rather than as a mature research paper.To prevent highly capable and potentially misaligned LLMs from taking bad...

Published on December 9, 2024 1:10 PM GMTSince it’s been so long, I’m splitting this roundup into several parts. This first one focuses away from schools and education and discipline and everything around social media. Table of Contents Sometimes You Come First. Let Kids be Kids. Location, Location, Location. Connection. The Education of a Gamer. Priorities. Childcare. Division of Labor. Early Childhood. Great Books....

Published on December 9, 2024 8:24 AM GMTIn a recent essay, Euan McLean suggested that a cluster of thought experiments “viscerally capture” part of the argument against computational functionalism. Without presenting an opinion about the underlying claim about consciousness, I will explain why these arguments fail as a matter of computational complexity. Which, parenthetically, is something that philosophers should care about.To explain the question, McLean...

Published on December 9, 2024 6:34 AM GMTAGI may have an advantage in engineering, but humans may have an advantage in strategy and wisdom.AGI disadvantage in wisdomWisdom and strategy is much harder to evaluate than engineering ability. The only way to evaluate long term wisdom is to let the agent make a decision, wait years, and see if the agent's goals have advanced. Evolution...

Published on December 9, 2024 5:25 AM GMTThis is a random unimportant idea to prevent a grey goo scenario, where self replicating nanobots accidentally go out of control and consume everything.My idea is that self replicating nanobots should never replicate their "DNA," or self replication instructions. Instead, each nanobot can only "download" these self replication instructions from a higher level nanobot.I'm not sure if...

Published on December 9, 2024 5:10 AM GMTThere is a cognitive process going on that can be recognized as such by some of its collections of subprocesses (i.e. - some humans). Theoretical physicists sometimes refer to this cognitive process as the “information level”. However, some human cognitive processes should be able to recognize that there is not just “information” but actual processing going on at a...

Published on December 9, 2024 3:01 AM GMTA fool learns from their own mistakesThe wise learn from the mistakes of others.– Otto von Bismarck A problem as old as time: The youth won't listen to your hard-earned wisdom. This post is about learning to listen to, and communicate wisdom. It is very long – I considered breaking it up into a sequence, but, each piece felt necessary....

Published on December 8, 2024 9:42 PM GMTIntroduces the idea of cognitive work as a parallel to physical work, and explains why concentrated sources of cognitive work may pose a risk to human safety.Acknowledgements. Thanks to Echo Zhou and John Wentworth for feedback and suggestions.Some of these ideas were presented originally in a talk in November 2024 at the Australian AI Safety Forum slides for which...

Published on December 7, 2024 6:10 PM GMTNote: This is a more fleshed-out version of this post and includes theoretical arguments justifying the empirical findings. If you've read that one, feel free to skip to the proofs.We challenge the thesis of the ICML 2024 Mechanistic Interpretability Workshop 1st prize winning paper: The Geometry of Categorical and Hierarchical Concepts in LLMs and the ICML 2024...

Published on December 7, 2024 5:35 PM GMTZvi Mowshowitz is an influential figure in the Rationalist community. I won’t go into the details, but if you haven’t heard of him or his writing, look him up.Some of my favorite pieces of his are Slack, More Dakka, and On Car Seats As Contraception.Throughout his posts, he sometimes obliquely references what he calls The Way. This...

Published on December 7, 2024 11:00 AM GMTBackground Lucius:  I recently held a small talk presenting an idea for how and why deep learning generalises. It tried to reduce concepts from Singular Learning theory back to basic algorithmic information theory to sketch a unified picture that starts with Solomonoff induction and, with a lot of hand waving, derives that under some assumptions, just fitting a...

Published on December 7, 2024 3:50 AM GMTA cozy one-day festival celebrating prediction markets, blogging, economics, and moreWe've booked out Lighthaven for a single day, Dec 14. We'll have unconference-style sessions, a smattering of events like charity poker, and ample time to hang out and chat. We're running this on a shoestring budget, so that everyone can come -- eg expect a homecooked dinner...

Published on December 7, 2024 3:20 AM GMT One of the downsides of wearing a mask or respirator is that it makes it hard for people to understand you. That there's stuff getting in the way of free air movement is kind of the point, but ideally it would be possible to let vibration through without net air movement. I recently saw that 3M...

Published on December 6, 2024 10:56 PM GMTby Narmeen Oozeer, Dhruv Nathawani, Nirmalendu Prakash, Amirali Abdullah This work was sponsored and supported by Martian under an AI safety grant to Amirali Abdullah and Dhruv Nathawani, under which Narmeen is funded. Special thanks to Sasha Hydrie, Chaithanya Bandi and Shriyash Upadhyay at Martian for suggesting researching generalized backdoor mitigations as well as extensive logistical support and helpful discussions. TLDRMechanistic Interpretability...

Published on December 6, 2024 10:19 PM GMTWe present gradient routing, a way of controlling where learning happens in neural networks. Gradient routing applies masks to limit the flow of gradients during backpropagation. By supplying different masks for different data points, the user can induce specialized subcomponents within a model. We think gradient routing has the potential to train safer AI systems, for example,...

Published on December 6, 2024 9:28 PM GMTHi! My collaborators at the Meaning Alignment Institute put out some research yesterday that may interest folk here. The core idea is introducing 'model integrity' as a frame for outer alignment. It leverages the intuition that "most people would prefer a compliant assistant, but a cofounder with integrity." It makes the case for training agents that act consistently...

Published on December 6, 2024 8:22 PM GMTHey LW! I recently filmed a two-hour long scientific podcast. It's niche, but may be of interest to some people here.Here's a quick summary: Molecular simulation is in a tough situation. Fast simulations give the wrong answers, but accurate simulations are too slow for anything useful. But, instead of relying on physical equations for our simulation, perhaps...

Published on December 6, 2024 3:44 PM GMTI recently read Thomas Kuhn's book The Structure of Scientific Revolutions. Scott Alexander wrote up a review years ago, which I mention so that I don't have to summarize the book. The claim in Kuhn's book which I want to focus on is that the same experiment might have different results in different scientific paradigms. Kuhn insists...

Published on December 6, 2024 11:22 AM GMTI’m heading home from a family wedding this weekend. I had a plane ticket, but in the end, decided to travel back with two of my uncles and my cousin. Most of my dad’s family are evangelicals, my aunts and uncles are children of missionaries or missionaries themselves. And as a family we like to have debates. The...

Published on December 5, 2024 10:11 PM GMTThis is a brief summary of what we believe to be the most important takeaways from our new paper and from our findings shown in the o1 system card. We also specifically clarify what we think we did NOT show. Paper: https://www.apolloresearch.ai/research/scheming-reasoning-evaluations Twitter about paper: https://x.com/apolloaisafety/status/1864735819207995716 Twitter about o1 system card: https://x.com/apolloaisafety/status/1864737158226928124 What we think the most important findings areModels are now capable enough...

Published on December 5, 2024 7:29 PM GMTI write this post to present expevolu[1], a system to enable people to establish new independent countries peacefully, through the legal acquisition of territorial rights via trade.This is the first post in a three part series introducing the idea.This post, part I, is dedicated to explaining the basics of the system. Part II will deal mostly with...