~www_lesswrong_com | Bookmarks (687)

Published on December 29, 2024 12:01 PM GMTfrom aisafety.world The following is a list of live agendas in technical AI safety, updating our post from last year. It is “shallow” in the sense that 1) we are not specialists in almost any of it and that 2) we only spent about an hour on each entry. We also only use public information, so we are bound to...

Published on December 29, 2024 10:42 AM GMTI believe that AI research has not given sufficient attention to learning directly from biology, particularly through the direct observation and manipulation of neurons in controlled environments. Furthermore, even after learning all that biology has to offer,  neurons could still play a part in the post TAI world economy as they could be cheaper and faster to...

Published on December 29, 2024 3:13 AM GMTThis post examines the virtue of altruism. I’m less interested in breaking new ground, more in synthesizing the wisdom I could find about this virtue and how to cultivate it.Much about altruism on LessWrong and nearby sites concerns “effective altruism,” which takes altruistic motivation as a given and investigates how to do it most efficiently. This post...

Published on December 28, 2024 5:52 PM GMTI've heard many people say something like "money won't matter post-AGI". This has always struck me as odd, and as most likely completely incorrect.First: labour means human mental and physical effort that produces something of value. Capital goods are things like factories, data centres, and software—things humans have built that are used in the production of goods...

Published on December 28, 2024 5:01 PM GMTThe post makes the suggestion in the title: hopefully, it's second kind of obvious, if you take Character layer of models seriously. [1]Often, the problem of aligning AIs is understood as an instance of a broader Principal-Agent problem. If you take this frame seriously, what seems to be happening is somewhat strange: the Agent is mostly not serving...

Published on December 28, 2024 4:05 PM GMTConsider the following argument made by Tim Babb:So every (non-American) reader is forced to either bet against the market or concede that their credence is at least 16%.However, there is an important 3rd possibility. Since the market cannot resolve before August, it could also imply that Polymarket has an extremely high interest rate!Basically, betting against bird flu...

Published on December 28, 2024 3:42 PM GMTIn The Mystery Of Internet Survey IQs, Scott revises his estimate of the average LessWrong IQ from 138 to 128. He doesn’t explicitly explain how he arrived at this number, but it appears to be an average of the demographics norm method (123) and the SAT method (134). However, using the information in his post, the SAT...

Published on December 28, 2024 4:24 AM GMTThe section "The Future":As we enter 2025, we will have to become more than a lab and a startup — we have to become an enduring company. The Board’s objectives as it considers, in consultation with outside legal and financial advisors, how to best structure OpenAI to advance the mission of ensuring AGI benefits all of humanity...

Published on December 28, 2024 12:12 AM GMTLenses of Control addressed one of the intuitions behind the theory of Substrate Needs Convergence (summarized in What if Alignment is Not Enough?): the importance of understanding a system in the context of its environment.  This post will focus on another key intuition: the physical nature of an AGI and its levers of control on the world.The RobotOne...

Published on December 27, 2024 11:59 PM GMTEvery major author who has influenced me has "his own totalising and self-consistent worldview/philosophy". This list includes Paul Graham, Isaac Asimov, Joel Spolsky, Brett McKay, Shakyamuni, Chuck Palahniuk, Bryan Caplan, qntm, and, of course, Eliezer Yudkowsky, among many others. In this post, I'll attempt to lay out my own worldview and philosophy. If you have one too,...

Published on December 27, 2024 11:34 PM GMTMuch of this content originated on social media. To follow news and announcements in a more timely fashion, follow me on Twitter, Threads, Bluesky, or Farcaster.ContentsMy essaysFellowship opportunitiesAnnouncementsEventsNewsQuestionsLive gloriouslyWhere being right mattersOff-grid solar for data centersPermitting vs. complianceMirror life FAQCrossword maniaDo we want to democratize art-making?PolioHow many people could you feed on an acre?Verifiable videoLinks and tweetsMy...

Published on December 27, 2024 7:47 PM GMTGreedy-Advantage-Aware RLHF addresses the negative side effects from misspecified reward functions problem in language modeling domains. In a simple setting, the algorithm improves on traditional RLHF methods by producing agents that have a reduced tendency to exploit misspecified reward functions. I also detect the presence of sharp parameter topology in reward hacking agents, which suggests future research...

Published on December 27, 2024 7:40 PM GMTSomething I've been surprised by is just how fierce opposition to AI art has been. To clarify, I'm not talking about people who dislike AI art because they think it looks worse, but specifically, people with extreme animus towards the very concept of AI art, regardless of its aesthetic quality or artistic merit.I'm interested in this issue...

Published on December 27, 2024 4:36 PM GMT"You are beautiful, Enkidu, you are become like a god.Why do you gallop around the wilderness with the wild beasts?Come, let me bring you into Uruk-Haven,To the Holy Temple, the residence of Anu and Ishtar,The place of Gilgamesh, who is wise to perfection,But who struts his power over the people like a wild bull.”- Shamhat, from The...

Published on December 27, 2024 2:29 PM GMTCurrently, to get a drug approved by the FDA you need to prove that it helps with a particular illness and while having an acceptable side effect profile. If you have a drug that improves people's quality of life but you aren't curing an ICD-11 accepted illness, you don't get your drug approved. If the FDA would...

Published on December 27, 2024 2:18 PM GMTTake a stereotypical fantasy novel, a textbook on mathematical logic, and Fifty Shades of Grey. Mix them all together and add extra weirdness for spice. The result might look a lot like Planecrash (AKA: Project Lawful), a work of fiction co-written by "Iarwain" (a pen-name of Eliezer Yudkowsky) and "lintamande".(image from Planecrash)Yudkowsky is not afraid to be...

Published on December 27, 2024 1:21 PM GMTSummary: The Many-Worlds interpretation of quantum mechanics can help us respond virtuously to improbably good circumstances. Contemplating the less lucky branches of the quantum multiverse can be seen as a beneficial 'negative visualisation' practise comparable to those in Stoicism and some Buddhist traditions.I’ve previously suggested that when we think about the ethical implications of the many-worlds interpretation (MWI) of quantum mechanics,...

Published on December 27, 2024 1:20 PM GMTCause wow what is everyone even doing?! You know how you sometimes hear about people seeing colors when they hear words, or how they memorize thousands of phone numbers on sight, or that some navigate intricate mind palaces full of detailed sensory information?Ever wonder if you are secretly one of those people that have some highly unusual inner...

Published on December 27, 2024 11:53 AM GMTThis was a prose piece I performed for the 2024 KC Winter Solstice celebration, written by me I have a scar running from right above my navel to the right end of my abdomen. You see, I was born premature, with a volvulus, which means that my intestines were tangled and blue. I spent the first months of...

Published on December 27, 2024 11:21 AM GMTCapitalism is a force that has lifted billions out of poverty, where even poor remote villagers enjoy luxuries that would have been unimaginable to medieval kings. When someone takes a job, even the worst job, it’s because both parties expect mutual gain. And yet companies routinely get accused of exploiting their workers for offering low pay and...

Published on December 27, 2024 4:21 AM GMTSummary:This is a write-up of some rough work I did on analysing duplicate token neurons. It's not as thorough or well-written as I would like it to be, but hopefully, the explanation is at least somewhat legible.I examine duplicate token heads and the neurons they influence. I extract circuits the model uses to compute these neurons.I discuss...

Published on December 27, 2024 3:05 AM GMTI want to build a nice testing ground for human rationality (that is to say, solving arbitrary complex problems in different domains using limited information and time)This would be a lot of annoying work to assemble, but (un)fortunately there's an AI industry that's designing a lot bunch of evals to test the ability of their AIs to...

Published on December 27, 2024 1:58 AM GMTConscious beings can infer the physical contents and laws of their world in various ways - instinct that then fails to be contradicted by experience, trial and error, theory. But as Descartes noted, self-awareness logically necessitates at least one epistemic belief - the belief in the self. Physics could in principle exist without consciousness - and many...

Published on December 26, 2024 5:34 PM GMTBased on a 2-day hackathon brainstorm. Current status: 70% of the tooling is done, unsure of how to proceed. Not enough experience with multi-month sized projects to judge for feasibility. I'm looking for some feedback. Specifically I want feedback regarding my current implementation. The statement "SAEs could be useful for AI Control" is obvious enough. Working  hypothesis In general, AI...