~www_lesswrong_com | Bookmarks (692)

Published on December 5, 2024 9:04 AM GMTThis is a Part II of a long essay. Part I introduced the concept of morality-as-cooperation (MAC), and discussed how the principle could be used to understand moral judgements in human societies. Part III will discuss failure modes.Part II: Theory and ExperimentThe prior discussion of morality was human-centric, and based on historical examples of moral values that...

Published on December 5, 2024 8:16 AM GMTAbstractThe AI alignment problem is usually specified in terms of power and control. Given a single, solitary AGI, how can we constrain its behavior so that its actions remain aligned with human interests? Unfortunately, the answer, to a first approximation, appears to be "we can't." There are myriad reasons, but they mostly boil down to the fact...

Published on December 3, 2024 5:29 PM GMTTLDRIt is currently plausible (35%) to me that average orcas have at least as high potential for being great scientists as the greatest human scientists, modulo their motivation for doing science[1]. To see why I think this, see my LW question (including the considerations in my answer).I now want to test how intelligent orcas are as soon...

Published on December 3, 2024 5:27 PM GMTIn the spirit of the season, you can book a call with me to help w/ your interp project (no large coding though)Would you like someone to:Review your paper or code? Brainstorm ideas on next steps? How to best communicate your results? Discuss conceptual problems Obvious Advice (e.g. being affected by SAD because it's winter, not exercising,...

Published on December 3, 2024 12:30 PM GMTFor our annual update on how Balsa is doing, I am turning the floor over to Jennifer Chen, who is the only person working full time on Balsa Research. For my general overview of giving opportunities, see my post from last week. Previously: The 2023 Balsa Research update post, Repeal the Jones Act of 1920. tl;dr: In...

Published on December 3, 2024 12:20 PM GMT Our kids have been riding the bus since they were little, though less often since we started sharing a car almost four years ago. For a while the oldest, age 10, has been asking when they could start taking the bus on their own. A few days ago they did, and it went well! We worked...

Published on December 3, 2024 10:44 AM GMTTLDRLast weekend, I attended an AI evals hackathon organized by Arcadia, where the outputs were proposals for AISI’s evals bounty. My main learnings:Keep your evals simple, measuring one specific capability.It is surprisingly easy to come up with ideas, once you have seen examples.AISI extended the deadline for submissions by two weeks. Translation: they want more ideas!Story of...

Published on December 3, 2024 10:11 AM GMTTom did the original thinking; Rose helped with later thinking, structure and writing.Some plans for AI governance involve centralising western AGI development.[1] Would this actually be a good idea? We don’t think this question has been analysed in enough detail, given how important it is. In this post, we’re going to:Explore the strategic implications of having one project instead...

Published on December 3, 2024 9:29 AM GMTPrefaceSeveral friends have asked me about what psychological effects I think could affect human judgement about x-risk.  This isn't a complete answer, but in 2018 I wrote a draft of "AI Research Considerations for Human Existential Safety" (ARCHES) that included an overview of cognitive biases I thought (and still think) will impair AI risk assessments. Many cognitive...

Published on December 3, 2024 5:26 AM GMTEpistemic status: brief writeup of some interesting work I found in[1][2][3][4], among other places. Probably subtly wrong in places & if you have additions / comments to make, please do. Finite state automata (FSA) can be modeled with reactions of the form A+B→C+D..mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal;...

Published on December 3, 2024 4:33 AM GMTMIRI is a nonprofit research organization with a mission of addressing the most serious hazards posed by smarter-than-human artificial intelligence. In our general strategy update and communications strategy update earlier this year, we announced a new strategy that we’re executing on at MIRI, and several new teams we’re spinning up as a result. This post serves as...

Published on December 2, 2024 2:30 PM GMTThere is little sign that the momentum of the situation is changing. Instead, things continue to slowly get worse, as nations in holes continue to keep digging. The longer we wait, the more expensive the ultimate price will be. We will soon find out what the new administration does, which could go any number of ways. Table...

Published on December 2, 2024 1:28 PM GMTOn Carcinogenic Complexity, Software Senescence and Cognitive Provenance: Our roadmap for 2025 and beyondIt is mandatory to start any essay on AI in the post-ChatGPT era with the disclaimer that AI brings huge potential, and great risks. Unfortunately, on the path we are currently on, we will not realize those benefits, but are far more likely to...

Published on December 2, 2024 5:30 AM GMTThe Less Wrong General Census is unofficially here! You can take it at this link.The oft-interrupted annual tradition of the Less Wrong Census is once more upon us!If you are reading this post and identify as a LessWronger, then you are the target audience. If you are reading this post and don't identify as a LessWronger, you...

Published on December 2, 2024 4:55 AM GMTTwo months ago I attended Eric Drexler's launch of MSEP.one. It's open source software, written by people with professional game design experience, intended to catalyze better designs for atomically precise manufacturing (or generative nanotechnology, as he now calls it). Drexler wants to draw more attention to the benefits of nanotech, which involve large enough exponents that our...

Published on December 2, 2024 4:14 AM GMTWe were down between around 7PM and 8PM PT today. Sorry about that.It's hard to tell whether we got DDosd or someone just wanted to crawl us extremely aggressively, but we've had at least a few hundred IP addresses and random user agents request a lot of quite absurd pages, in a way that was clearly designed...

Published on December 1, 2024 9:05 PM GMTWhen I was 21 I wanted to become a video game programmer.Except I didn’t know how to program.I tried to teach myself and failed. Then I went back to school and learned the basics of Java in a week, aced the course in two months, and earned a cum laude MSc in Computer Science three years later.I’m...

Published on December 1, 2024 3:40 PM GMTIn other words, what apparent imperfections in human reasoningA) Remain apparent after the Replication Crisis,B) Aren't secretly adaptive/reasonable in some counterintuitive way, andC) Deal most damage to the people they inhabit (and/or those close to them, and/or wider society)?Discuss

Published on December 1, 2024 3:13 PM GMTAbstractThis paper introduces ESTIMAT (Evaluation System Through Information Management and Analysis Tool), a framework that bridges the art of self-discovery and the science of systematic estimation. Inspired by Fermi estimation principles, ESTIMAT transforms the challenge of understanding and optimizing cognitive states into a structured, measurable process. I could say that it is a proposal on how to...

Published on December 1, 2024 11:50 AM GMT How does the growth of Bluesky compare to Mastodon in 2022? What about Google Plus in 2011? I can't answer this globally, but (now that my comment archive is up to date) I can look at it for my immediate social network by looking at a proxy: how many of the replies people write to my...

Published on December 1, 2024 6:38 AM GMTDear lsusr,I am a lesswrong user interested in learning Mandarin and living in China. My goal is understanding Chinese culture more broadly and geopolitics and Chinese tech policy more specifically. I could get CELTA and get a teaching job in China (not difficult), but it seems like I would gain far more value if I actually learned Mandarin.Do...

Published on November 30, 2024 6:16 PM GMTSummary:Key Concerns with Lethal Autonomous Weapons⚠️ ❗ Unpredictability and Lack of Control⚠️ ❗ Exponential Lethality⚠️ ❗ Ethical and Legal ImplicationsRecommendations1️⃣ Immediate Moratorium: Enact a temporary ban on deploying LAWS untilcomprehensive regulations are established.2️⃣ Classification as WMDs: Classify lethal autonomous weapons, like 'loitering' AI missile systems, under weapons of mass destruction due to their scalable lethality.3️⃣ Ban Non-Compliant...

Published on November 30, 2024 12:40 PM GMT I want comments on my social media crossposts to show up on my blog as a comment section, and mostly this works well: modern systems ( Mastodon, Bluesky, LessWrong, etc) provide APIs where you can load the replies associated with a post. On the other hand, older systems like Facebook are more locked down: they want...