~www_lesswrong_com | Bookmarks (687)

Published on January 2, 2025 2:10 PM GMTThe Rationalist Project was our last best hope for peace. An epistemic world 50 million words long, serving as neutral territory. A place of research and philosophy for 30 million unique visitors A shining beacon on the internet, all alone in the night. It was the ending of the Age of Mankind. The year the Great Race...

Published on January 2, 2025 11:08 AM GMTIt seems like unlike most exciting economic ideas that economists swear by but governments ignore, Georgian Land Value Taxes should be fairly doable for a private development company to test. They would have to buy up a large area of rural land at a cheap price, and then rent it all out with a contract that specifies...

Published on January 2, 2025 10:16 AM GMTexpectation calibrator: stimulant-fueled vomiting of long-considered thoughtsIn a 2004 essay, "An Intuitive Explanation of Bayes' Theorem", Yudkowsky puts forth that it's not clear where Bayesain priors originally come from. Here's a dialogue from the post poking fun at that difficulty.Q. How can I find the priors for a problem?A. Many commonly used priors are listed in the...

Published on January 2, 2025 9:37 AM GMTIntroduction This is the first installment of my January writing project. We will look at generative neural networks from the framework of (probabilistic) "formal grammars", specifically focusing on building a complex grammar out of simple “rule grammars”. This turns out to lead to a nice, and relatively non-technical way of discussing how complex systems like language models...

Published on January 2, 2025 5:37 AM GMTI’m curious how alignment researchers would answer these two questions: What alignment progress do you expect to see in 2025? What results in 2025 would you need to see for you to believe that we are on track to successfully align AGI? Discuss

Published on January 2, 2025 5:01 AM GMTOn Jan 8 2024, I wrote a Google doc with my AI predictions for the next 6 years (and slightly edited the doc on Feb 24). I’ve now quickly sorted each prediction into Correct, Incorrect, and Unclear. The following post includes all of my predictions for 2024 with the original text unedited and commentary in indented bullets.Correctthere...

Published on January 2, 2025 4:42 AM GMTThe PuzzlesThere's a simple Monty Hall adjacent probability puzzle that goes like this:Puzzle 1I have two children, at least one of whom is a boy. What is the probability that both children are boys?A more complex variation recently went viral on Twitter:Puzzle 2I have two children, (at least) one of whom is a boy born on a...

Published on December 31, 2024 3:10 PM GMTWhat should we make of DeepSeek v3? DeepSeek v3 seems to clearly be the best open model, the best model at its price point, and the best model with 37B active parameters, or that cost under $6 million. According to the benchmarks, it can play with GPT-4o and Claude Sonnet. Anecdotal reports and alternative benchmarks tells us...

Published on December 31, 2024 2:06 PM GMTSome time ago I happened to read the concept of training rationale described by Evan Hubinger, and I really liked it. In case you are not aware: training rationales are a bunch of questions that ML developers / ML teams should ask themselves in order to self-assess pros and cons when adopting a certain safety approach.I decided to take...

Published on December 31, 2024 1:29 PM GMTThis post is a follow-up to The Plan - 2023 Version. There’s also The Plan - 2022 Update and The Plan, but the 2023 version contains everything you need to know about the current Plan. Also see this comment and this comment on how my plans interact with the labs and other players, if you’re curious about that part.What Have You Been...

Published on December 31, 2024 5:14 AM GMT I met a man in the Florida Keys who rents jet skis at $150/hour. Since nobody jet skis alone, he makes at least $300/hour. When there’s no customers he sits around watching sports. After work he plays with his two sons. I asked if he likes his lifestyle. He loves it.  Later when I was in...

Published on December 31, 2024 3:30 AM GMT I recently tried giving up sweets for two weeks. In early December I attended a conference, which meant a break from my normal routine. After a few days I realized this was the longest I'd gone without eating any sweets in 2-3 decades. After getting home I decided to go a bit longer to see if...

Published on December 30, 2024 10:50 PM GMTThanks to Jean Kaddour, Tomáš Dulka, and Joseph Bloom for providing feedback on earlier drafts of this post.In a previous post on Toy Models of Feature Absorption, we showed that tied SAEs seem to solve feature absorption. However, when we tried to training some tied SAEs on Gemma 2 2b, these still appeared to suffer from absorption...

Published on December 30, 2024 9:37 PM GMTA post on difficulty of eliminating malaria using gene drives: "I worked on gene drives for a number of years jointly as a member of George Church and Flaminia Catteruccia’s labs at Harvard. Most of my effort was spent primarily on an idea for an evolutionary stable gene drive, which didn’t work but we learned some stuff,...

Published on December 30, 2024 7:43 PM GMTTLDR:I think Eliezer Yudkowsky & many posts on LessWrong are failing at keeping things concise and to the point. Actual post:I think the content from Eliezer Yudkowsky & on LessWrong in general is unnecessarily wordy. A counterexample of where Eliezer Yudkowsky actually managed to get to the point concisely was in this Ted Talk, where he had the external...

Published on December 30, 2024 7:49 PM GMTThis is a linkpost for https://jbostock.substack.com/p/prologue-train-crashEpistemic status: fiction, satire even!I am writing a short story. This is the prologue. Most of it will just go on Substack, but I'll occasionally post sections on LessWrong, when they're particularly good.At some point in the past, canals and railways were almost equals: each had their merits and drawbacks, and their...

Published on December 30, 2024 2:10 PM GMTOpenAI presented o3 on the Friday before Christmas, at the tail end of the 12 Days of Shipmas. I was very much expecting the announcement to be something like a price drop. What better way to say ‘Merry Christmas,’ no? They disagreed. Instead, we got this (here’s the announcement, in which Sam Altman says ‘they thought it...

Published on December 29, 2024 4:14 PM GMTI’m considering translating my work into English to share it with the LessWrong community, but I’d like to first ask if it aligns with the community's interests and could be valuable. Below is a summary of the work to help evaluate its relevance: Beyond HaHa: Mapping the Causal Chain from Jokes to KnowledgeSummaryWe explore the specific causal mechanisms...

Published on December 29, 2024 4:13 PM GMTSummary. Zero to one is a collection of notes on startups by Peter Thiel (co-founder of PayPal and Palantir) that grew from a course taught by Thiel at Stanford in 2012. Its core thesis is that iterative progress is insufficient for meaningful progress. Thiel argues that the world can only become better if it changes dramatically, which...

Published on December 29, 2024 3:40 PM GMT Last night was the ninth Boston Secular Solstice ( 2023, 2022, 2019, 2018). This is not counting the 2020 one which was everywhere nearly simultaneously, with carefully calibrated delay. Skyler continued as lead organizer, with me coordinating music and Taymon helping in a bunch of ways. If you were there, it would be great if you...

Published on December 29, 2024 3:17 PM GMTTo many people, the land value tax (LVT) has earned the reputation of being the "perfect tax." In theory, it achieves a rare trifecta: generating government revenue without causing deadweight loss, incentivizing the productive development of land by discouraging unproductive speculation, and disproportionately taxing the wealthy, who tend to own the most valuable land.That said, I personally...

Published on December 29, 2024 2:53 PM GMTThe intended audience of this post is not the average Lesswrong reader; it is more for our family and friends that do not grasp the transformative power of AI in its current form. I am posting it here so that I can come back in a year and see how my predictions are doing. I hope that...

Published on December 29, 2024 2:35 PM GMTFollow up to: Could orcas be smarter than humans?(For speed of writing, I mostly don't cite references. Feel free to ask me in the comments for references for some claims.)This post summarizes my current most important considerations on whether orcas might be more intelligent than humans.Evolutionary considerationsWhat caused humans to become so smart?(Note: AFAIK there's no scientific...

Published on December 29, 2024 1:15 PM GMT "Computer Science" is not a science, and its ultimate significance has little to do with computers. The computer revolution is a revolution in the way we think and in the way we express what we think. The essence of this change is the emergence of what might best be called procedural epistemology—the study of the structure...