~www_lesswrong_com | Bookmarks (696)

Published on February 26, 2025 2:24 PM GMT(This post is to a significant extent just a rewrite of this excellent comment from niplav. It is one of the highest-leverage insights I know for learning faster.)TheoryTo a large extent we learn by updating on feedback. You might e.g. get positive feedback from having an insight that lets you solve a math problem, which then reinforces...

Published on February 26, 2025 12:50 PM GMTWe just published a paper aimed at discovering “computational sparsity”, rather than just sparsity in the representations. In it, we propose a new architecture, Jacobian sparse autoencoders (JSAEs), which induces sparsity in both computations and representations. CLICK HERE TO READ THE FULL PAPER.In this post, I’ll give a brief summary of the paper and some of my thoughts...

Published on February 26, 2025 1:58 PM GMToutlining is specialized work which reduces a text to complete summary statements and collapsed detail.an outline containing a work sprint. note the collapsed points in the 'old sprints' which hide all the old sprint detail.outlining is historically recent, since particular digital interfaces (such as Workflowy, Org Mode, Dynalist or Ravel) make it orders of magnitude easier to...

Published on February 26, 2025 1:50 PM GMTThe more I learn about urban planning, the more I realize that the American city I live in is dystopic. I'm referring specifically to urban planning, and I'm not being hyperbolic. Have you ever watched the teen dystopia movie Divergent? The whole city is perfectly walkable (or parkourable, if you're Dauntless). I don't know if it even...

Published on February 26, 2025 1:00 PM GMTAnthropic has reemerged from stealth and offers us Claude 3.7. Given this is named Claude 3.7, an excellent choice, from now on this blog will refer to what they officially call Claude Sonnet 3.5 (new) as Sonnet 3.6. Claude 3.7 is a combination of an upgrade to the underlying Claude model, and the move to a hybrid...

Published on February 26, 2025 7:07 AM GMTI have discussions that ignore the future disruptive effects of AI all the time. The national debt is a real problem. Social security will collapse. The environment is deteriorating. You haven't saved enough for pension. What is my two year old going to do when she is twenty. Could Israel make peace with the Palestinians next generation? And...

Published on February 24, 2025 6:31 PM GMTA new paper by Yoshua Bengio and the Safe Artificial Intelligence For Humanity (SAIFH) team argues that the current push towards building generalist AI agents presents catastrophic risks, creating a need for more caution and an alternative approach. We propose such an approach in the form of Scientist AI, a non-agentic AI system that aims to be...

Published on February 24, 2025 5:46 PM GMTepistemic status: clearing my own confusionI'm going to discuss what we mean by preferences of an intelligent agent and try to make things clearer for myself (and hopefully others). I will also argue that the VNM theorem has limited applicability.What are preferences?When reasoning about agent's behavior, preferences are a useful abstraction. Preferences encode epistemic information about an...

Published on February 24, 2025 4:59 PM GMTOne way in which I think current AI models are sloppy is that LLMs are trained in a way that messily merges the following "layers":The "dream machine" layer: LLMs are pre-trained on lots of slop from the internet, which creates an excellent "prior". The "truth machine": LLMs are trained to "reduce hallucinations" in a variety of ways,...

Published on February 24, 2025 4:51 PM GMTThis work was done as part of the MATS Program - Summer 2024 Cohort.Paper: link Website (with interactive version of Figure 1): linkExecutive summaryFigure 1: Low-Elicitation and High-Elicitation forecasts for LM agent performance on SWE-Bench, Cybench, and RE-Bench. Elicitation level refers to performance improvements from optimizing agent scaffolds, tools, and prompts to achieve better results. Forecasts are generated by predicting...

Published on February 24, 2025 3:51 PM GMTMy local gym is surrounded on all four sides by tent cities. At first I was nervous to walk there after work, but soon I got used to the tents and their residents.When I’m passing through on my way to the gym, I feel like I’m traveling from Rome to the Vatican—a city within a city.They have...

Published on February 24, 2025 2:20 PM GMTThis is a post in two parts. The first half is the post is about Grok’s capabilities, now that we’ve all had more time to play around with it. Grok is not as smart as one might hope and has other issues, but it is better than I expected and for now has its place in the...

Published on February 24, 2025 1:23 PM GMTeven if you're immortal, you have a countable infinity of time, but every choice you make leads to an uncountable infinity of possibility. you may have an infinity of breakfasts, but you still have to choose what to eat for each one. you can choose to dedicate your life to every possible variant of chocolate, but it...

Published on February 24, 2025 1:05 PM GMTPicture the following situation:You're in bed with a hot woman. Your clothes are off and so are hers. You (consentually) tie her wrists and ankles to the bed so you can have your way with here. She tells you to do whatever you want with her. And you think to yourself…This is exactly like what we do...

Published on February 24, 2025 5:53 AM GMTTrade surpluses are weird. I noticed this when I originally learned about them. Then I forgot this anomaly until…sigh…Eliezer Yudkowsky pointed it out.Eliezer is, as usual, correct. In this post, I will spend 800+ words explaining what he did in 44.A trade surplus is what happens when a country exports more than it imports. For example, China...

Published on February 22, 2025 4:59 PM GMTThis post is a summary of a paper recently posted here that describes, in my opinion, a very possible scenario that modern society will have to face in the near future.This post is not really intended for the average LW reader, as most of you probably read the original paper. I wrote this post for someone like...

Published on February 22, 2025 4:33 PM GMT[Cross-posted from my blog]WARNING: this post might press some pain-points of humans in general, and of rationalist community in particular. I do believe our triggers point to our blind-spots or dogmas – so maybe you can find here an opportunity for new depth. I was recently quite impacted by yet another very nicely written AI-apocalypse scenario. This isn’t...

Published on February 22, 2025 4:17 PM GMT [1]IntroTo everyone running an anniversary party, thank you. Someone had to overcome the bystander effect, and today it seems like that’s you. I’m glad you did, and I expect your guests will be too. This guide aims to give you some advice and inspiration as well as coordinate.The BasicsIf you’re up for running an anniversary party, pick...

Published on February 22, 2025 1:05 PM GMTIn my last post, I investigated potential severity and timeline of AI-caused disasters. This post goes into detail about something that could potentially precede such disasters: uncontrolled spread of AI.While AIs are often released as open source, you might at least hope that the AI developers think twice about releasing a genuinely dangerous AI system for everyone...

Published on February 22, 2025 9:39 AM GMTDate: Around the last week of May 2025Location: Science Park, University of Amsterdam (tentative)Organizers: Jan Pieter van der Schaar (University of Amsterdam), Nabil Iqbal (Durham University) and Karthik Viswanathan (University of Amsterdam)We are excited to announce a two-day workshop on "Interpretability in LLMs using Geometrical and Statistical Methods" during the last week of May (tentative), with around 20...

Published on February 22, 2025 11:54 AM GMTThere’s a type of perspective shift that can bring a lot of clarity to the behavior and limitations of algorithms and AIs. This perspective may be called seeing through the eyes of the algorithm (or AI, or LLM). While some may consider it obvious and intuitive, I occasionally encounter people – such as inexperienced programmers struggling with...

Published on February 22, 2025 11:54 AM GMTReligions can be divided into proselytizing religions (e.g. Mormons) who are supposed to recruit new members, and non-proselytizing religions (e.g. Orthodox Jews) who are the opposite. Zen Buddhism is a non-proselytizing religion, which makes me a bad Buddhist, because I've dragged three other people to my Zendo so far. All three had a great experience. One has...

Published on February 22, 2025 7:15 AM GMTBiological humans appear, across many domains, to have have an information throughput of at most about 50 bits per second. Naively multiplying this by the number of humans gives an upper bound of about 500 gigabits per second when considering the information throughput of humanity as a whole.Current frontier LLMs collectively produce around 10 million tokens per...