Will we have Superintelligence by 2028? With Anthropic’s Ben Mann

By the benchmarks, 4 is dramatically better than any other models that we've had. Even fine on it is dramatically better than 3.7 on it, which was our prior best model. Some of the things that are dramatically better are, for example, encoding, it is able to not do its sort of off-target mutations or over-eagerness or reward hacking. The new models, they just do the thing, and so that's really useful for professional software engineering where you need it to be maintainable and reliable.

So more agentic, longer-horizon tasks are newly unlocked, I would say. And so in coding in particular, we've seen some customers using it for many, many hours unattended and doing giant refactors on its own. But in non-coding use cases as well, it's really interesting. So, for example, we have some reports that some customers of Manus, which is a agentic model in a box startup, people ask it to take a video and turn it into a PowerPoint, and our model can't stand audio or video, but it was able to download the video. Use FMPEG to chop it up into images and do keyframe detection and maybe with some kind of old school ML based keyframe detector and then get an API key for a speech-to-text service. Run speech-to-text using this other service. Take the transcript, turn that into PowerPoint slides content, and then write code to inject the content into a PowerPoint file.

If you give Opus a tool which is Sonnet, it can use that tool effectively as a subagent. And we do this a lot in our agentic coding harness called cloud code. So if you ask it to look through the code base for blah blah blah, then it will delegate out to a bunch of subagents to go look for that stuff and report back with the details. And that has benefits besides cost control. Like latency is much better and it doesn't fill up the context. So models are pretty good at that. But I think at a high level when I think about cost, it's always in relation to how much it would have cost the human to do that.

I think we're going to start to get insight into what the models are doing under the hood from our work on mechanistic interpretability. Our most recent papers have published what we call circuits, which is for real models at scale. How are they actually computing the answers? And it may be that based on the mixture of experts architecture there might be specific chunks of weights that are dedicated to more empathetic responses versus more tool using or image analysis type of problems and responses.

Maybe I'll use coding as an example. So we noticed that our models were much better at coding than pretty much anything else out there. And I know that other companies have had code reds for trying to catch up in coding capabilities for quite a while and have not been able to do it. Honestly, I'm kind of surprised that they weren't able to catch up, but I'll take it. So things are going pretty well there for us. And based on that from a classic startup founder sense of what is important, I felt that coding as an application was something that we couldn't solely allow our customers to handle for us.

I read AI 2027, which is basically exactly the story that you just described, and it forecasts in 2028, which is confusing because of the name, that's the 50th percentile forecast for when we'll have this sort of recursive self-improvement loop lead us to something that looks like superhuman AI in most areas, and I think that is really important to us. Part of the reason that we built and launched Cloud Code is that it was massively taking off internally, and we were like, well, we're just learning so much from this from our own users. Maybe we'll learn a lot from external users as well.

I guess the way I define my metric for when things start to get really interesting from a societal and cultural standpoint is when we've passed the economic Turing test, which is if you take a market basket that represents 50% of economically valuable tasks and you basically have the hiring manager for each of those roles, hire an agent and pass the economic Turing test, which is the agent contracts for you for a month. At the end you have to decide, do I hire this person or machine? And then if it ends up being a machine, then it passed. Then that's when we have transformative AI.

So we pioneered RIF, which is reinforcement learning from AI feedback. And the method that we used was called Constitutional AI, where you have a list of natural language principles that some of them we copied from the Declaration of Human Rights and some of them were from Apple's Terms of Service and some of them we wrote ourselves. And the process is very simple. You just take a random prompt like how should I think about my taxes? Or something and then you have the model write a response, then you have the model's criticize its own response with respect to one of the principles and then if it didn't comply with the principle, then you have the model correct its response and then you take away all the middle section and do supervised learning on the original prompt and the corrected response. And that makes the model a lot better at baking in the principles.