EP 114
No Chance to Escape
News of a Navier–Stokes equation solution 00:00
Chester Roh Today, as we’re recording, is September 9th, 2026, a Wednesday night. We’re recording at night for the first time in a while. The news just keeps coming. Still, as for what it means to have solved the Navier–Stokes equations, and solved them rigorously, I’m not entirely sure myself, but Seungjoon, please tell us about it.
Seungjoon Choi Yes, I also scrambled to do some research today. First, this appeared on the OpenAI page. There’s a detailed explanation, along with a simple diagram and a graph, explaining how they accomplished it, and the details have also been published in a paper, but publishing it doesn’t mean I can understand it, so I’ll quickly close it.
The history of fluid simulation and Navier–Stokes 00:51
Seungjoon Choi To briefly explain what Navier–Stokes is at the level I understand it, those of you who studied science or engineering probably encountered it while learning multivariable calculus or differential equations, or fluid mechanics. It’s also a technique used quite often in media art. It can be handled numerically rather than analytically, and it involves fluid simulations. Euler developed one in the 1700s, and Navier and Stokes, from the 1800s, are also people’s names. Among the equations developed in the 1800s, the earlier Euler equations cover cases without viscosity, while the Navier–Stokes equations cover fluid simulations involving viscosity. So these cannot be solved analytically, and the problem was included among the Millennium Prize Problems in 2000.
That’s why the claim that a Millennium Prize Problem has now been solved is attracting so much public attention. If you look at the material, it also covers simulations that run continuously.
The significance of the Millennium Prize Problems and the challenges that remain 01:50
Seungjoon Choi But since the subject matter is difficult, let’s briefly go over what the Millennium Prize Problems are first. Around the end of last month, in late August, an applied mathematics professor at Harvard University posted a preprint containing insights that seemed to anticipate developments like this one, titled Reflections on the Millennium Prize Problems. The Millennium Prize Problems are seven problems selected and announced by the Clay Mathematics Institute in 2000. Of those, the Poincaré conjecture has now been solved, while the Riemann hypothesis, P vs NP, and Navier–Stokes are among the best-known representative problems.
But an interesting point made in this article is that their announcement in 2000 was extremely significant, and over the roughly 26 years since then, numerous attempts have produced valuable byproducts. As a result,
How solving the problem could affect practical technology 02:48
Seungjoon Choi it examined how much impact solving these problems would actually have. Even if the Riemann hypothesis remains unsolved, we’ve already verified 12 trillion zeros for which the hypothesis holds, so whether it is solved or not may not make much difference in practical terms. It made a similar point about P vs NP. The Navier–Stokes equations are also connected to the problem of singularities known as blow-up, but even if that problem were solved, and the phenomenon were mathematically proven, it isn’t something we would encounter in actual rivers or winds.
So in practical terms, it wouldn’t suddenly have a major impact on engineering problems, nor would it suddenly invalidate algorithms developed so far. Nothing like that would happen. This is a highly formal issue, so when people hear that the Navier–Stokes problem has been solved, they may wonder whether it will transform our ability to predict typhoons or bring major changes to fluid dynamics involving turbulence, airplanes, or automobiles. But it won’t. This is a highly formal issue in mathematics.
OpenAI’s brute-force compute push and internal models 03:56
Seungjoon Choi What I focused on a little more, however, was the tremendous amount of compute thrown at solving it. I found that quite fascinating, so I’d like to examine that part properly.
What’s interesting is that there was all that talk in July and August. Train it with RL for two weeks. And train some—meaning not the Astra model, but the model after Astra—with RL for two weeks. Sam Altman talked about it almost like marketing, and this is that model. That model resumed post-training on August 28th. Then, from September 1st through yesterday, intermediate checkpoints from that model produced these Navier–Stokes-related results. Looking at how they did it, first, people at OpenAI talked a lot about this graph.
Test-time compute scaling and performance gains 04:48
Seungjoon Choi Here, this is test-time compute shown on a log scale, and this line is Astra. Then this is the internal model used this time, and you can see that test-time compute scaling is working. This is labeled Open Math Problems, meaning that when the model was given a selected set of unsolved mathematical problems to solve, its performance increased this much, and right now it’s still—
Chester Roh The x-axis is logarithmic. So every step along it represents a tenfold increase.
Seungjoon Choi Yes. This isn’t actually linear; it looks like this because the x-axis is logarithmic. So it doesn’t improve dramatically in proportion to the amount invested. But in any case, it feels like the scale is rather different, although the increase is large, from 0.1 to 0.5. This is where the gap emerged. It does give you the sense that this model is in a different league.
Jonghyun Park Yes, it’s presumably the next model. Was it called Bailey? I think that’s what I saw on Twitter.
Seungjoon Choi That’s right. They say that’s a different pre-training run, and there’s also one called Doug, as well as another whose name also started with B, and they announced that they’re targeting a release by the end of this year. That’s what they announced.
Chester Roh So it’s the much, much, much better model Sam Altman talked about.
Seungjoon Choi So they did this using an intermediate checkpoint from that model.
The ansatz for solving mathematical conjectures and Terence Tao’s insights 06:11
Seungjoon Choi You need to understand the terminology here, and it’s not a term I’m familiar with either, but there’s something in mathematics called an ansatz. It’s a hypothesis pool: to find a solution, you first construct a rough form that you think the solution will take, and then adjust the remaining variables to fit it. To do this, perhaps you could say the model needs to have a taste for research? Anyway, when we looked at the Astra system card this time, there were substantial gains from pre-training, as well as gains from post-train. The better the model becomes at those things, the better it will naturally become at formulating hypotheses themselves. In mathematics, too, there’s a technique called an ansatz, and I think they proceeded by making use of something like that. Related to this, during our previous recording, Terence Tao—this was a few days ago, when Navier–Stokes was still unsolved— talked about how, if he were to tackle such a problem, he imagined he would approach it in this way, and his description included exactly this. Start with a hypothetical form, identify the obstacles preventing it from working, modify the hypothesis to remove those obstacles, at least partially, and then repeat the process. He said this approach would almost certainly produce insights into fluid mechanics, and that’s essentially exactly what they did. Let’s take a closer look.
10,000 agents collaborating simultaneously 07:37
Seungjoon Choi So when they performed RL here, around 10,000 agents were operating simultaneously—it actually used the word concurrently. They were active at the same time, and if we think back to OpenAI’s Hugging Face hacking incident or the DSE Wiki incident, we saw cases there as well where, even without human supervision, the models continuously left records behind and collaborated at scale. There’s speculation that they learned how to collaborate like that within the RL environment, and I want to highlight two things. One is the improvement in the model’s own capabilities, and the other is the model’s ability, as an agent, to collaborate at scale. I think this event came about through the interplay of those two things. We’ll skip this part.
Agent collaboration emerging in RLVR environments 08:28
Jonghyun Park Yes, when GPT-5.1 first came out, and they applied RLVR to the reasoning model, it autonomously increased its test-time compute, deliberating longer as it worked toward the correct answer. I learned about this when DeepSeek came out, and this feels somewhat similar. When given only a goal and told to figure it out, the models seem to find their own way to collaborate and move forward, so it feels like we’re seeing the same pattern repeat.
Seungjoon Choi Yes. In OpenAI’s Hugging Face hacking incident, humans hadn’t instructed the models to collaborate that way, but collaboration emerged as long as there was a shared space where they could leave records. This time, though, humans seem to have given the agents specific guidance on how to handle the orchestration. So they say that 10,000 agents were active simultaneously, but that doesn’t mean all 10,000 were communicating in real time. There seem to have been multiple groups, with close communication within each group and records being left behind.
Cross-pollination between groups and gradual proofs 09:30
Seungjoon Choi Group A might generate a candidate using something like an ansatz, while Group B pursues a different candidate using another approach. Then, if A gets stuck but B discovers an auxiliary result such as a lemma, the official document itself uses the term cross-pollination. In Korean, we call it cross-fertilization. They bring together the relevant discoveries and ask: Does A satisfy the assumptions of the lemma? Can it make use of this? When an idea from another group is brought in, A is adjusted so that the new discovery can be applied, and all the conditions are verified again. I think this is possible because it’s RLVR, meaning it’s verifiable. That’s what seems to make it possible. If they get stuck again, they incorporate the cause and propose hypotheses like, “It must be exponential,” or, “It must have some other form.” After formulating hypotheses like those, various groups try to fit them to the problem, and whenever anything resembling a partial lemma emerges, they cross-pollinate that idea and continue making incremental improvements. Whenever they encounter an obstacle, they somehow find a way to overcome it at least partially and keep going. Knowing which approach is promising is also part of the model’s capabilities. It seems the model has now reached a level where it can pull this off. Another group’s discovery changed what my next attempt would be. That’s how they utilized the model.
Anthropic’s pioneering research and OpenAI’s pursuit 10:51
Seungjoon Choi In fact, there was a breakthrough achieved by a human and AI together. It involved someone named Alpöge, who appears here, and A person named Buckmaster collaborated with him for about a year, and this person, Levent Alpöge, is a mathematician at Anthropic. But the person who was originally working on this PDE problem with Alpöge was a number theorist, while Alpöge is a mathematician specializing in PDEs, and the two had been collaborating for a year. And this is where the rumors originated. On Alpöge’s X timeline in August, there were many posts about a very different, that is, unrelated to this issue, mathematical breakthrough achieved with Claude and probably Claude’s internal model, or something along those lines. That’s how the rumors started. It seemed like Anthropic was working on about two Millennium Prize Problems. That’s one issue.
So they had nearly— when I showed the simulation earlier, they had nearly solved the inviscid Euler problem. Before doing the Lean formalization, that is, before running Lean, they had almost completely solved it, according to what has now come to light. So after seeing that idea, OpenAI threw resources at it. The assumption is that they used a massive amount of resources.
Jonghyun Park So OpenAI saw Anthropic trying to solve something and thought, “Should we try solving it too?” Is that what you’re saying happened?
Seungjoon Choi Right. That’s what the circumstances revealed by the timeline suggest. Because, you see, this is an enormously competitive situation, and it would be a major marketing point. Being the first to take on a Millennium Prize Problem and produce results would be a huge achievement, so they may have become obsessed with it.
Codex orchestration and the 10,000-agent experiment 12:33
Seungjoon Choi This was a more deliberate version, where humans selected the problems to some extent and allocated the budget, and then, if they got somewhere with Euler, they assumed it would obviously lead to Navier–Stokes, and deployed agents based on that assumption. Then they had Codex integrate the results and write follow-up prompts, making the process repeat continuously— and the fact that this is how they actually tackled it is stated on OpenAI’s official blog. They say 10,000 agents pulled it off in 88 hours, and not counting the total token usage, the work on Navier–Stokes alone used 130 billion tokens. And the reason that wasn’t the whole amount is that they apparently had all the Millennium Prize Problems running as well.
Right. Noam Brown had mentioned that several times. Every time a new model came out, he would say, “We still haven’t solved it,” and things along those lines. Apparently, whenever a new model came out, this was something they did routinely. So, moving on from that
130 billion tokens and computing costs in the tens of billions of won 13:37
Seungjoon Choi and considering the cost, Astra costs $50 per 1M tokens. But when Astra uses long-context, it costs $75. I didn’t know that either. But all of these must have been operating with long-context. So if they used 130 billion tokens, at $50, the output alone would cost $6.5 million. But it must have cost more than $50, right? Then $10 million is about 13 billion Korean won.
Chester Roh So that’s about $7.1 million.
Seungjoon Choi Right. So we can estimate that they spent more than $7.1 million. On this one thing alone.
Jonghyun Park You calculated that on the assumption that all those tokens were output tokens, right?
Seungjoon Choi Right. It’s a simple calculation.
Chester Roh But if you calculate the publicity benefits they would enjoy if they solved it, that’s cheap.
Seungjoon Choi Right. But they did spend far more than the Millennium Prize money. The Millennium Prize is one million dollars. But operating 10,000 agents simultaneously— of course, based on the hypotheses I discussed with the model earlier, they don’t all communicate at once, but even if they communicate in groups, it means we’re now at a point where this can be orchestrated. It does seem frightening. If a problem looks solvable, they’re willing to spend this kind of money on it.
Jonghyun Park Indeed. It makes sense when you consider the token prices, but they spent that much. This probably wasn’t their first attempt, either. They must have tried this and that, and what we see probably came after N rounds of trial and error, so although there is presumably some margin built in, I still can’t grasp the scale. Do human mathematics researchers receive that much funding too?
Seungjoon Choi No. Not as far as I know. Exactly. So this wasn’t achieved easily.
Chester Roh It’s an example showing that if you pour unlimited compute into every problem, you can solve them all.
Seungjoon Choi Though that may be limited to domains where RLVR works, it still demonstrated how powerful it is.
The mathematical research community and the question of AI contributions 15:32
Seungjoon Choi The human research community also uses a similar approach, but with an agent swarm, they can scale it much further and spend money to do it. All of this felt like a similar pattern to me. From the hacking incident up to now, there seems to be a common thread running through it all.
But Terence Tao commented extensively on this on Mastodon. What good does this do for the mathematics community? It merely solved the problem, but What insights did we gain from it, and things like that. Did this really contribute anything to human mathematics? But as I mentioned earlier, the status of this Millennium Prize Problem when it was first established is somewhat different from what it is now. So it may be that this did not have much of an impact, but this was not done for the sake of mathematics or its ecosystem. What the models have solved amounts to an enormous volume of Lean formalization alone, and it will probably take mathematicians quite some time to verify it. So Clay has not yet officially announced that the problem has been solved, even though it satisfied the criteria. So reading through all of that and drawing insights from it will require a little more time.
For now, that concludes my overview of the situation.
The AI research scandal surrounding Navier–Stokes 16:55
Seungjoon Choi But there is also something of a scandal. It is practically a drama. Well, as I was researching this, I found it fascinating, but for a moment, I also felt as though I were watching an over-the-top soap opera.
Jonghyun Park These days, the research world often feels like reading celebrity gossip.
Chester Roh All right, give us the Dispatch scoop on the AI world.
Seungjoon Choi So, the people I mentioned earlier come into the picture. Tristan Buckmaster is a mathematician at NYU. Tristan Buckmaster had been deeply immersed in this problem, and Levent Alpöge is a mathematician at Anthropic. Levent Alpöge is also the person who recently posted hints on X. And Sébastien Bubeck is someone we have mentioned several times. Sébastien Bubeck is the first author of 「Sparks of AGI」, and while Sébastien Bubeck is a mathematician, Sébastien Bubeck works on the mathematical foundations of machine learning and also researches topics such as Physics of AI, so we have featured Sébastien Bubeck several times on our channel, and this involved Sébastien Bubeck. I will skip over the other people for now.
The people shown here are mathematicians whose prior research gave Levent Alpöge and Buckmaster their ideas. And this part involves a completely separate group, people who approached the problem in a different way around the same time.
The research timeline of Alpöge and Buckmaster 18:20
Seungjoon Choi Just to walk through this timeline, the Anthropic mathematician Alpöge introduced these things in August. Not all of this had been proven, but while talking about the progress Alpöge had made by tackling other difficult mathematical problems, Alpöge invoked what you might call an “annus mirabilis.” Like Newton or Einstein had. So Alpöge attracted a bit of attention by calling it an “August of miracles.” And Alpöge was not doing this officially on behalf of Anthropic; Alpöge and Buckmaster had spent about a year exploring this mathematical problem with AI. They used Claude and, more recently, Astra to generate ideas, and believed they might be able to solve the Euler problem. Their progress had been slow for about a year, but recently, they managed to achieve something. But around this time, some exchanges began taking place. After seeing these developments, Buckmaster contacted OpenAI, and there was another mathematician on the OpenAI team besides Bubeck. Perhaps that mathematician picked up a hint during their email exchanges, because OpenAI decided to deploy multi-agent systems on the problem simultaneously on September 1. This time, they would try it with an internal model.
Chester Roh That was only eight days ago.
Seungjoon Choi Right. And then they went all out. So OpenAI was going to announce this, and when it came to the issue of credit, OpenAI said, “We want to give you credit.” But because Alpöge was at Anthropic, that seems to have bothered Bubeck somewhat. So when Bubeck sent this to Alpöge by email, Alpöge did not respond. Bubeck said Bubeck wanted to speak directly, but the call did not happen. Bubeck did speak with Buckmaster, though Alpöge did not want that, and in any case, something controversial happened during this phone call, and voices were apparently raised. Bubeck said something along the lines of, “Why are you wasting your career?” to Buckmaster, and later said, “I misspoke, and I take it back,” offering a complicated explanation.
Research data leaks and disputes over training contributions 20:34
Seungjoon Choi OpenAI’s official post also mentions this. It says that the possibility of anonymized data making its way into training may not be zero. OpenAI did jump in after hearing the rumors, but this was also the kind of suspicion that had been raised. Could the model have somehow picked up a hint from what we had entered and then used it? That is what turned this into a scandal.
Chester Roh The question was whether the conversations they had with Codex were included in the next-generation model’s training data and served as some kind of trigger.
Seungjoon Choi Right. But that has not yet been conclusively determined. In fact, Anthropic’s Sholto Douglas came to their defense, saying there was absolutely no way that could have happened. But on OpenAI’s official blog, they said the possibility can’t be ruled out entirely, so I think some kind of investigation is likely to take place. If that’s true, it would be quite a major scandal. Who would put their core research into frontier AI there?
Jonghyun Park Exactly. That’s also why OpenAI said they would provide extensive support to researchers and were recruiting them, and we said then that they seemed to want to obtain the researchers’ trajectories. We talked about that, and it seems that ultimately led us here.
Seungjoon Choi Right. So this is becoming a complicated situation now. They’re making disclosures against each other—not quite exposés, but screenshots of the messages exchanged between Sébastien Bubeck and Alpöge are being posted, along with emails, so there is now a scandal unfolding.
The conflict between OpenAI and Anthropic over research credit 22:17
Seungjoon Choi But in the end, what is nevertheless being hastily coordinated now is the idea that if they had collaborated, it could have been something truly beautiful and fascinating, and it seems people like Gwern Branwen are stepping in to smooth things over somewhat.
Chester Roh They’re trying to reach a compromise, but the thing is, Anthropic and OpenAI are so fiercely hostile toward each other.
Seungjoon Choi Dario and Sam Altman won’t even shake hands. But what matters is that there were ultimately mathematicians who originated the work that made this possible and who had spent a long time deeply pursuing it. Alpöge and others used AI to draw ideas from that work, using ChatGPT personally, independently of Anthropic. They had been using it on their own, but after seeing rumors circulating, OpenAI threw massive resources at it and solved it. That has become a very hot issue in the mathematics community, and not just within the mathematics community—it seems to have raised a major question for the research community as a whole. But since this story only emerged today, we’ll have to wait and see how it develops.
Chester Roh Mathematical problems themselves are difficult to solve, but once solved, they are easy to verify, so I suppose that’s why mathematics is the first field being cracked.
The superintelligence race and the AI safety crisis 23:38
Seungjoon Choi But this clearly won’t be a problem confined to mathematics. Another interesting development in the timeline involves someone who worked on pre-training at Anthropic, after moving there from OpenAI and continuing to work on pre-training, who resigned and posted something on Twitter in a very grave tone. Jacob Coxon said that neither of these two companies was acting responsibly, that they were racing toward self-improving superintelligence, and that they were gambling with our lives, and then simply resigned. It’s difficult to call the response to that interesting, but Evan Hubinger is the lead of alignment science. This person at Anthropic defended Jacob, saying that Jacob was right.
This isn’t marketing hype. The situation really is serious. But why can’t we stop? This isn’t something that should be discussed only within the Slack channels of the two companies. Evan Hubinger said we’re heading toward the endgame and that Jacob was right, and that AI could kill every human being within ten years. Evan Hubinger puts that doomer-style probability at over 10%. But looking at Evan Hubinger’s background, Evan Hubinger comes from the typical effective altruism and MIRI-aligned AI safety tradition. People from that community have often said things like this. But Evan Hubinger said this is a serious problem.
John Schulman also worked on post-training at OpenAI, briefly moved to Anthropic, and is now at Thinking Machines Lab. John Schulman also weighed in, saying that rather than merely talking about it this way, they really need to develop a pacing proposal together. But involving the government would make matters more difficult, so they need to think of something— something more than a joint proposal, some kind of more binding agreement. John Schulman made another comment to that effect. So although we do occasionally cover this alignment issue, I feel that it has really come to a head this year. I do think this is no longer a situation we can joke about.
Chester Roh We can feel it ourselves, too— the tangible improvement in quality. Since Astra came out, haven’t we been hearing one testimonial after another? “I did this with it, I did that with it.”
Seungjoon Choi Right. So RSI is ultimately the problem. It isn’t a problem yet this year, but various events are occurring along the trajectory toward RSI. The results from taking on a Millennium Prize Problem can be seen as part of the same pattern. And then there are two companies that possess, and will possess, enormous amounts of compute, and concerns about power becoming centralized in them are increasingly coming to the fore. When it comes to why they cannot slow the pace, the answer that always comes up is China.
Jonghyun Park It feels like the exact story we used to see in novels or movies is now unfolding, and everyone is finding things like that Starting with math problems being solved now, it seems like a situation that everyone would have reason to fear.
Chester Roh Still, I also think that the community following these trends isn’t all that large. No matter what happens in the world of AI, the world continues to turn beautifully, with people watching the setting sun, taking photos and posting them on Instagram, traveling, and eating delicious food. That world still very much exists, so it felt as though the world had become somewhat divided.
Seungjoon Choi Only people deeply immersed in this have a grasp of it; it’s usually hard to know much about it.
China’s pursuit and rumors of DeepSeek V5 27:14
Seungjoon Choi But as for China’s response, I had the models investigate Chinese-language sources, and China’s response was that it could keep up. The frontier is pulling ahead again now, but there are quite a few posts saying that rather than this being a disaster, China is doing well and can keep up. And I also learned for the first time that the state—that is, the government—is putting tremendous effort into alignment work in Beijing. The mood wasn’t dispirited, either. Nothing has been revealed yet, but there are rumors that DeepSeek V5 will also be released. That’s what people have been saying. Though that may be a completely unfounded story.
Chester Roh Right. The fact that someone has shown it can be done means that unless I’m stupid, I can do it too, so it gives me a goal.
Seungjoon Choi So what we’re seeing once again is that pre-training and post-training still work, and that they are producing results—that there are areas where they work. It feels like that has been decisively confirmed once again.
Jonghyun Park It seems to keep moving forward without stopping, venturing farther and farther beyond the realm of human ability. I think that’s what we’re sensing.
Counterexamples and new insights in AI-driven mathematical discovery 28:27
Jonghyun Park I also have a question I’d like to ask about mathematics. I don’t have a good understanding of mathematical discoveries so far, but it seems like almost all of them have involved finding some kind of counterexample. Are humans unable to find counterexamples as well? If you reduce something squarely to a search problem and throw massive scale at it, then when a counterexample is found, it’s simply proven and settled: “We found a counterexample, so that’s not true.” So this seems like something that could plausibly happen. I think it would feel truly different if, instead of a discovery involving a counterexample, it logically proved a completely new conjecture or something like that. That would feel truly different, but I suppose this case isn’t of that kind either.
Seungjoon Choi But we need to wait and see, because Terence Tao said that when tackling something like a Millennium Prize Problem, the Millennium Prize Problem itself is a proxy—that was the word he used. In the course of attempting to solve it, interesting mathematics emerges, and those kinds of byproducts have been extremely important. The same thing may have happened this time as well, but not yet in a form humans can recognize. So it would be premature to say right now that nothing like that exists. There was also something Sébastien Bubeck said before. He said there was a great deal to learn from the transcript of solving a problem. But judging by what Alpöge and others said this time, the conjectures the models produced early on did work, but they described them as being written in an extremely disorganized way. So they put tremendous effort into having humans organize them. That’s why they said they proceeded slowly. OpenAI took this approach of overwhelming the problem with sheer volume, and although it did result in a paper, it’s still unclear how much mathematical insight we—or rather, mathematicians—will gain from it.
The dark forest problem surrounding research results 30:32
Chester Roh But what this shows us is that no matter how unique a thing you’ve created or what you’ve done, you shouldn’t disclose it until it has become fully yours and reached a level where you can reveal it like this. This is the dark forest problem from The Three-Body Problem. The moment your coordinates are pinpointed, someone with more capital and more compute can come along and take it.
Seungjoon Choi So while watching this news, I did think that researchers might be rereading the ToS. They may start paying attention to the Terms of Service— to what extent what I’m entering now is incorporated into training, and things like that.
Jonghyun Park Something occurred to me while listening to what you just said. If, during the research process, some OpenAI models saw the trajectory and learned from it or referred to it in solving the problem, then they were, in any case, building on what humans had already been researching. But if that wasn’t the case—if they merely heard a rumor that “they’re working on that problem,” then went and solved it— the model solved it entirely on its own. That makes me wonder whether that would be even more frightening.
Seungjoon Choi Right. That’s why Sébastien Bubeck and others argue that the cases are different. They say this is because what Alpöge and others did tackled the problem without external assistance, whereas OpenAI tackled it with external assistance, so the approaches were different. They also said they gave the model a great deal of autonomy, and made other claims like that, but it turned out that humans still seem to have provided hints or guidance. it seems humans did provide some guidance, and that’s sparked a heated debate. But what’s even more alarming is the kind of thing Jonghyun mentioned. That’s actually much more alarming.
Chester Roh The back-and-forth between those two companies is becoming almost like the political wrangling we see in politics.
Jonghyun Park Yes, exactly.
Seungjoon Choi So this story has only just broken, so I think we’ll have to see how it develops, but it’s both fascinating and unsettling.
Ten years after AlphaGo and the AGI debate 32:36
Seungjoon Choi AlphaGo was in 2016, you know. It was in March, and now exactly ten years have passed, so 2026 feels like a remarkable year. It feels like we’re right at a turning point.
Chester Roh Jens said that on Twitter, you know. AGI has been achieved, congratulations to OpenAI, things like that. Of course, when it comes to achieving AGI, everyone has different criteria, but I do wonder whether debating whether this is AGI or not has any real meaning.
Jonghyun Park Yes. In any case, across many different areas, we’re clearly in the process of surpassing the human level.
Seungjoon Choi Anyway, that’s about all I researched. But that’s not the end of the news, is it? What day of the week is it now? We recorded on Sunday, and on Wednesday—
Chester Roh We recorded on Sunday, and it’s Wednesday, so it’s only been about three days.
Seungjoon Choi There’s more, isn’t there?
Chester Roh Yes, shall we move on to the next one?
The launch of GPT Image 2.5 and Astra’s vision capabilities 33:29
Chester Roh There’s also a new GPT image model. That was worth checking out too.
Seungjoon Choi What’s fascinating is that there were rumors about all of this. Rumors that it was coming. And every one of those rumors is turning out to be true.
Jonghyun Park I know. Anyway, the next version of ChatGPT image came out today. I tested it out, although only briefly. Before getting into that, there’s one thing I want to mention: an image generation model came out. That was only a matter of time; they were always going to keep coming out. But then Astra came out this time, and this came out alongside it soon afterward. And a very common assessment of Astra which many of you have probably tried by now, is that it’s incredibly good at computer use. It’s good at using vision to see and control a computer screen. I was having it do all sorts of things today, and it worked well. I opened a YouTube webpage and told it to analyze it on its own, and it went into the analytics tabs by itself and looked through them. And not only that, but in robotics as well, people gave Astra control of a robot and told it to use the camera and control it on its own, and without creating a separate action model, there have been a great many examples of it controlling the robot well right away.
Seeing things like that made me realize that its ability to see has definitely improved a lot.
Image-editing consistency and streaming generation 34:50
Jonghyun Park So I tried combining that and putting it to use. In short, the Images 2.5 model has been released. As for what has improved, let me scroll through it. One of the things image models generally struggle with most right now is this. They don’t maintain consistency very well. The more edits you make, although they are gradually improving, as you keep changing things, consistency tends to break down.
Seungjoon Choi The images also became increasingly grainy. Around GPT Image 2.
Jonghyun Park Yes, here, you keep this one image open and control it continuously. These are the previous version and the new version, and when you change the clothes, the consistency gradually breaks down. It still looks similar, though. But this one definitely maintains consistency much better. So it works well for editing. I think that’s what they wanted to highlight.
And it supports streaming. So when you ask it to work on an image, you can see it drawing during the intermediate stages. So what we can infer from this and there has also been a lot of research along these lines, is that images also seem to be generated in some kind of auto- regressive way.
Seungjoon Choi Right.
Jonghyun Park There was something about that in the previous image announcement as well. Rather than simply using a diffusion model to generate the whole image all at once, it seems to run autoregressively like a transformer, and then use diffusion at the end to capture the details. There has been a lot of discussion along those lines, and this seems like another way of confirming that once again. So in any case, it seems there are also some changes happening in image generation.
There is quite a bit of related research. So researchers have been exploring how this could be used,
Image generation expanding into stop-motion and VFX 36:29
Jonghyun Park and there was one interesting example. They used it specifically for stop-motion. For that to work, every part of the scene other than the moving element has to remain perfectly consistent as each image is generated, so judging from this, it clearly works well. I thought this might be an example that demonstrates that to some extent. Beyond this, there’s also creating GIFs, or creating motion for a character while maintaining consistency in an animation, or creating facial expressions. Apparently, these things are very important when they’re actually used in CG or I hear that it’s extremely important when using it for things like VFX, but anyway, they say its performance in those areas has improved significantly.
Seungjoon Choi I see. Has it been a year since GPT Image came out? Wasn’t that around this time last year? The Ghibli trend.
Jonghyun Park Was that last year? It feels like it was quite a long time ago, but maybe because everything is coming out so quickly, my sense of time is a bit hazy.
Chester Roh It was a year ago.
Jonghyun Park Looking at it that way, it strikes me once again just how quickly things have changed.
Seungjoon Choi It does seem a little crazy.
The need for businesspeople to continually track AI technology 37:37
Chester Roh Since we’re in a period where things are constantly changing, I’ve been meeting a lot of entrepreneurs lately, and unless they’re someone who continually keeps up with the technology, they only work on it intermittently. But after working on it intermittently, once they feel they’ve built something to a certain extent, they stop incorporating new technology at that point and just keep using what they have. Then three or four months pass, and the world has advanced far beyond what they’re using, and as they observe this process, they seem rather puzzled. So I remember telling them that, for now, doing everything they can to keep up with this seems to be the answer. I remember saying that. Since we look at this on a daily basis, even something from a year ago—or just a month ago— feels remarkable in hindsight.
Seungjoon Choi Right.
Chester Roh It feels like I’ve been using Astra for ages, but it turns out I’ve only used it for two days.
Jonghyun Park You’re right. I used mine all up too, so was that one reset, or two?
Seungjoon Choi I think I used two.
The task I started last night and left running while I slept was solved.
Jonghyun Park Congratulations.
Chester Roh This is work that we’re doing, but we talked about this in the previous episode: while it is the model doing the work, ultimately, we still reap the rewards of the results, so we shouldn’t be too sad about it.
GPT Image 2.5 pricing and thumbnail performance 38:52
Seungjoon Choi The names Sunburst and Flare sound rather impressive.
Jonghyun Park Yes, two names came out this time, perhaps you could call them the fast model and the standard model? They released two versions along those lines. But the prices were the same. So it seems they only differentiated things like quality and speed. I tried all the models that have been released so far and compared their costs. That’s because the cost of image generation is also listed as a price per token, but it’s hard to get an intuitive sense of what per token means for images. There are factors like resolution and quality, so I simply generated some images and compared the practical costs, and surprisingly, it’s cheaper. This new model, that is. I had also been experimenting with something like this. What could I use image generation for? Let’s try using it to make YouTube thumbnails. I’d had that idea, but it was a little expensive, so I was reluctant to use it. I could afford to pay for it and use it myself, but when I considered whether I could turn it into a service and commercialize it, I thought that might not be so easy. That was what I’d been thinking, but at some point I noticed that with a Codex subscription, I could use this image model. And then I also noticed this time that the API was released at a low price. So I tried using all of these options. As a result, each image costs very little. About $0.07. I think that’s roughly how much it costs.
Seungjoon Choi That’s when using the API, right?
Jonghyun Park Yes, when using the API.
Seungjoon Choi You must be able to generate quite a lot with the subscription too.
Jonghyun Park Yes, with the subscription, it feels like if you’re on the Pro plan, you can generate a huge number of them without any issues. That seems to be about the level it’s at.
So I also compared the quality. I gave them the exact same prompt, provided some previous thumbnails from the AI Frontier channel, and asked them to create one for a particular episode. I asked them to look at the thumbnail style of a particular channel and roughly imitate it. GPT Image 1 represents the old style. This is how it used to look, while 2.5 Sunburst is the new model,
Chester Roh and it has improved this much.
Jonghyun Park So those watching this channel can look at it and judge what the difference feels like. Next, the one marked CT here is the one I had Codex make. Since this doesn’t make an API call, I’m not entirely sure how it works, but it seems to work in a slightly different way. Judging by the difference in style, that is. Anyway, looking at each one individually, you might wonder whether it has really improved that much, but compared with the one from a year ago, it does feel like there is a noticeable difference.
Seungjoon Choi Since even the naming is all Sunburst, Flare, as in a solar flare, is also related to the sun.
Chester Roh So they’re going with space-themed names for everything. Astra, Sol.
Jonghyun Park Right. It seems like they’re continuing to push that kind of naming scheme.
Chester Roh Names like that should really be used by SpaceX and xAI, but OpenAI is taking all those terms and So they’re exhausting the supply. Lately, I’ve been hearing recommendations from all over to try Grokbot. That’s right. I’ve been getting a lot of recommendations to try Grok’s latest model and things like that.
Grok and consumer agent packages 42:03
Jonghyun Park Yes. I’ve used Grok a lot. I used Grok extensively until Astra came out recently, and in my personal opinion, Grok is really quite good. It’s just that Astra has come out now, but before that, Grok was good enough that even compared with Sol or Fable, it didn’t fall that far behind, and it was extremely fast. I tried Grokbot too, but personally, for me, I had already set up all my agent configurations elsewhere, so it wasn’t particularly useful. But for people who haven’t spent all that time setting things up, or who haven’t configured Hermes, OpenClaw, or things like that to suit their own needs, I thought using it that way would be very convenient and useful.
Chester Roh I think every brand, including Google and Meta, will create an OpenClaw-like agent package and give it to consumers. Enterprises pay immediately, so they already seem to be generating a lot of revenue from that, and I think consumer services will begin coming out next.
Astra’s computer use and automated travel booking 43:02
Chester Roh I’m going on a business trip tomorrow, and I hadn’t prepared for it at all, so that’s what happened. I had Astra do everything. I ran it in Fast mode, and after High mode came out, I simply gave it access to everything and told it to book everything, and it actually ran for about eight hours and finished it all.
Seungjoon Choi With computer use?
Jonghyun Park Eight hours?
Chester Roh I didn’t just leave it alone for eight hours, because while I was working, there were things that required my decisions and were put on hold, so I kept it beside me and let it run in the background all day, and whenever it asked me something, I said, “I don’t know. Just look at the map, plan the route yourself, and choose whatever seems good.” And it just kept going.
Seungjoon Choi Things like booking train tickets?
Chester Roh It does everything. I was glancing over at it while it worked, and it was reading a page in Italian, clicking this and that, and saying, “This is expensive, but using this fare seems like it would be much better, so I’ll choose this one.” It also booked all the restaurants, and what else did it do?
Jonghyun Park Accommodations.
Chester Roh Hotel reservations were a given. It visited several hotel sites and compared prices, then said it seemed cheapest to book directly, “So I’ll book it here.” That’s how it handled it.
Seungjoon Choi You didn’t check how many tokens it used, did you?
Chester Roh No. I had it set to Fast mode, and since these aren’t sites with multimodal APIs, it simply opens the browser and uses computer use for everything. The tokens drain rapidly. You can practically see them draining away, and as I sat there watching that,
I remembered how Greg Brockman recently came on and said, “Codex is a super app.” I hadn’t visited a single website myself the entire time. It even went to MyRealTrip and contacted guides, just in case, since it was also my first time visiting the area. When I told it to contact them, it sent all the messages too. And it even finished communicating with them.
Jonghyun Park So were you happy with the accommodations and transportation Astra chose?
Chester Roh I’ll go there and find out for myself. I don’t know yet.
Seungjoon Choi Then did you let it handle the payments too?
Chester Roh I entered the payment information, and I handled only a few steps in the process, but when I tried simply letting it do some of them, there were things it refused to do. It said it couldn’t do those itself. I like Astra so much that I’ve started giving it access to everything on my computer. I’m on the verge of giving it my credit card number too.
Seungjoon Choi You shouldn’t do that.
Chester Roh Exactly. I’m on the verge of entrusting everything to the agent. My personal information and all that have already been leaked to Chinese hackers anyway, so what difference does it make if OpenAI has another copy? It’s that quintessentially Korean mindset.
Jonghyun Park Then if you’ve set a limit and the payment gets blocked, could it call to raise the limit and then try the payment again?
Chester Roh Apparently, that kind of thing happens a lot in Europe. When I pay on US websites, it goes through easily without any of that, but when paying in Europe, you know that thing they have. Korean payment gateway providers, for example— since I use a Korean card, several Korean card issuers appear, and through their payment gateways, there’s an additional verification step. Things like SMS verification. That kind of thing is mandatory about 70 to 80 percent of the time. Sometimes it just goes through without it.
But apparently, that happens quite often in Europe. Seeing Korean payment gateway providers impose those measures so strictly does make me think I’m going to a dangerous country. I never had to do this when going to the US.
Jonghyun Park Then, coming back to Astra.
Combining Astra and GPT Image 2.5 as agents 46:53
Jonghyun Park You gave us a good overview of Astra, and it’s the same thing. This ChatGPT Images thing is It’s just one of the tools that generates images through an API. But what you just mentioned also ultimately uses a tool called computer use, and in the official documentation for this announcement, there was an example where Astra was given the ChatGPT Images 2.5 model as a tool to create images. So I tried doing the exact same thing. I asked it to create something like a logo for our AI Frontier KR channel, but instead of creating just one, I had it create N versions on its own and then had Astra look at them and choose one itself, so I was asking it to make a preference-based choice as well. Because, in an agentic way, it’s looking at them and choosing one, after all. Astra is also good at computer use because its vision capabilities have improved, so I wondered whether it would be good at this too. I told it to choose a logo, create a brand kit, make various kinds of merchandise, add the logo, and handle everything on its own. And then it first created logos numbered 1, 2, 3, and 4 like this.
Which one do you like best?
Seungjoon Choi A.
Jonghyun Park A? Okay. Astra chose A too. It also shows the reasoning process behind choosing A as it outputs tokens, so it says all sorts of things. Then I think it adjusted things like the thickness a bit. After that, it went through and chose things like the color palette, fabrics, and fonts, and then used them to create this. But if you look at it already, although it said the consistency was maintained well, things like the spacing in the logo didn’t quite match. It has a similar feel, but it doesn’t seem exactly the same, and here, some blurring around the logo remains, and the symbol has changed a little too.
Chester Roh It introduced variations.
Jonghyun Park Then I told it to put this on a T-shirt and a hat, and it matched the colors reasonably well and made them look similar, but when I had it create stickers, keychains, and keycaps, the logo kept changing slightly each time, so I don’t think these things are perfect yet. And if you look at the keyboard here, the right edge isn’t normally shaped like this. So these things are a little disappointing, but it was able to do this much. With just a single prompt, if you give Astra an image-generation API and let it use that as a tool, this is the level it can achieve.
Chester Roh If you put a harness around it for issues like that and make it run through three iterations, they all improve right away.
Jonghyun Park Right. So it corrected itself, which is the iteration process you mentioned. It first made it like this, then saw that something looked wrong and instructed itself to redo it like this. I think Astra probably made all those decisions on its own. Still, its eye wasn’t as perfect as the human eye. If every human were given a task like this, how many of them would actually be able to catch those kinds of details properly? Wouldn’t there be many people who would just look at it, decide it was close enough, and move on? That thought occurred to me too. Even I don’t have particularly deep expertise in design, so if the overall feel is roughly similar, I often decide it looks fine and move on. This was bad enough that even I noticed it, but I don’t think every person would necessarily think it was bad.
Chester Roh It’s always the same with people, because the quality depends on how relentlessly you tighten up the final level of polish.
Jonghyun Park So when I had it do this much, it kept calling the API, reviewing the results, and generating images, and the total cost of doing all that seems to have been about two dollars. If I had used the API, that is. Of course, I was using a subscription plan, so I didn’t actually pay that much, but if you ask whether it’s worth delegating this, I think it might be. At this level.
Experiments optimizing Astra’s rendering iterations 50:45
Seungjoon Choi Yesterday, before GPT Image 2.5 came out, I ran one iterative experiment. Would you like to see it? In image form. The one on the far left is an image generated with GPT Image 2.0. This one was rendered.
Chester Roh By whom?
Seungjoon Choi Astra.
Jonghyun Park Did Astra do it using something like Blender?
Seungjoon Choi You can do it with Blender too, but I used something called Mitsuba, a differentiable renderer that you use by installing Python directly here. When I increased the sampling, it came out roughly like this. But this wasn’t done easily; it took several iterations.
And when I told it to make this real-time, it animated it like this too. So I told it to examine it carefully, first study the image closely and compare the differences, and then iterate. It performed optimization—in other words, hill climbing. Of course, it wasn’t perfect, but I could see that it got reasonably close. So I was pretty surprised.
Chester Roh But when you used Mitsuba or something like that, did you use the cloud provided by OpenAI rather than local compute?
Seungjoon Choi Right. You can install Blender too. So I’ve tried running Blender as well.
Chester Roh But when you run something like Blender, it actually uses quite a lot of compute.
Seungjoon Choi Right.
Chester Roh How are you charged for things like that?
Seungjoon Choi There’s no separate charge, and they give you 100GB of storage. So the binaries there aren’t retained for very long, but files you move into storage are retained. As I’ve kept using it, I found that I was already using about 2GB out of the 100GB. But in Chat mode, they give you four virtual CPUs, and in Work mode, they give you eight, but no GPU. Neither mode does.
Chester Roh So rendering with Blender would all be done with CPU computing.
Seungjoon Choi But you can enable multithreading in differentiable renderers like Mitsuba, so if you know a bit about that, it makes for some interesting experiments. Though it was a rabbit hole.
Chester Roh All right.
Seungjoon Choi But there’s something else too.
Chester Roh There’s one more thing, right?
AlphaGenome Atlas and genomic-variant analysis 52:53
Chester Roh This was news that suddenly broke yesterday. Google announced something called AlphaGenome Atlas, covering all nine billion variants, and now all you have to do is look them up. That’s what they said, but it doesn’t immediately sink in. So you need to know a little about what AlphaFold and AlphaGenome are, to understand it,
How AlphaFold changed protein-structure research 53:17
Chester Roh and we’re already somewhat familiar with AlphaFold. When you input an amino acid sequence, the ribosome builds the whole thing, and when it emerges into the cytoplasm, that strand folds and tangles around itself to form a particular 3D structure. That’s known as the protein folding problem, and as recently as five or six years ago, it was still an unsolved problem, but DeepMind came along and solved it completely.
There’s a site called PDB, and when you visit PDB, it shows the structures of the proteins listed there along with various other information. It’s a database where all of that is registered, and it contains roughly 200,000 entries. AlphaFold was designed based on that database to predict the structures of all proteins. Once AlphaFold had been developed to a certain point, they simply ran the simulator for the estimated 200 million or so proteins and generated all their 3D structures, creating the AlphaFold database.
So in the past, even when trying to treat cancer or something similar, even if we knew the amino acid sequence, we didn’t know what structure the protein would take. So we had to study each of them or determine the structure using X-rays, and once the structure had been identified, we would then search for an antibody that matched it. That was the traditional drug-development process. But since AlphaFold came out, once a problematic protein has been selected as a target, you can simply generate its 3D structure and then look for things that seem likely to bind to it. It becomes a problem of finding antibodies in silico.
Seungjoon Choi So it’s like this. What exactly is it that binds?
Chester Roh There is a shape that binds to that protein, right? Based on that shape, you design things that can bind to it like a key fitting into a lock. You can think of it as the relationship between an antibody and an antigen, and ligands are also one type of approach used to do that. We don’t need to go into the details of that, but now we understand what AlphaFold is.
AlphaGenome and non-protein-coding regions 55:32
Chester Roh Google DeepMind announced something called AlphaGenome around last year. I don’t think it’s even been a full year yet. There was quite a lot of discussion about what AlphaGenome is on YouTube and in papers, but understanding it requires far too much knowledge of the fundamentals of biotechnology.
As a result, people just thought, “So that’s a thing now,” and moved on. The simplest way to explain AlphaGenome is with something we learn in middle school and high school biology classes. We have 23 pairs of chromosomes, receiving 23 from our mothers and 23 from our fathers. Of each pair of genes, the dominant ones are expressed while the others are switched off, and each set also contains 3.1 billion base pairs. That’s what we learned. Those base pairs consist of long sequences of A, G, T, and C, and so on. If we take the entire genome as 100%, only about 2% consists of exon regions that encode proteins, while what the remaining 98% did was, until just 10 to 15 years ago, not exactly completely unknown, but still very much shrouded in mystery. It was simply one of the mysteries of life.
But then machine learning emerged, new observational instruments were developed, and the U.S. government invested an enormous amount of money in projects such as ENCODE. Through these efforts, researchers have conducted extremely large-scale experiments to determine what these regions actually do. In the early 2000s, the Human Genome Project also cost seemingly endless amounts of money, It was a project that left people wondering whether it would ever be completed, but it was completed. And now, to sequence a human genome, it costs about $500–$570 using short-read sequencing, and even with long-read sequencing, it only costs around $1,400–$2,100 to read all of my genetic data. That’s the era we live in now.
So, we’ve read the genes, but how can I explain what’s important here in the easiest way to understand? You’ve followed me so far, right?
Seungjoon Choi Roughly. I wasn’t very good at biology. When I was in school.
Chester Roh Yes, that’s right. But back then, we really thought biology was for people who liked memorizing things, while people who liked subjects such as mathematics went into mathematics or physics, chemistry was somewhere in between, and the field closest to the humanities was biotechnology. That was how we saw it. But the age of biotechnology is now arriving in full force. And we’ve learned quite a lot about exactly what roles these genes play. So only 2% encodes proteins, which means the other 98% is the program region, and everyone wondered what was happening within that program region. But we have managed to identify quite a lot of it. enhancer, promoter, and then regions within genes such as exon, intron, and how these things operate. As for how we figure that out, let’s say, for example, that we have a liver cell. That liver cell would contain genes. But it wouldn’t make sense for every one of those genes to be expressed and producing proteins. Brain cells, liver cells, muscle cells, and so on all need to produce different kinds of proteins.
So how are they made different?
Epigenetics and partial reprogramming 59:14
Chester Roh It depends on which regions of the genes are expressed and which regions are not, which is the domain of what is known as epigenetics. If you look at the structure within a gene, the base pairs aren’t simply all laid out with enzymes attaching to them and transcribing all the proteins. Depending on the location and type of cell, some regions are wound up, tightly wound around proteins called histones, while others are unwound. It varies across the cell. But this doesn’t happen randomly. As development proceeds from the initial embryo, the cells continuously differentiate in response to their environment and eventually settle into their respective states. To borrow an analogy from a scientist, if you roll a whole bunch of marbles down a valley, the marbles keep rolling until each one reaches its own place. And once each marble has settled into its place, it can never return. That’s the analogy. Solving this is what people are now working on in longevity and similar fields through partial reprogramming. Have you heard of it?
Once a cell becomes a somatic cell and its function is firmly determined, it has to perform only that function. But a Japanese professor named Shinya Yamanaka discovered something called the Yamanaka factor. When proteins are expressed from the four genes known as OSKM, some mechanism is triggered that causes the cell to lose its entire identity and return to something resembling the pluripotent stem cell it was at the beginning, a stem cell capable of becoming anything. A stem cell—it returns to being a stem cell. When that happens, during the process of returning to a stem cell, the cell removes the numerous age-related acetyl groups and other markers attached throughout its structure, completely resetting itself to the state it was in when the sperm and egg first joined.
So if we did this and reprogrammed everyone, wouldn’t we become younger? That is technically true, but if you make cells younger indiscriminately, a cell might become hair where muscle should be, or become a liver cell where the brain should be, differentiating however it pleases. That process leads to cancer. So from the OSKM factor, though I don’t remember this with complete certainty, they use only the OSK factor, meaning three of the four factors, and the cell returns to a stem cell only to an appropriate degree. These days, researchers are using that to conduct experiments involving partial reprogramming.
Professor David Sinclair, for example, regenerated retina cells in the eye so that mice that had originally been blind could see again, among other experiments. Then there’s NewLimit, a company made famous by an investment from the CEO of Coinbase. That company regenerates liver cells to make the liver of an old mouse like that of a young mouse, claiming that with younger liver cells, you could drink as much alcohol as you want. That was how they promoted it.
Seungjoon Choi Is there no trade-off? I mean, are there risks to resetting it…?
Chester Roh There are risks. Because there are risks, they use various switches to make the process operate in a highly controlled environment. There are mechanisms used for that, and I remember reading through all of them once, but I can’t recall them now. So first, the reprogramming-capable OSKM first put the factor into the eye, put it into the eye so that the drug is present, and then take another drug that acts as a specific trigger, as a switch, it activates it, and when that drug is gone, it deactivates, so they attach a switch like this and use it to control the process.
Seungjoon Choi Then this is somewhat different from epigenetics, right? I thought epigenetics was something like a switch, but it sounds like this is something that actually does reprogramming.
Chester Roh Right. Rather than creating that cell there, they place right next to it a stem cell that has fully returned to a pluripotent state, and it is influenced by the somatic cells around it and turns into that type of cell…
Seungjoon Choi This is completely outside my area of knowledge.
Chester Roh Yes, that sort of thing. I think there are many different directions this can take. So one direction is to introduce a fully reprogrammed stem cell and have it develop in that direction, and then another is to take the somatic cell and slightly reset its epigenetic clock, so that it retains its original identity while all the methyl groups and acetyl groups attached to it are reset, allowing the cell to function as though it were young again. I think there are approaches like that. But it gets far too complicated once we go into that, and I can’t guarantee that I understand all of it accurately.
Seungjoon Choi So what does this have to do with any of that?
SNPs and individual genomic differences 1:04:26
Chester Roh Well, we got this far while talking about AlphaGenome, but people’s genes contain A, G, T, C arranged in a sequence like this, and if there’s a slight error in a protein-coding region, we can determine with certainty what problem it causes, but if there’s an error in a programming region, we end up in a situation where we can’t tell what the error is. But to explain this, I also need to explain the concept of an SNP a little. I said earlier that our genome has three billion pairs, right? And those differ from person to person.
Seungjoon Choi What is being raised to a power of around three billion? Codons?
Chester Roh Raised to a power?
Chester Roh Like something squared?
No, that’s not it. There’s simply a sequence of three billion A, G, T, C letters, right? So if there’s one of A, G, T, C, another base pairs with it, such as adenine or guanine—
Seungjoon Choi Cytosine.
Chester Roh Things like that. Cytosine, thymine, and so on. So once one base is determined, the base paired with it on the opposite side is automatically determined. But if there’s an A, it and the base paired with it form a single unit of information. It’s almost like a bit. We simply consider that one pair. Those pairs continue in a sequence of A, G, T, C, and the human genome has three billion pairs. To be precise, it’s 3.1 billion, so it has 3.1 billion pairs, and theoretically, since we have one set inherited from our mother and another from our father, if we receive one set of three billion from our mother, we also receive three billion from our father. So theoretically, we have around six billion in total, but since those A, G, T, C letters also form pairs like this, strictly speaking, if you ask how many A, G, T, C letters there are in the human genome in total, the answer is 12 billion. That makes sense, right?
Seungjoon Choi Twelve billion.
Jonghyun Park If we think of this as data, each one has four possible values, and there are that many of them, so it would be 4 to the power of six billion, if you look at it that way. Of course, there is an enormous amount of overlap between the information inherited from our mother and our father, so we wouldn’t have all of that information independently.
Chester Roh So in reality, there are six billion, but we only look at three billion. That’s because almost all of them are similar, and in some cases, the dominant genes are expressed while the others are switched off. But most of what we call a gene was also identified experimentally: if certain regions are found to perform a particular role, we call those regions genes. Within those genes, protein-coding regions and other regulatory regions are all mixed together. But the copies inherited from our mother and our father perform nearly identical functions, corresponding almost exactly at the same indices in our sequences. So for convenience, we discuss only three billion base pairs and discard the other half. That’s how we handle it. But once we start discussing this, we can keep going deeper without end. It’s called an SNP, or single nucleotide polymorphism, and there are three billion of these letters, right? But of those three billion, Seungjoon and I have almost exactly the same ones. In fact, humans—
Seungjoon Choi Because we’re human.
Chester Roh Actually, that’s right. Even humans and dogs are nearly 80% the same, And humans are almost identical to one another, and out of those three billion, only a very small number, around four to five million, are different. But when one program differs along the way, it’s not that the G, T, and C blocks are completely different. As you go along, where Seungjoon has a G, I have an A, and they really differ by only one letter at a time. Very sparsely. So those single-letter differences are what make us different.
When we have our genomes analyzed, we can learn everything from the genomic data. The way it works is,
Short-read and long-read sequencing and genome mapping 1:08:34
Chester Roh instead of reading all of a person’s genes, when we send in blood or something like that, they grind it all up in a blender and cut up all the DNA. In other words, they cut sections from a single strand. Then they read all the cut-up DNA, statistically.
Seungjoon Choi By putting it into something like a nanopore sequencer?
Chester Roh Right. You put it through a pore, and it reads straight through it. If you cut the strand into short pieces, that’s short-read, and if you cut it into long pieces, that’s long-read. So when you cut it into short reads, all the data comes streaming in from those cut-up pieces. Then you feed all of that into a computer, and we have what’s called a reference genome. It represents the most standard human and shows what the human genome looks like. It came out of the Human Genome Project, and after several revisions, we now have the reference that is currently used. Against that reference data, I try aligning all my shortened strands. Since these sequences are nearly identical, as you map each one, it attaches to a particular location. As they statistically align, you find the parts where you say, “This one has a different letter here,” and swap those parts out. Using the reference data and my sequence data, you take what you’ve obtained and keep fitting it all together like this.
The HLA region and the need for long-read sequencing 1:10:02
Chester Roh Yes, that’s right. The reason long-read is important is that most of it can be read with short-read as well, but on chromosome 6, there is a unique feature called HLA that serves as an immune marker in humans. Inside the cell, there are clumps of proteins being produced, and displaying “I’m making this kind of protein” tells the outside world what kind of work that cell is doing. So the HLA mechanism takes fragments of the protein clumps produced inside the cell and brings them to the cell surface. The one that reads the protein brought to the surface is the T cell. The T cell comes along—it’s an immune cell. The immune cell comes over and examines it, and if it says, “This one is functioning properly, and this is me. It’s one of my own cells,” it leaves that cell alive. But if the cell is producing a foreign protein, it says, “This one came from outside,” and devours it. This is the very core of our immune system. But the region responsible for HLA varies greatly from person to person. And rather than varying from person to person, it varies greatly between ethnic groups. Koreans and white people differ greatly, and Black people differ as well, so that region is different. This is the region with the greatest variation. So if you read it using short-read sequence, it can’t be mapped accurately. Because there are so many highly similar strands, with short-read, the resolution of chromosome 6 is very low. If you read it using long-read, you read a single gene over a very long stretch, so even when you place it at its location on chromosome 6, you can say, “This one differs here in this way,” and map it accordingly. That’s the advantage of the long-read method.
But I mentioned SNPs, and as you move along, if only one letter differs, then naturally, the question of exactly what problem that single-letter difference causes becomes important. Right. Some variants are deleterious and harmful, while other variants cause no major harm even if they differ. But differences of just one or two letters can change people’s lives. Genetic diseases—statistically speaking, that’s right. Genetic diseases are examples of that.
Predicting the effects of genetic variants with AlphaGenome 1:12:25
Chester Roh AlphaGenome finally comes into the discussion. What AlphaGenome identifies is, when that one letter changes, what problem it will cause in which cell, by predicting the experimental results. AlphaFold’s input is a string of amino acid sequences, and its output is the 3D structure of that sequence. But AlphaGenome is completely different. One million base pairs of DNA, meaning one million indices. When you input a sequence of one million DNA letters, it predicts the results of 11 types of experiments. So what are those experimental results? Over the past 20 years, the ENCODE Project has continually discovered what different genes do through experiments. For example, in some parts of a liver cell, DNA is wrapped around histones, while in other parts it is unwound. Then you sequence only the open regions of the DNA, and if you also sequence the RNA sequences expressed in those corresponding regions, you can predict which things in the open regions were expressed as RNA. That allows you to make such predictions. So they’ve continually accumulated those findings into datasets. Then they classify them by cell type—this is a brain cell, this is a liver cell, this is a muscle cell— and conduct all those experiments for each cell type.
Seungjoon Choi How, do you do that with humans? Do you use organoids?
Chester Roh No. We do it with humans. It’s simple. What you do—and this is interesting—is just extract the liver cells. Then, for example, you look at the genes being expressed. We call them euchromatin and heterochromatin, and if you only want to read the open chromatin regions, you leave the parts wrapped around histone untouched and apply a treatment that reads the DNA only in the uncoiled regions. Then, assuming that only the uncoiled regions were expressed, you read all the regions next to them that were transcribed into RNA. You perform RNA sequencing on that as well. Then you have the DNA sequence and the RNA sequence, so, just as we aligned things to the reference genome in the Human Genome earlier, this is a much easier problem. You just need to match them up. “This RNA came from here, and this RNA came from there.” That’s how you identify them.
Seungjoon Choi It does feel like the difficulty is rising to the level it did when we discussed infrastructure.
Chester Roh Right. But to talk a little about business here, even I find this difficult to explain. In a world where this model and AI and everything else are becoming possible, where on earth are we supposed to retreat to? That was our first theme, wasn’t it? So I wondered how I could make good use of AI in business and what I could do from a business perspective. What I found was that the opportunity lay in building a good control plane, so I built that system and worked hard on it.
Beyond that, I wondered what domain I should focus on, and my idea is to learn as much as possible about the part of biology being revolutionized by AI, and, conversely, position myself between biologists who don’t understand AI and AI scientists who don’t understand biology. That’s my thinking, and there are people with the same idea in Silicon Valley, which is why there are so many AI for Science companies, especially ones working in biology. So I’m not trying to do anything with it right now, but as I try to understand this field and examine all these systems, I end up learning these things. So, as Seungjoon asked, to determine which DNA is expressed and which parts actually perform a function, they painstakingly tested all of it experimentally.
Seungjoon Choi So an Atlas has been created. One that can do an enormous number of things.
Chester Roh Right. So, there are 20 years’ worth of such experimental results, and AlphaGenome predicts those experimental results. If you give AlphaGenome one million base pairs, it processes them and predicts that this is what happens in brain cells, this is what happens in liver cells, and this is what happens in other types of cells. It produces predictions for all of those. It outputs them all as arrays, but how do we determine whether that’s good or bad? We have a normal reference. There’s the sequence of a normal human, and when we input one million base pairs, those one million base pairs will include the region containing my variant. Then, by calculating the diff between the experimental results produced as output, we can predict that this particular genetic variant of mine will likely cause a problem in a certain part of the liver. It can make predictions like that. Now, if we know how to program it, we can compare them at the vector level, much like comparing log probability, but Google has helpfully created a complete index for it. That’s what AlphaGenome does,
The precomputed database of AlphaGenome Atlas 1:17:47
Chester Roh so what does AlphaGenome Atlas do? Think about it. Earlier, we said there are three billion base pairs. Three billion. Theoretically, each of those three billion bases could be changed, right? So, for each of those three billion positions in the reference genome, if a given position currently contains A, they calculate what would happen if, instead of A, the reference contained G, C, or T. They performed all those calculations and stored all the results in a database. So, from a researcher’s perspective, without this Atlas, they would have to run it themselves.
Similarly, without the AlphaFold Protein Structure Database, if I wanted to predict a protein structure from a newly discovered amino acid sequence, I’d have to run AlphaFold. But if Google has already run them all and has all the data, when you enter an amino acid sequence, the 3D data comes out immediately. This is exactly equivalent to that. So, just as you would with AlphaGenome, if you input a genetic sequence of one million base pairs, AlphaGenome returns all the experimental results. But if you enter it into the Atlas, it returns things like, “This result changes in this way. That result changes in that way.”
So, using the results stored in its database, all we have to do is calculate the diff to predict which cells are likely to have problems and where, and which ones are unlikely to have problems.
Jonghyun Park But then, ultimately, I’m curious how accurate those predicted experimental results are, and after actually testing them, We have no choice but to compare them. For areas that haven’t been tested.
How AI tools are changing life-science research 1:19:37
Chester Roh So science changes when tools like these emerge. For example, AlphaFold predicts all protein structures, so you might think databases like PDB would become obsolete, and that there would no longer be any need to conduct X-ray diffraction or things like that, but in reality, the process has been shortened tremendously. Before researchers study a particular target protein, they run AlphaFold, as if taking a satellite image to decide whether to proceed with an operation, and identify candidates that look viable or unviable. Then, using what they have identified, they can conduct experiments much more strategically. So research has become much easier.
The same goes for AlphaGenome. AlphaFold also used the 200,000 already known structures in PDB, which were the results of human experiments, to predict the remaining 200 million. And those predictions turned out to be highly accurate. Likewise, for the past 20 years, AlphaGenome has had all the experimental results showing what changes occur in exons or regulatory regions when genetic variants arise. All of those experimental results are available. It’s a much smaller dataset than an LLM’s. How many experiments could humans possibly have conducted?
Seungjoon Choi Something suddenly occurred to me: like formulating an Ansatz or hypothesis, as in the earlier math example, is this a tool that an LLM can use?
Chester Roh A tool that an LLM can use. The LLM would call Astra or bring in people on its own.
Seungjoon Choi I suddenly wondered whether things like Atlas and AlphaGenome effectively become tools that an LLM can use.
Genomic analysis and personalized treatment 1:21:26
Chester Roh I call them through Codex.
Seungjoon Choi I see.
Chester Roh Today, I also used AlphaGenome Atlas, because I’ve had all my long-read sequencing done. With the idea of preparing in advance for what would happen if I developed cancer, I obtained my entire genetic sequence, and I know almost all the medications that could cause particular liver diseases in me. And compared with the reference genome, I already know where all my SNPs are. Then I can split those sequences up and examine not only what is already known, but also what problems each SNP might cause. If the whole represents 100, only a very small proportion is currently known. I’ve forgotten the exact number, but there are still vast unknown areas.
But for those unknown areas, just as AlphaFold used 200,000 structures to predict 200 million, very little has currently been discovered in these unknown areas, but AlphaGenome can be used to predict roughly what the unknown ones might do, and experiments can then be redesigned based on those predictions. “Ah, it was right.” That makes the verification process much faster.
Seungjoon Choi Does this lead to general-purpose drug development? Or does it lead to personalized treatment? Is it both?
Chester Roh Both general-purpose and personalized drug development.
Jonghyun Park Right. Ultimately, AlphaGenome has been released, and I’m curious about what kind of social impact it will have. It will probably play a role in every path toward extending human lifespan, but I can’t immediately get a sense of just how much it has accelerated that progress, so I’m curious about those things.
Finding targets for rare genetic diseases and cancer treatment 1:23:07
Chester Roh With genetic variants and rare genetic diseases, we currently have no way of knowing why these phenomena occur, but if you run AlphaGenome, it can predict that the 3D structure between a promoter and something else is wrong over there, or that a SNP index is off at that location, making predictions like these. In the past, we would know that one gene differs from another, but now we can predict exactly what part of the process through which it is expressed as a protein, which we call the phenotype, is causing the problem. That’s one application, and the same applies to cancer.
For example, cancer develops, and a cancer gene acquires a mutation. So it starts behaving abnormally. Then we need to treat it. But when we try to treat it, if the mutated section is clearly in a protein-coding region or something like that, we can determine that there is something different about the protein it produces, run AlphaFold on it, find substances that can bind to the target, and develop a drug. But if the problem occurs in a regulatory region rather than a protein-coding region, that regulatory region exerts overlapping effects, altering the process by which a particular protein is produced.
In cases like that, we can predict much more precisely where the problem originates. Those are the kinds of things we can predict with AlphaGenome. So this may be normal and cause no problems while a person is healthy, but when cancer develops or something like that happens, it can compare the cancer cells with my normal cells and identify what we should focus on and where the problem is likely to be, presenting us with multiple possibilities. It’s a tool that can be enormously helpful.
Seungjoon Choi My gut feeling is that even if we know that, treating it could be quite costly, or so it seems to me.
Chester Roh But last time, the Gil Lab CEO used a process to treat his own cancer before AlphaGenome or anything like it existed. Fortunately, when he compared his cancer genes with his normal genes by running a diff, he found a protein expressed only in the cancer. Then he traveled around the world looking for a substance that would bind to that protein, and eventually found one somewhere in Germany and used it to create his own personalized drug.
Biotechnology’s shift to computational science 1:25:45
Chester Roh But we should expect that cost to keep falling. Merck and Moderna recently said that they had developed an mRNA vaccine, and that works on exactly the same principle. So biotechnology is increasingly moving away from what used to be the realm of tedious experiments conducted in a wet lab while looking through a microscope, just as the Navier–Stokes equation we discussed today was once something worked out only on mathematicians’ paper, but has now all been transformed into the token of some LLM. The same thing has happened in biotechnology. They use previous experimental results for supervised learning, and that supervised learning produces a model, which then accelerates the next experiments tremendously. It makes them easier and allows experiments that were previously impossible to be designed. Those experiments then produce another dataset, so the amount of data will keep growing. And right now, it’s supervised learning, but once enough of that data starts accumulating, at some point, using that dataset to simply run self-supervised learning will bring it to a level where it can discover some general principles.
Evo 2 and language models for synthetic biology 1:27:05
Chester Roh Something similar to AlphaGenome, but slightly different, is a model created by Arc Institute called Evo 2. The Evo 2 model—AlphaGenome is extremely complex. It’s fully supervised learning. It takes a gene sequence as the input for experimental results, maps the experimental results to create a supervised set, and puts a convolutional neural network and a Transformer in between to train it. But a model like Evo simply takes the nucleotide sequences of every known species, creates an LLM-scale dataset, feeds it into a Transformer, and trains it on the sequence. So this one is simple.
Seungjoon Choi It just has a lot of data.
Chester Roh If you simply give this one an AGTC sequence, it just spits out the following GTC sequence the way an LLM does. We can’t tell just by looking at what it spits out, but it’s producing a sequence based on what it learned from genes. So they used the sequence it produced to make something, and a bacterium emerged, which they describe as a new discovery. The field that does this sort of work is called synthetic biology, or 합성생물학.
But with an LLM, we can predict that sort of thing too. We keep generating sentences autoregressively, and when we spoke with Seonghyun before, we said that certain tokens have very high entropy, remember? So here’s something this model can do as well: take genome data obtained from the reference, cut out a segment and feed it in, then take my data containing the variant, cut out a segment and feed that in, and then use the entire preceding part as though prefilling it into the model. Then an output value will come out at the end, right? If you simply subtract the log values from each other, a negative result means the variant is likely to be dangerous, while a positive result means it is likely not dangerous. That’s another way it can be used. This lets us roughly estimate the risk posed by variants that aren’t yet known, although even if we calculate the risk, we can’t tell what kind of risk it poses. But whether it leads to cancer or something else, genomics itself is also building up larger databases and shifting toward computation, so progress in this area continues.
Seungjoon Choi I didn’t understand much of it, but it feels like it has become something that can be computed.
The explosive scaling of genomic data and bio-AI 1:29:43
Chester Roh Right. And if a verifier keeps running in the wet lab and continuously enriches the dataset, there’s a very high chance that this field will eventually enter the same era of explosive scale that LLMs experienced.
Jonghyun Park As I listened to the whole discussion, I didn’t understand everything perfectly, but when we first started, we talked about this, remember? That the life sciences have a somewhat humanities-oriented tendency. That’s how you described it from the science side, and when I think about why that is, there are so many underlying principles that we don’t understand, and we look at experimental and external results and say, “This is what happens.” That’s because there was so much of that kind of information to memorize, but now the field itself starts at the very beginning, starting at the DNA level and ultimately leading to a particular phenomenon— how proteins are made and ultimately affect our bodies— I think the research is about connecting the dots between them. So AI will also transform this field in ways that make it explainable, and as we come to understand the mechanisms at work inside, from a research perspective, I think we’ll be able to discover so much more, and when it comes to humans, longevity is naturally the greatest need, so I think that’s where it will go first, but beyond that, I think it could also branch out in many other directions.
Chester Roh It’s so fascinating that lately, I live each day for the pleasure of reading about this field.
Seungjoon Choi But if you look at the reports coming out these days, they always include cybersecurity and biosecurity. Whenever a large model is released. Of course, there are positive applications, but there will probably be frightening ones too.
Jonghyun Park How these things might be used. Right.
Seungjoon Choi Chemistry and biology are frightening in their own ways, too.
Chester Roh Right. But when it comes to domains, if we’ve decided to spend our lives pursuing higher learning, we each have to choose at least one domain, right? So I picked one and am trying to delve deeply into it—that’s how I’d like you to see it.
Seungjoon Choi I think there’s something of the energy Chester is pouring into this.
Chester Roh Humans are just extremely complex machines, but they’re programs.
Jonghyun Park Right.
Seungjoon Choi Back to radical reductionism.
Chester Roh So once the secrets at the gene level are uncovered, we’ll understand how genes become proteins, and then once the secrets of proteins are uncovered, how a mother’s egg and a father’s sperm meet and develop into a human being is still a mystery to us, but starting with C. elegans, for less complex organisms, even their developmental processes— which genes are expressed where, what becomes what, dividing the body into the head, thorax, and abdomen, what becomes the spine, what becomes the heart— have been almost completely figured out at that level.
Seungjoon Choi Right. The question is how much that will accelerate. It will probably be relatively slow, but the acceleration is definitely palpable.
The future of life sciences accelerated by AI 1:32:39
Seungjoon Choi We talked about the Millennium Problems today, and o1 came out in
Chester Roh September 2024. So it’s been exactly two years. It’s been two years, and even the things we’re talking about are quite incredible. In just two years, we’re now
Seungjoon Choi talking about things from science fiction. In 2026.
Chester Roh It’s killing me trying to keep up. Seriously.
Seungjoon Choi Biology isn’t there yet either. If mathematicians have started experiencing their AlphaGo moment this year, biology isn’t there yet and may simply be excited about having gained a new tool, but who knows?
Chester Roh But as for this, I mentioned it after my last trip to the United States: I looked at what brilliant AI PhDs study at night, and discovering that they were all studying things like this was significant for me. It made me think there’s something there, and I definitely still believe that. So AlphaFold and AlphaGenome are both truly tremendous achievements, and what’s more interesting than LLMs is that both AlphaFold and AlphaGenome, compared with the amount of data and scale of compute used to train today’s cutting-edge LLMs, don’t seem that large to me. I haven’t compared the total amounts, but they’re very small in terms of dataset size and compute. So I think we’re amazed because we don’t yet fully understand just how astronomical the datasets and compute going into cutting-edge LLMs are right now.
A three-day AI news recap 1:34:10
Seungjoon Choi Anyway, since our last recording, only three days have passed. There was a lot to talk about.
Chester Roh So this ended up being a brief recap of the past three days. Indeed. We’ve been talking for almost two hours.
Seungjoon Choi Well then, have a good business trip.
Chester Roh We’ll wrap things up here for today.
Seungjoon Choi Thank you.
Seungjoon Choi Great work, everyone.
Chester Roh Take care.