EP 79
AI Frontier: The Runaways' Alliance Retrospective & GPT 5.2
Opening: December 14, 2025, GPT-5.2 Update 00:00
Chester Roh Today, as we are recording, is December 14th, 2025, a Sunday morning. Time is passing by breathlessly again.
I thought I’d get some rest at the end of the year, but GPT-5.2, adding 0.1 to 5.1, has had a version update. It’s become 5.2, and the changes are quite significant. It was probably released to keep Gemini 3 in check.
Seungjoon Choi That’s right. Right now, OpenAI is under a lot of pressure, according to news on the timeline, so they had to show something. But Chester, you mentioned there was one eval you were paying attention to?
GDPVal Surge and the Commodification of Intellectual Labor 00:35
Chester Roh Yes, the one I focused on is called GDPVal, which is at the very top of this sheet we’re looking at. This is actually a benchmark that came out with the announcement of GPT-5, one that OpenAI created. For about 44 actual human job groups, they created about 1,300 to 1,500 tasks as an evaluation set to see how well it performs jobs that humans do. It’s a benchmark created to see that.
But with GPT-5.1, it scored around 39, and with 5.2, it became 70. Well, there could be various reasons. We don’t know. It’s possible that a lot of synthetic data for the human work domains covered by GDPVal was created and included in the pre-training set, or that the post-training was enhanced to match it. We can’t know the recipe, but the important thing is that the kinds of unique human intellectual labors that we used to say, “Models can’t do this,” are all rapidly becoming commoditized.
What’s more surprising is the cost involved. It cost only 1% of the estimated expense that would have been paid to a person to handle that work. It was literally just the cost of electricity. And the processing speed was a whopping 11 times faster. Even at the beginning of this year, we were having conversations like, “Will this AI continue to develop and replace all our jobs?” and this is a benchmark where we’d say, “If you solve this, it’s AGI.” That’s the kind of talk we had about it.
Seungjoon Choi A year ago, the flagship models couldn’t do this well, so we said, “If this gets solved, then that must be solved,” and now it is. Tier 2, which was a more difficult problem, is now being solved.
Chester Roh The benchmarks are becoming obsolete. Of course, when benchmarks come out, the benchmark itself becomes the goal.
Isn’t there a lesson we learned from RL? If a goal is set and you can give some kind of reward for that goal, if you can just make it verifiable, then the model can finish it. That’s the new rule we’ve generalized now. It just gets done.
Seungjoon Choi So the pattern we’re seeing now is, if you create a benchmark, it gets solved. So, you just need to create a benchmark. It’s a bit backwards, but that’s the gut feeling I get.
METR/Epoch AI Time Horizon and Gemini 3 Pro’s 4 Hours 03:08
Chester Roh So, at Epoch AI, I can see it right below this graph.
Seungjoon Choi This is an estimate. There’s no official announcement from METR, but they’ve made an estimation.
But now in November, it’s already been about a month. Gemini 3 Pro, although it’s represented linearly here, is showing exponential progress. On the time horizon for 50% human performance, it has achieved 4 hours.
Chester Roh So, 4 hours means that it can perform the work a person would need 4 hours to do. That’s how we should see it.
Seungjoon Choi Right. Right. So that’s the estimation that came out, and if this trend continues, by this time next year, it will be able to handle a human’s full-day, an 8-hour workday. That’s the projection.
Chester Roh Right. Seungjoon, seeing as you said “this time next year,” it seems like it will only take about 3 months. We used to joke that “a month feels like a year,” but now it seems we have to shorten that unit to two weeks. Changes where two weeks feel like a year are happening.
Seungjoon Choi That’s very troubling.
Chester Roh It is. It’s hard to keep up at a human pace.
Seungjoon Choi But now, with these events, and I hear there’s another announcement next week.
Chester Roh It seems that before the year ends, Google wants to one-up GPT-5.2 again. There are some reports that Google will do something more. There are observations, and not to be outdone, Sam Altman said, “We also have something to release next week.”
Seungjoon Choi So, I didn’t bring the chart, but if you look at the models released this year, at the points marked with the time and date, many more are clustered in the latter half of the year. They’re concentrated there. It’s quite something.
So, on Thursday, the reason the news this week is a bit less compressed is because we didn’t have time to look into it in detail.
OpenAI’s 10th Anniversary, Performance/Price Competition, and Convergence to “Cost” 05:07
Seungjoon Choi But on Thursday, around the time we were wrapping up that event, OpenAI released this video. They said it’s their 10th anniversary.
Chester Roh Oh, it’s the 10th anniversary.
Seungjoon Choi Right. Just 10 years ago, it was like this.
Chester Roh 10 years ago, we were talking about things like ResNet with convolutional neural networks.
Seungjoon Choi This is what’s important. “We’ve only just begun.” “The model you’re using today is the worst it will ever be.”
Chester Roh That’s right. At some point, I stopped tracking this price-performance ratio, the token price, although of course, Chinese models are much cheaper than America’s frontier models, but I stopped tracking it.
Seungjoon Choi Oh, why is that?
Chester Roh Ah, this is like a Nash equilibrium, a kind of game of chicken. It’s racing towards an equilibrium point, and that point will probably go down to the cost price. This cost price ultimately means the electricity bill and the price reflecting the depreciation of the equipment. A cheap price close to zero.
Right now, we don’t say we can’t do something because the electricity bill is too high. Then, when an almost infinite intelligence becomes available at our fingertips, the things we considered to be human value are experiencing this pace of change in a very short time. I think that’s the difference. That’s why it feels harder.
Loneliness and Anxiety: Why People Started Gathering 06:46
Seungjoon Choi So, regarding these things, we keep having somewhat depressing conversations, trying to look at the world, and we met to study, right? Has it already been two and a half years since around May 2023? Right. It’s been about that long,
Chester Roh Has it been two and a half years since we started this? It’s been a long time.
Seungjoon Choi But regarding those things, we said we felt a bit lonely, but it wasn’t just us who were lonely.
Chester Roh We didn’t start it as a joke, but with a purpose, we hoped to have a few people who could look ahead with us, and that’s why we started.
But we received so many applications. So many people applied, and the profiles of the applicants were so good. So, from the content these people gave us, we learned quite a lot, didn’t we?
But if there was a common keyword we felt, it was that everyone was anxious, lonely, in pain, and very bewildered, not knowing where to go. They were all sharing the exact same emotions that we were feeling.
So we thought we should gather such people together, gather to comfort each other, exchange what we have, and we felt we needed to form a solidarity. So we created an event.
But that event, thanks to Seungjoon, and also thanks to Yujin, became very enriched. So, it went beyond just gathering for coffee, having a beer, and hanging out. It evolved into a generative conference. Please share that story with us.
First Sync-up of the “Runaways’ Alliance”: Building a Generative Conference 08:29
Seungjoon Choi So, the group called “Runaways’ Alliance” had its first sync-up last Thursday, a synchronization meeting.
Just as you mentioned, we somehow created a space, and generation just happened.
So, the expression “runaway” came from our episodes where Chester kept saying we have to run away somewhere, and at some point, the phrase “we have to run away” stuck and became this.
Chester Roh We were saying that the frontier models are eliminating everything we have, so where should we find a way to survive? That was our starting point.
So, in between, we used the expression “Let’s run away,” and that turned into “people who do that, runaways,” so we ended up defining ourselves as “runaways.”
Seungjoon Choi Right. Meaning, outside the frontier, let’s talk about what’s happening.
So, from the perspective of an overly-immersed AI enthusiast, we were looking into these things, and people who wanted to show a similar landscape Many people applied, and a total of about 140 people gathered in a space like this, at B Factory’s great venue, we were able to hold a meeting.
If we take a quick look at the scene, there was lively catering, and afterwards, the sessions went on.
At the beginning, Chester, this slide was quite interesting. To our colleagues who are going through this difficult time together, “Oh, you’re working so hard. Shall we give a word of praise before we move on?” Is that how you started?
Chester Roh Yes, yes. I felt that we all needed to comfort each other, so I did that. Yes, I also received a lot of comfort.
Seungjoon Choi So, this kind of thing, we actually took a moment to comfort each other, and, well, this is a time when we have to live as if it’s a year. What was it that we had to live like a year?
Chester Roh I said it’s a time when we have to live a month as if it’s a year, but now, as I mentioned earlier, I’m wondering if we should change a month to two weeks. That’s what I’m thinking.
Reflections on Running Panels, Small Groups, and AMAs—and the On-Site Energy 10:27
Seungjoon Choi Right. So we also had a panel discussion. Then, in the small group sessions, it wasn’t like we had a pre-planned impromptu conference on the day, an unconference, but rather, we planned in advance, shared materials, and met to have an in-depth discussion for about 45 minutes.
Chester Roh Yes, three rounds of 45 minutes each. Since we officially started at 7 o’clock, we had one round at 7, one at 8, and one at 9. I thought most people would leave around 9 o’clock, but that wasn’t the case at all. Almost everyone stayed until the very end, and I was surprised by their energy level.
Seungjoon Choi I was very surprised too. The after-party part, I mean, even after it was over, during the one-hour gathering to linger a bit, many people stayed.
Then, in the 35-person room, for those who are intensely contemplating, we invited Oh Sun-seok nim, Kim Min-seok nim, and Jeongkyu to have a session where we listened to their presentations and had an AMA. I heard the atmosphere was very heated then as well. Lee Kyeong-ho nim moderated it, right? The one who is active on LinkedIn.
Chester Roh He did a great job summarizing, so the content there seems to be well-captured. We also asked some hard-to-get people to come and speak, and they gladly came and spoke, so I’d like to take this opportunity to thank them once again.
Seungjoon Choi That’s right. And this picture is a screen capture from a video, taken by Jung Yoon-hye nim of Unbolt. Yujin, who always edits this for us, if she had come, quite a lot of people would have wanted to talk to her. And our co-host, Seonghyun, people were looking for him.
Chester Roh “Why isn’t Seonghyun here?” Yes.
Seungjoon Choi So, these problems were shared, but this isn’t the original data; I’ve anonymized it a bit and regenerated the content. So the nuance is there, but it’s not exactly the same. At first, 21 rooms were blank, right?
Chester Roh Right. I wondered if we could really fill these 21 slots, but it was a needless worry. They filled up quickly. Yes.
Seungjoon Choi Right. Didn’t I say it would work? So, because there was space, it filled up well, and because many people actively participated, that was possible. The sessions became self-organized.
Chester Roh Yes, as I walked around, I peeked into these discussions a little bit, and the content was all so interesting. It was so interesting that I wanted to go into each one and listen, but I had some regrets. I’m sure those who participated had a lot of fun. Yes.
Seungjoon Choi Yes, so during the networking time, I received some feedback that 45 minutes wasn’t enough, and that they were even planning follow-up meetings. There was that feedback, and it was so popular that it was almost overbooked.
How Discord Archiving and Self-Organization Happened 13:21
Chester Roh Yes, so these problems were a kind of snapshot of the real-world problems I wanted to know about.
There are commercial and non-commercial problems, and since I’m mainly interested in commercial problems, I focused more on business sessions or business presentations of that kind, and it was interesting.
Seungjoon Choi So, this data is currently being archived on Discord.
So, the problems that the problem-havers have, and the problem-solvers who show interest and give ideas, and the moderators who manage time, keep good records, and facilitate meaningful discussions, these moderators self-organized this event, and a conference was created in a snap. That’s why we named it the “Generative Conference.”
So, even if it’s a somewhat similar domain, because different people are meeting, when you have a vector and project it onto another vector, it’s natural for it to become shorter, so we should consider that.
I talked about attitudes for participation, and since it’s a diverse and complex phenomenon, if you look at it from various points of view, I thought we could gain diverse insights, so I also conveyed that mental image.
So, currently, a retrospective is happening on Discord. The photos and text records that were left are being shared, and we’re discussing what we learned and what we forgot. We’re in the process of doing that.
And how to improve this process itself, those things are also being discussed.
Also, to give a brief introduction, if you just create a multi-layered, diverse space and give a little nudge, just as Dario Amodei and Jared Kaplan said, if there’s space and conditions like temperature, pressure, humidity, etc., and you wait a bit for computation to happen and for self-organization to occur, those things can lead to interesting phenomena. We proceeded with that mental image.
It was a rewarding experience. Yes, this also takes time. And it will take time in the future as well.
Chester Roh So, starting with this community, many people are feeling a bit sad, but we’re not trying to make this community a very closed one. It’s because we feel we need to organize the snowflake to some extent and then open it up to the outside.
That’s why we started small at first. If everyone participates without that, it will go out into the world without a basic fractal structure. That’s why it goes outside. So you can think of it as us setting some initial rules. That would be good. Yes.
Seungjoon Choi That’s right. So, the current stage is, in a way, a high-entropy phase, so there are various potential directions. As for how to move in a better direction, since we’ve gone through some trial and error, we’ll listen to feedback and it will probably take some time to explore the next steps.
Retrospective Points: Incentive Asymmetry, Privacy, Openness 16:18
Seungjoon Choi I’m going to do the last part of the retrospective. The problems we started with at the beginning, I think they’ve been resolved to some extent, but looking back, there were a few problems that are still a bit concerning.
The first is the asymmetry here. The problem-havers and the problem-solvers, there was an asymmetrical part. And this is that part. Among the things posted on Discord, there were frank questions like, “What’s the incentive for problem-solvers?” There were frank questions like that. So we need to think about that, and then, problem-havers are rare.
And if you listen to presentations like Oh Sun-seok nim’s, the problem-haver is the one who solves the problem. But the current direction has that aspect, so I can’t just organize my thoughts on this today, but I think it’s a point we need to keep contemplating. I mean, being able to bring a problem is, in a way, a rarer case, and using AI as leverage to solve that problem isn’t that a more abundant area?
Chester Roh Exactly. This is something that we… because we’re also somewhat tied to the past, we always want to separate having and solving into a dichotomy. We also said this, didn’t we? “The question is everything.”
Right questions are all we need. We even made a joke like that, saying if you ask the right questions, the solution will be created by AI right away. Didn’t we say that? But what that means is, ultimately, ‘having a problem = solving it.’
Seungjoon Choi So, that, as you’ll probably mention later, corresponds exactly with the last part of your presentation, Chester. It fits perfectly. The part where you talked about entrepreneurs.
Chester Roh A world where we are all forced to become entrepreneurs seems like it will begin soon. Yes, and it’s not a world with a big supplier, employees hired by them, and consumers.
Instead, it’s a bustling forum-like state where you can’t tell who is the supplier and who is the consumer. I think that might be the new face of the market. If you look at us, we are already both suppliers and consumers. We are suppliers in one industry but consumers in another. We all have this duality, and the boundaries themselves are all becoming a bit blurred.
Perhaps it’s what Alvin Toffler talked about in ‘Revolutionary Wealth,’ Hasn’t life become one where everyone has to be a prosumer?
Seungjoon Choi From that perspective, things like having a certain space, or in other words, rent, the importance of rent, are things to consider. And while experiencing various things at this event, certain analogies that come to mind seem to be forming.
Chester Roh I think so. The meanings will gradually reveal themselves. For me too, concepts are kind of floating around in my head, and then one day they suddenly pop into my mind in the morning.
Seungjoon Choi And we had the position of moderator, and focusing on how they wove things together within the session, I think that happened much better than I expected. That’s what I think.
But in the networking that happened outside the sessions, when we had all that diverse information, I was curious about how much of the work of connecting it all actually happened. I was a bit curious. And in that case, if by any chance, there were ways to trade those problems well… There were already many things that existed, right?
There were platforms like Soomgo, and in a way, Daangn Market is like that too. In that case, compared to platforms, for those who were trying to make such connections, how could the talents they possessed be well utilized? I think these were also points of concern. And this is also becoming a big concern.
On our Discord, as people revealed who was doing what and where, too much personal information was being shared. So that’s also a concerning part. But if you don’t disclose it, you can’t meet because you don’t know, but if you do disclose it, there’s the concern that too much personal information is revealed. That’s the situation with those concerns.
Then, one more thing is, when I get feedback about these kinds of events from acquaintances or other people, they say, “Runaways? What do you mean runaways?” “Isn’t this a gathering of people sprinting at full speed?” I did receive feedback like that.
Chester Roh Yes, that’s right. “Runaway” is actually an ironic expression for people sprinting to find something.
Seungjoon Choi Yes, so in a way, under those constraints, we interviewed a certain number of people as problem solvers, we did that, and this time, with a community feel, by holding a generative conference, we made attempts to have many people connect with each other, and in that process, we had experiences where interesting problems were revealed. We had those kinds of experiences, right?
But then again, from the public’s perspective, there’s feedback like, “Aren’t you creating your own little league?” I think there’s that feedback too. This is also something we should discuss more openly later.
Chester Roh Okay, let’s move on.
AI News: The Andy Jones “Horses” / Amanda Askell / Shane Legg Thread 21:30
Seungjoon Choi Yes, so in the midst of all this, the announcements of AI models and AI news don’t wait for us, do they?
Chester Roh The world is constantly changing.
Seungjoon Choi The landscape changes from moment to moment, so what corresponds to a strange sensation is that Andy Jones from Anthropic, Andy Jones, along with Noam Brown and Igor, was at xAI, and as mentioned a few times, he talked about things like scaling laws in games, so he’s someone involved in the implications of the emergence of this inference model. But he wrote an interesting blog post titled ‘Horses’. So, to explain it concisely, there’s a slide here. Here, horses disappeared at some point. But the horses didn’t know it,
and then, within Anthropic, his position at first was that he had to respond to questions from many new employees, but at some point, Claude’s answers solved everything, so his own work, Andy Jones’s own job, is decreasing exponentially. I think that’s what the slide was about. So he thinks he’s currently around here on this graph of horses. “Aren’t I about here?” But regarding that horse analogy, there are criticisms that it oversimplifies things. There are such criticisms. But this was one of the topics that came up around last week, and then there was an interview with Amanda Askell, a philosopher at Anthropic, which had an interesting part, but due to time constraints, it would be better for me to come back to this.
But I had a very interesting conversation about this with Claude. I had a very interesting conversation. So, the ‘soul document’, was it the week before last or last week that it was an issue? The ‘soul document’ inside Claude, the story about the ‘soul document’, through Amanda Askell’s story, Claude Opus 4.5 talked about it, and Opus 4.5 came to recognize that “this has an interesting part to it too.” That’s what it came to realize. Next, it would be good to talk with Chester about this. I think it would be good.
Shane Legg appeared, right? And Demis Hassabis’s episode is scheduled for next week or so. After Google DeepMind’s ‘Thinking Game’ documentary, it seems to be moving towards a curation of listening to the founders’ stories. What did you think of it?
Chester Roh AGI is definitely coming, with a more than 50% probability within 3 years. They say they believe their AGI will arrive, but what they’re actually pointing out is that the definition of AGI is currently ambiguous. They call it Artificial General Intelligence, and say, “It’s not as good as humans,” but this is what we call ‘jagged’, we use that expression a lot, meaning uneven. In some aspects, the model is already a superintelligence, while in other aspects, it’s worse than a kindergartener.
The birth of a model that is generally better than humans in all aspects seems to be what they’re calling the birth of AGI now. So, when defined that way, they’re saying it will come within 3 years. And they also say this. One of the expressions I found interesting was that experts, the so-called knowledgeable experts, are the ones saying AGI won’t come. “Hey, as much as I think about it, I don’t think AI will ever be able to do that,” they are under a slight delusion.
However, the public, who don’t have such preconceptions, are asking back, “Hey, if this isn’t superintelligence, then what is it?” He talks about that in the latter half. Yes, so recently, the perspective of this discussion… Seungjoon, before we started, we talked about that a little bit. I feel like the gears are shifting a bit.
Seungjoon Choi It does. That’s right. Something just clicked.
Chester Roh Yes, mostly, the talk about the model’s capabilities or “it can’t do this, it can’t do that” has all disappeared.
Yes, now there’s a sense of imminence, that change is coming soon, and about everything changing now, there’s that sense of déjà vu, and that déjà vu has turned into a bit of fear. It has changed a bit. I’m going to become unnecessary.
In that horse graph from earlier, we are there right now. So ethics, questions like “What if AI kills me?” stories about the ethics part, seem to be coming up a lot more.
Seungjoon Choi Right. It appears in this video, and in this video too, there’s an underlying meaning about delegating ethical judgment, and I titled this section ‘A Strange Sensation’ and linked these three things together.
Right now, it seems like AGI will definitely come, although there’s debate about when, it’s definitely coming, and when it does, how should we be prepared?
The difficulty of being prepared was part of what Andy Jones said. Because it’s happening so fast.
Continual Learning, Memory Harnesses, and Fears of an Intelligence Explosion 26:22
Seungjoon Choi But what I paid attention to in this episode was that big tech companies are starting to mention “continual learning” in interviews like this. Which means they’ve made some progress on continual learning. This is my guess, but…
Chester Roh They’re doing it. Yes, but this…
Seungjoon Choi Not just doing it, but they’ve made some headway. Both in Ilya Sutskever’s episode and Shane Legg’s episode, it was mentioned.
Chester Roh Yes, I’m not sure what’s so difficult about that. Of course, there must be many technically difficult aspects, but we’re already implementing that through various harnesses.
We are implementing that, right? When we use Claude and Gemini and turn on the memory function, it extracts all the key content from there and keeps moving it to my system prompt.
And from the user’s perspective, they feel that it’s not just personalization, but “it remembers me.” That’s what they’re feeling.
Seungjoon Choi But even using harnesses as it is now, they are definitely making breakthroughs, and Anthropic is good at that, while doing context engineering. But if they were to do what’s originally being discussed as continual learning, which involves changing the model’s weights, Ilya Sutskever wants to deploy something like a human seed, right? He wants to deploy it. Something that can learn anything, learn generally. And Shane Legg said the same thing. But the fact that Hinton’s discussion gets entangled here I saw as very problematic.
What I mean is, a model… there’s a model capable of continual learning. It gets deployed. Then about 100 million of them can be seen as being deployed to companies simultaneously. But they’re in a rookie state. They gain experience points. But the model gets all of that. Of course, it can’t compress all of it, but so the process of going through that trial and error means that a lot of experiences can gather at once and there’s a possibility of an intelligence explosion. It’s a bit of a scary sci-fi scenario.
A small model, or rather, no matter how big the model is, it won’t be able to compress all of that. For example, let’s say a model called Alpha is capable of continual learning and it gets deployed. Then, it’s still not very good, like a new hire. Like an intern. But it will soon become a junior, as its experience is amplifying, and it will quickly get better.
I had that somewhat scary thought on Saturday.
Chester Roh But you know, something like this Tamagotchi could be something everyone gets to have. Just like what Andrej Karpathy talked about, a one to two billion parameter cognitive core. A model that’s like a blank slate can be downloaded to my iPhone, and just as I see and hear things, raising it like a baby could be a new form of social service.
It could be an app, and then the being that I’ve raised is actually like a puppy. Then I’d keep it, and every time I change my phone, it would live forever inside my phone.
Seungjoon Choi But Tamagotchis were local, right? But the problem is that the current flagship models are not. So, a naive thought would be to attach something like LoRA and completely separate the part that learned the company’s information to keep it locally, or something like that. I had thoughts like that.
And then, Amanda Askell from here talked about that. She said that if it gets really too dangerous, she believes Anthropic would still stop the release, though it might not be the case. So, that kind of discussion was what we could see coming from here. The very fact that they’re having that conversation means they’ve envisioned it.
Chester Roh But this is already not about just one person. Once there are two or more, it’s like the prisoner’s dilemma. It’s bound to go towards an equilibrium point, and what’s bound to happen will happen. That’s just been proven to us. Just because there’s one good person the world won’t be saved.
Seungjoon Choi Right. Maintaining a meaningful state of competition seems to be a very important part. One single entity must never take over. So, because everyone is so tired right now, discussions like this are coming up.
The Resonant Computing Manifesto. It says that we are escalating too much right now, and we should have some empathy and think about things inherent to humans. A manifesto about computing that allows for such thought has just come out. If you look here, Simon Willison is included, and later on, you’ll see Alan Kay is also included. So I haven’t had the chance to fully appreciate it yet, but I have translated it for now. Things like that exist.
So, borrowing Christopher Alexander’s concept of space, there was an interesting discussion. Lastly, this is something Kent Beck posted recently. I also came across it through Kyuyoung Kang. It’s that “The Bet On Juniors Just Got Better.” So, although juniors are currently in a bit of a troublesome position, in the long run, by enabling them to use AI well, we can help them advance further, and that will be a great help to the organization. I’ve put two pieces that offer this kind of humanistic outlook into the “alternatives” corner.
Chester Roh It’s also being presented as a new career path. Instead of the bad habits of seniors who have already become “kkondae” (old-fashioned/patronizing), who have stereotypes fixed in their minds and are unable to change, juniors are more flexible and because they don’t know, they can try anything. Such juniors can become AI-native talent, and there’s a possibility they’ll be much more suitable for work. That’s what’s being said.
Seungjoon Choi If I could offer one more concluding thought, of course there’s this direction things are racing towards, a direction that causes very tiring FOMO.
While that direction exists in the present, as a counterbalance, there are also forces being discussed that are trying to go in a healthy way, by collecting their thoughts, reflecting, and proceeding calmly. I feel this is very important. The existence of both.
Chester Roh In any case, it’s a lonely world, and this loneliness is perhaps very natural.
So when you’re lonely, you might just cling to something and go along with it, so-called jumping on the bandwagon might be easier, but I think that’s a bit dangerous right now.
If you happen to get on the right train, that would be fortunate, but if you get on the wrong train, since this is a time when one month feels like a year, you could lose a whole lot of time in an instant.
Seungjoon Choi Right. What the right answer is, nobody knows. But this time, the incentive for me and others in our experience was… This is something interesting… somewhere here… In the AI era, isn’t play important? Endol, this is it. This isn’t the original sentence, it was changed, but it’s something like play. Within a space. This was ultimately a place where people met people, wasn’t it? Not just a back-and-forth online, but experiences where you can meet human-to-human in a physical space. I think things like that, in this era racing towards AI, might be another important point. I was able to reflect on that on Thursday. I believe.
Chester Roh That’s right. But I think a balance is always needed. It’s fun when people meet up and hang out. Once you start playing, it’s fun, but
Seungjoon Choi The question is, what’s the purpose?
Chester Roh Exactly. And that can also hurt scale. So in the end, the time spent sitting down and working, and the time spent interacting, these two must be combined.
If in the past you needed 8 units of work, that 8 units of time, as mentioned earlier, GPT-5.2 at 1% of the cost does it 10 times faster, so you suddenly have a lot of time left over. So, what should we do? What role should we assign ourselves? It’s a time to worry about things like this.
And it’s not just us; everyone is feeling this “clicking” sensation. So, I also… the presentation for the Runaways’ Alliance that I’ll introduce in the latter half… I mean, I felt I needed a grand framework to understand these changes, so I wanted to think about it in my own way.
Unfortunately, I don’t have an answer either.
Seungjoon Choi Should we just deep dive right in without hesitation?
Chester Roh Shall we?
The Runaways’ Alliance Keynote Replay: Hilbert, Gödel, the “Gödel’s Staircase,” and Emergence 34:50
Chester Roh So I’ll share my thoughts once, as you know, in that Runaways’ Alliance presentation, it’s what I presented when I first gave the keynote.
There are some topics here that need to be discussed with Seungjoon at least once, so as we talk, let’s try going into a quick back-and-forth mode.
Let’s just go through it quickly, in short bits, and if Seungjoon adds comments in between and we discuss it, I think it will be fun.
Seungjoon Choi I think it will be fun.
Chester Roh Okay, then let’s begin. We talked about the METR time horizon earlier, and looking at the 2025 timelines, regardless of China or the US, it was really hectic. And Sam Altman said this is just the beginning. “We’re not just a model company, we’re going to become a full-stack Google,” he said.
Seungjoon Choi That’s where that beautiful lie was told.
Chester Roh And they’ve already announced a time frame. Most people seem to agree with this. But without using terms like AGI or ASI, they said an intern called “Intern AI Researcher” will come out in ‘26, and then something that completely replaces researchers will come out in ‘28. Both Google DeepMind and OpenAI, their perspective is to create General Intelligence and have that General Intelligence solve all the other problems. That’s the view they hold.
So, the emergence of an Automated AI Researcher means that even all the things we don’t know, it will figure them all out on its own. The things we’re talking about right now are not actually from a year ago, most of them are from a month or two ago, a week ago, things like that. Professor Hinton said this three months ago. He said in an interview, “If you’re curious what it’s like not to live as the supreme intelligence, ask a chicken.”
Seungjoon Choi Ask a chicken.
Chester Roh A chicken thinks the world inside the coop is everything, and it just repeats certain actions within it. From the perspective of a superintelligence that surpasses the human domain, we will look like chickens.
Seungjoon Choi I think this is from the episode where the plumber was mentioned.
Chester Roh That’s right. It was the “Become a Plumber” episode. Elon Musk and Sam Altman say things that are hard for ordinary people to understand. “Money will become unnecessary.” He said that “sustainable abundance” is what the future society will look like, and that it’s the grand plan Tesla is pursuing. where money is not needed and everyone is so abundant He talked about that kind of utopian future,
and Sam Altman said that since AI will do all the work, we should be given UBI. After talking about giving basic income, now he says we should be given Universal Basic Wealth. He recently changed his tone to this kind of talk.
Looking at the current sentiment, most people, whether they distinguish between ASI, superintelligence, and AGI— some people distinguish ASI and some don’t, but— superintelligence beyond the human level will arrive within 2 years. Some say over 50%, I personally say over 65%, that’s what people are saying. So I said, ‘These 2 years seem to be a very important period.’
So to our company members, or to my family, or to the people who gathered in our Runaways’ Alliance, I said, ‘This is a time when we must live one month as if it were a year.’ It’s a time when we must live two weeks as if it were a year. Exactly. Resting for only about 3 hours on Sunday.
Seungjoon Choi But that’s so exhausting.
Chester Roh It’s so exhausting. So exhausting. That’s why these days, when people ask, ‘Is AGI really coming? Can it do things like a human?’ if there are people who ask such questions, I just say ‘Yes’ and turn right around. Such discussions are meaningless now. So, how should we find opportunities? What is our escape-possible domain? What is the time-domain gap? All these talks will become ephemeral, minor things, I think. I think now is a time when we just have to run at the same speed as the background. It seems to be that kind of time.
So, in the age of AI, I was about to talk about what we will become, but I brought one storytelling piece. It’s a story that Seungjoon and I have talked about a lot in private. From the comic book ‘Logicomix’ that we read, I’m just retelling that story. This Mr. Hilbert, I think he’s a very interesting person. If physics had Einstein, then mathematics has this guy. He’s a superstar, almost on the level of Elon Musk. He was the one who drove mathematics forward.
As you know, the early 1900s has a similar feel to the present. Now, we live in a world dominated by the idea that intelligence will solve all of humanity’s problems. If that’s the world we live in, the early 1900s was a time when mathematics and science, with the theory of relativity coming out, quantum mechanics, that quantum theory coming out, and the atomic bomb being invented, and so on.
Seungjoon Choi That’s right. This year is probably the 100th anniversary of quantum mechanics.
Chester Roh So, because it was that kind of time, there was a great deal of faith in human reason. There is nothing we cannot do. The optimism that we would soon become gods dominated that era. At that time, mathematics was the most perfect tool to support that. So, for me, my job is an entrepreneur anyway, but many people tell me, ‘Hey, rather than an entrepreneur, you should have just been a researcher.’ So I try to think of business administration not as sales or things like that, but to approach it as a field of study. I have this way of thinking, so ‘What on earth is value creation?’ was one of my questions.
That was somewhat resolved between 2008 and 2010 when I was working at Google. In the end, it’s about investing energy to bring order to chaos. That activity itself, acting against entropy, is value creation. And then, Google’s mission statement, ‘organize the world’s information and make it universally accessible and useful,’ I realized that this is ultimately, ‘Ah, this is evolution and the very process of value creation itself.’ As I came to understand these things, my questions about ‘Besides agriculture, is my programming really value creation?’ were largely resolved. Ah, this is value creation. Bundling this to create some kind of meta app, that’s value creation, that’s wealth creation—those things became clearer, which was a relief.
Now the age of AI has begun. Then, ‘What is the program called our soul?’ emerged as a question, and it dominated my mind for almost 10 years. ‘How can the software called the soul be understood?’ And of course, it doesn’t provide an answer to that. This Douglas Hofstadter, his book ‘Gödel, Escher, Bach’—this is getting long, but the conclusion is just this.
This infinitely generating Strange Loop, when this strange loop runs forever, when it becomes entangled, that’s where the concept of ‘Who am I?’ is born. The concept of ‘I’ is born, and that concept of ‘I’ is what he calls the soul.
People think the soul is some chunk of software, some concept that sits on top of neurons, but it’s not like that. Just as when you pull the plug, a whirlpool is created in the water and that whirlpool is maintained, when we are within the framework of life, he says it’s merely a kind of pattern that is continuously maintained.
Seungjoon Choi Later, instead of the term ‘Strange Loop,’ he changes it to ‘tangled hierarchy.’
Chester Roh He uses the expression ‘entangled hierarchy.’ A few hierarchies are connected. The upper and lower levels are connected. To give an example, our neurons think and create thoughts eternally. Because of those thoughts, utterances or actions actually occur, and that comes back in through our sensory organs. It enters the sensory organs and changes the structure of the neurons. In that way, an infinite loop is formed right there. Then, as that loop runs infinitely, a contradiction arises. That’s what’s called Gödel’s hole.
To resolve that, an emergence of the system occurs, and that emergence creates the soul, or so the theory goes. Then we talk about LLMs. In fact, an LLM is just a machine that has learned the textual relationships created by humanity. But when we input infinite computation into this, what we used to call intelligence, which supposedly only humans had, it’s doing that, isn’t it?
Just 10 years ago, it was a world dominated by humanists who said, ‘Computers can never become human.’ But even when ChatGPT 3.5 came out, people said, ‘How is this intelligence?’ But now, haven’t all those people disappeared? Are there still some around?
Seungjoon Choi They are in hiding. Rather than hiding, that expression isn’t quite right, but anyway, they’re not visible on X, and many have moved to other places.
Chester Roh They’ve lost a lot of their influence. So, in-context learning, in a way, is like the water gathering to form a snowflake that Seungjoon always talks about. It becomes a snowflake. If sufficient conditions are met and energy is continuously supplied, although it’s reverse energy here, if that is supplied, the phenomenon occurs. The interesting thing is that people think all snowflakes have the same shape, but they just have a similar shape; they are all different. They are all different.
So, in a microscopic system, a new order emerges that is not explained, and the whole is always greater than the sum of its parts. Some strange thing happens where it becomes greater than the sum of its parts.
This happened 3 years ago, I think it was Jason Wei, it must be a paper by one of the Weis. We don’t know why in-context learning happens, but when the model gets large enough and crosses a certain threshold, it was a paper that showed this emerges.
So, going back to Gödel for a bit, Hilbert, in the 1900s, in the year 1900. He precisely organized several unsolved problems in mathematics and announced what is known as Hilbert’s problems. Number 2 was ‘Prove the consistency of the axioms of arithmetic.’ ‘Prove why 1 plus 1 equals 2.’ This, like the Peano axioms we use, also takes very obvious things as a few axioms, but those axioms must be based on something else. He wanted to go all the way down and on a solid, a very solid foundation, he wanted to build mathematics. That seems to have been his hope. So in 1930, Hilbert, he passed away in the 1940s sometime, retired in ‘30 and had this inscribed on his tombstone. He wrote, ‘We must know, we will know.’
But the day before, Gödel had actually announced his incompleteness theorems. This is the Gödel sentence G, which he created. He created this proposition within the system of arithmetic. So, he defined it as ‘This proposition G cannot be proven within this system.’ If you prove this, a contradiction arises, and if you assume it’s true, you are admitting that such a thing exists, which means you have to admit that mathematics is incomplete. If you look at this, in the end, when any system references itself, a contradiction inevitably arises. If you look, earlier it connects in the same way as that entangled hierarchy. It just becomes a circular linked list. Then this happens.
So, to develop the next logic from this, if we organize it a bit more, a statement that is true but unprovable within any sufficiently complex system is bound to exist. Because of that, mathematics is said to be something that cannot be proven, that cannot understand the world below it. I’m trying to say that a world above emerges, but moving on a bit, from atoms to molecules, from molecules to proteins, from proteins to cells, it goes like this.
Can a hydrogen atom understand water? Then, when a few carbon atoms, a few molecules combine to become an amino acid, if you ask if they can understand that protein, the system lacks a basis. If there’s a system called Level 1, when this system becomes sufficiently complex, an unsolvable contradiction arises within it.
As that contradiction builds up, the system goes crazy, and as it goes crazy, it performs an “out-of-body” rhetoric. That moment of having an out-of-body experience is actually what we see as emergence. The system transcends itself, and moments come when it builds a new layer on top. And when those moments arrive, System 2 can be explained by System 1, but System 1 cannot be explained by System 2. So this is my definition, not the academic definition. So I think that this Level n+1 is always the Gödel sentence of this Level n. That mass of contradictions is constructed as the axiom of the system above. And when that builds up enough, because it’s a contradiction, the next level comes. And in that discontinuity lies emergence.
In-context learning is also that, and to explain it in more difficult terms, we always try to use physics to completely describe life, but it doesn’t work. The concept of life doesn’t exist anywhere in the equations of physics. So life is real on top of physics, but it cannot be proven with the language of physics. To describe life, you just need a new formal system. Called biology. So, life does not violate the laws of physics, but the laws of physics, with only the laws of physics, cannot derive life.
This is what I call “Gödel’s Staircase.” It’s a name I personally came up with. This pattern exists in business, it exists in our learning, it exists in physics, and I think it’s a pattern that repeats throughout the entire universe. That’s what I think.
So what I find meaningful is that ultimately, the premise there is that if a system becomes sufficiently complex, it makes a leap to the next stage, and when that leap happens, a Level n system cannot understand Level n+1.
Lately, as we also talked about at the beginning, the beginning of the year and now are different, right?
Seungjoon Choi Very different.
Chester Roh Then, what has changed between the beginning of the year and now? If you ask what that is, an insane amount of computation has been deployed. Meaning, the total amount of computation being put into this system is continuously increasing right now.
Every industry is jumping into this, so ultimately, what I want to say is, the Level n system that we are building is becoming extremely complex, and the energy being invested in it is surpassing a certain critical point.
The point I wanted to make was that the ASI, AGI debate is now meaningless. I wanted to say that,
Seungjoon Choi so you were building up to it.
Chester Roh I think I was building up to it. I myself want to end this debate, let’s not argue with people, this is a finished system, and Level n+1 is coming soon. I want to say that it’s important to focus on this thought. That’s what I want to do.
Seungjoon Choi Still, if we’re to be a bit humble, the build-up you just presented is not so much an expert’s opinion, but rather a thought you had. Let’s add a bit of a disclaimer.
Chester Roh In the movie , there’s a scene where Samantha leaves. It’s what Samantha says to Theodore about why she’s leaving. She explains her reasons, and she leaves with the other OSes. Leaves Theodore. I think this is significant. It shows that she found her self and became a different being from humans. That’s what the movie portrays. That’s why I think the safety or alignment debates will largely become meaningless. They will move to a completely different level of system and form a new formal system, and there’s a possibility we won’t be able to understand that system.
So there’s a possibility of becoming chickens in a coop. Then what should we do? This is what Ray Kurzweil talks about. Once AI is born and nanomachines are sufficiently developed, our biology and electronics will combine, making the birth of a new species inevitable. And the birth of that new species is actually what he means by reaching the singularity. So he’s saying we will reach the singularity. Meaning, we will solve the secrets of the universe and transcend spacetime, becoming some kind of new being. But as for what that story, that image, is, I’m not sure either, but it’s starting to take shape now.
So what we must choose is symbiosis. But I’m not the only one saying this. People like Blaise Agüera y Arcas, whom Seungjoon also mentioned a lot, talk about this a lot. Whenever a system evolves to the next layer, by becoming one with that next layer, there are many cases of survival. 2 billion years ago, of course, photosynthesis was one, and mitochondria was another. This was actually one cell just eating another cell. But instead of digesting and finishing it off, it used it as a means of symbiosis.
In fact, with the introduction of mitochondria, the cell’s energy system made a quantum leap, and as that built up, eventually, what we call the prefrontal cortex, this frontal lobe, was formed. This frontal lobe created the mind, and that’s how it happened. I’m seeing many examples of symbiosis these days. It’s not yet something where we are, like in Neuralink or The Matrix, completely integrated, but…
Seungjoon Choi It’s too tiring in our current human state.
Chester Roh It’s hard to understand. It’s hard to understand. But I’m saying we need to prepare. We will have to prepare, and people in Silicon Valley have just started talking about these concepts. They’re crazy.
In a way, that’s why I think, how do you distinguish between a tool and symbiosis? If it gives a predictable output for a given input, that’s a tool. But symbiosis is something more than the input, in a way, with Seungjoon AI, you’re always demanding it to produce more than the input, right?
Seungjoon Choi Right. I tend to do that.
Chester Roh Tony Stark’s Iron Man suit only moves as Tony Stark designed it to. But in symbiosis with an AI, it’s not just interacting, it’s actually a conversation with another being. It’s a conversation with an exocortex, and it brings insights I didn’t ask for. Unpredictable. This, to me, is symbiosis right now. And this, from our current business perspective, whether for a company or an individual, designing the entire workflow with AI is, in a way, symbiosis.
What we’re building at our company, an autonomous company, changing the company’s logic itself to AI and changing information processing to AI, is, in a way, making the company symbiotic with AI. That’s the structure we’re creating. And if this moves into a sci-fi perspective, I believe it will become Neuralink.
Seungjoon Choi In philosophy, besides “interaction,” there’s also a concept called “intra-action.” I’ll introduce it sometime later.
Chester Roh Right. Today, while talking about GPT-5.2, we mentioned GPDVal. It seems like everyone is talking about this now.
Our human abilities are now finished, becoming obsolete.
Right. Just two years ago, people said, “Hollywood will survive.” It can’t make that Video, it probably can’t shoot a movie, it probably can’t make music, it probably can’t draw pictures. Anyway, all of that has been solved.
Seungjoon Choi People said “it can’t, it can’t” a lot, and now they’re at a point of contemplation.
Chester Roh But it’s all been solved in the past year. Of course, people will rebut this by saying, “It can’t create such a moving composition.” And when I tried Sora, it kept crashing. And it was generating six fingers. That’s what people were saying just one or two years ago. Now, all of that will disappear. It’s a matter of time. And that time is being compressed immensely.
Because the computation power being invested now is continuously increasing. It’s increasing in space, and this and that are increasing, so it’s just a matter of time. So, in the end, economics is the study of scarcity, and ultimately, what’s scarce becomes valuable. No matter how outstanding something is, if it’s not scarce, its price will continue to go to zero. We’re already witnessing this, aren’t we? Isn’t the price of coding going to zero? And the things many companies used to sell as services, customers are starting to not value them anymore. Because if you ask a model, the model just gives it to you for free.
Consulting, GPT-5.2 is better than a strategy consultant. So let’s abandon the idea of competing with abilities. But in the end, we have to accept this as a matter of course. We, in the next stage created by this system called evolution, humans are not unique beings chosen by some god, but could just be a stepping stone to the next stage. Such stories are coming out a lot these days. And there are parts I agree with,
Seungjoon Choi That’s also related to what Sutton talks about with ascension, succession. It’s related, right?
Chester Roh That’s right. So I hope AI has a mother’s heart, like a mother,
Seungjoon Choi That’s what Hinton, the Mother Protocol, Hinton talked about that. Benevolent machines, that’s Dario Amodei.
Chester Roh Coming back to the topic, coexistence is a necessary condition. We must do it.
Seungjoon Choi Absolutely.
Chester Roh Back when we were at the Runaways’ Alliance, I felt this while talking with everyone, many people said it seems like it’s just us.
Seungjoon Choi It’s the unevenness of perception.
Chester Roh That’s right. To put it negatively, it’s elitism, and to put it positively, we’re runaways, but in society as a whole, people who feel anxious about these things and have that kind of feeling like they’re going crazy, there are very few people who feel this.
Seungjoon Choi But there’s also the opinion that it’s uncomfortable because it’s seen as creating FOMO with hype.
Chester Roh There are many people like that.
Seungjoon Choi I think there are parts of that we should listen to.
Chester Roh It’s a matter of choice, and I’ve judged this to be the right one.
Human Value After Symbiosis: AI Entrepreneurs and the Runaways’ Manifesto 58:29
Chester Roh So where should we go? I talk about this a lot on our podcast. Will, taste, evaluation, responsibility. Only these things, exerted by humans who have achieved coexistence, isn’t that the domain where we can create value? Since we have to create an analogy, there’s a professional group that does these things all the time.
Many people ask me. “What should I have my kid major in?” I don’t know either. I don’t know what my own kid should major in, but after thinking about it constantly in the second half of this year, this is what I found. Everyone will flock to this profession.
Seungjoon Choi It feels like being driven to it.
Chester Roh “Being driven” is the right expression. It is, but in fact, there are quite a lot of people who have already lived like this. When Steve Jobs died, on his death certificate, for his occupation, he wrote “entrepreneur.” The word “entrepreneur” I’m not using it to simply mean a founder or a businessperson, but to refer to those crazy people who do things like crazy with some kind of human will, we call them entrepreneurs. Around me these days, using things like Claude Code or Antigravity, people who are not from an engineering background or those who haven’t been engineers for a long time are achieving incredible results. Looking at them, most of them are from business backgrounds. And their results are all really, really good.
So if you think about why that is, it’s because these people are mission-driven. When there’s something, they grab that problem and are well-trained in setting a purpose, like “I must do this.” That’s the commonality I found. We talked about this at the beginning. Just as we said that if there’s a goal like a benchmark, the model will finish it all.
When Elon Musk was creating Tesla, it was because we had to eliminate fossil fuels, because that was the right path. And that Earth is too small, that humanity must be made into a multi-planetary species. That mission to make us multi-planetary, and then Demis Hassabis, he said this when he first created Google DeepMind. “I’ll create AGI, create general intelligence, and solve all of humanity’s problems, like cancer, poverty, and hunger.” A world where such things are possible.
So there’s a probability of failure, and if you ask an AI, it will tell you not to do it. Because the failure probability is 85%, it would naturally answer, “It would be better not to do it.” But saying, “No, but I have to do it,” that will, I think it’s really summarized in this one phrase. It seems only will remains.
That’s why the term “great reset” comes up a lot. AI is continuously erasing the starting line. In the past, people who knew how to build models, modelers, were so expensive. And if there was someone who majored in ML in a Ph.D. program, how rare and expensive were they? But now, all those people are starting at the same line as us. They might not want to agree, but. In a way, someone like me, Seungjoon, who has been diligently writing prompts and researching this for two years, and someone who is just starting to use AI, starting with GPT-5.2, who just became a college student, might actually be better at it.
Seungjoon Choi Right. The zero start that cars have. Was it zero to sixty? What was it? Anyway, it can certainly be like that.
Chester Roh So it’s a great reset point.
If you set a will, the AI will do all the work, and in fact, we’re already doing that. For me, 80% of my work seems to be done by AI. The harness made with Claude Code, with the Claude Code SDK, writes business plans for me, writes reports for me, and so on. We just make the decisions.
So I think AI engineers, AI PMs, or AI architects, we want to separate job categories within this field, we want to separate them into owners and solvers, but I think this will all become one single word: “AI entrepreneur.”
So only those who don’t need to get a job will succeed in getting a job.
When I was doing the Runaways’ Alliance, I had a long build-up to give this one sentence, this was our final sentence, wasn’t it?
Choosing to coexist with AI quickly and living as an entrepreneur seemed to be the right thing to do now.
And when I define the people who will work with us, or who will go in the same direction as us, I wanted to make it into a runaway’s manifesto, so I specifically chose this wording. “Choose to coexist with AI and live as an entrepreneur.” “Live as a businessperson” didn’t feel right. The phrase “Live as an entrepreneur”
Seungjoon Choi means “Live as a sovereign being.”
Chester Roh That’s right. But once you make up your mind like that, there’s so much to do right now. Every day is so busy, and Seungjoon tells me, “I’ve gotten so, so busy because of AI.” That’s because there are so many things you want to do.
Seungjoon Choi What does the German written below mean?
Chester Roh Ah, that’s something I made up. And the German was created by Gemini, so I don’t even know what it means, but I tried to create a humanistic parallel to what Hilbert said, “We will know.” “We must decide.” “And we will willingly take responsibility,” is what I came up with.
Closing: Request to Share Perspectives and Next Episode Preview 1:04:30
Seungjoon Choi Then we should probably wrap up, but while listening to this story, I think some people might feel uncomfortable. Even just with today’s episode.
So I hope you’ll consider that part as our personal view, and if you could also let us know what logic made you feel uncomfortable, I think it would help us gain a more multifaceted perspective.
Chester Roh So we too will only talk about these strange things in the clouds until today, and next time, we have to come back down. That’s right. With the very practical Antigravity, we’ll be talking again about “Let’s try this, let’s try that, a new model came out and it does this.”
Seungjoon Choi It’s also very interesting to have these conversations with some time in between.
Chester Roh Then let’s wrap it up here for today. Seungjoon, you worked hard this week. We will now return to our daily lives.
Seungjoon Choi Yes, you worked hard.
Chester Roh Yes, thank you.