《How to avoid AI brain rot》英文逐字稿（Miles Cranmer / 剑桥，55 分钟，带时间轴）

主讲：Miles Cranmer（剑桥大学，助理教授；PySR / SymbolicRegression.jl 作者）· 场合：给剑桥 MPhil 学生的讲座 · 时长：约 55 分 36 秒 来源：Here is the full talk（@MilesCranmer） · 讲座介绍推文：原文 中文精读：《AI 把所有人拉向均值》

说明：这份逐字稿由 Whisper（large-v3-turbo）自动转写，只对明显的专有名词做了校正（Miles Cranmer、ChatGPT、pxstack、PySR、MAP、Hitzig et al.、Deming & Noray 等），口语重复、语气词与断句照录，未作润色。 时间轴与视频一致（mm:ss 为句首时间），方便和中文精读里引用的截图对位。 文中所有研究数据、人名与产品判断都来自讲者口述与现场幻灯片，本文未独立核验。

章节速览（视频时间轴）

• 00:00 开场：每个人都拿到了同一个外挂
• 01:26 什么是 AI brain rot：产出线还在涨，能力线停了
• 02:07 所有人都在用 AI
• 02:44 默认使用会把你拉向均值
• 04:21 语言模型的“均值坍缩”
• 05:52 谁被抬升得最多：客服与顾问
• 08:27 往上尾部走：需要主观判断时，专家赢
• 09:54 一次 14 小时的自动研究
• 11:31 你 100% 对 AI 的产出负责
• 12:26 能讲清楚，就可以交出去
• 15:11 学习会悄悄停止：土耳其的高中数学课
• 17:19 要答案，还是要解释
• 18:38 先做题还是先听课：举重比喻
• 20:17 多学四遍，还是自测三遍
• 21:52 计算器没有腐蚀大脑：一张 2×2 决策表
• 23:00 只给最小提示的 AI 家教
• 23:58 让 AI 反过来教你：苏格拉底式循环
• 29:06 演示：高斯先验与 L2 的苏格拉底式推导
• 31:10 让它搭一个你能动手玩的 demo
• 32:34 让它画图，而不是写小作文
• 34:49 agent 的默认偏好与三条规矩
• 36:51 写下你自己的标准
• 38:11 默认的 AI 产出会越来越好，曲线还会继续前移
• 39:35 站在最新工具上，哪怕只有 50% 熟练度
• 40:59 把目标抬高
• 42:13 问答

00｜开场：每个人都拿到了同一个外挂

00:00 Okay, cool. So, hi everyone. So, I’m Miles Cranmer. I’m an assistant professor in science. 00:07 So, I’ll see you all in M2. For now, I thought I could, you know, give a talk about using AI 00:15 while learning, specifically to avoid this, you know, AI brain rot and kind of atrophy of your 00:22 brain. So, you know, I guess to start with, we have all been given this alien tool. I use it all day 00:32 and I’m still figuring out how to use it. These tools are evolving so fast, like AI agents tools 00:38 that I guess I think the point to emphasize, I think, is that, you know, everybody has AI, right? 00:50 Everyone has AI. So, it could be like two or three years ago, it could be that, you know, 00:56 maybe you were the only one using AI and you kind of stood out because of that, but now everyone 01:02 has the same AI. So, the idea of like who gets a job or who gets a PhD place, whether your 01:10 work kind of stands out, this is kind of, you know, you need to understand how you can stand 01:17 out when everyone is using AI. So, we’re going to talk about understanding using AI better and

01｜什么是 AI brain rot：产出线还在涨，能力线停了

01:26 kind of adapting fast to more changes. So, what I mean by brain rot, so, I kind of mean this 01:37 is that the work you produce, you know, maybe the work that you hand in gets better over time. 01:46 Your ability to produce that yourself or to judge it, it stops growing. So, I think this is something 01:52 that a lot of us fear that AI will kind of, you’ll be able to improve our outputs, but we won’t really 02:01 be able to do that ourselves. So, there’s various interesting news articles about this over the

02｜所有人都在用 AI

02:07 past few years. And to really emphasize the scale of this, I mean, even 70 year olds, like senior 02:14 citizens are using AI. So, this is a study in 2026, U.S. adults using AI chatbots, you know, even the 02:22 senior crowd is using AI. Most young people are using AI. So, it’s really, like, you know, a vast amount 02:29 of people using this. And these AI tools, they get more and more capable over time. And to put in 02:37 perspective, like, you know, just two years ago, it was just 12%. So, using AI is, at one point, like, it was kind of like a cheat code, give you an advantage over people, but now it’s not. It’s just 12%.

02｜默认使用会把你拉向均值

02:44 It’s no longer a cheat code because everyone has it. I think what I, the way I like to think about it is it kind of pulls you towards the population mean. So, before AI, you know, the quality of everyone’s work, kind of, you could picture it on a bell curve. 03:11 Maybe, you know, we have some really good students in this program. I think, you know, people are on this side of the bell curve. 03:19 And, but the difficulty now is that with AI, you know, everyone’s kind of pulled up. So, the, you know, the weaker set of the population on any given task is kind of pushed forward. 03:35 Uh, the, the higher end of this bell curve, you know, maybe they can do it for less effort. 03:41 And so, everybody kind of collapses to this, uh, this mean, right? Because everyone has the, the same tools. 03:48 Um, so, we kind of see this phenomenon of, like, a narrowing of the, um, population. 03:55 So, this is with default AI use. So, if you just use it in a simple way, uh, the kind of default way that everyone’s using it, you know, you kind of get, you get pulled to the population mean. 04:07 So, like, if you are, uh, content with this, that’s fine, but I guess this talk is kind of about, um, uh, pushing beyond that. 04:17 So, trying to, to go into this side of the book.

04｜语言模型的“均值坍缩”

04:21 Um, so, to give you an example about this, there was this interesting study, um, they asked different AI models, write a, write a metaphor about time. 04:34 Uh, they asked 25 different language models 50 times each for a metaphor about time. 04:40 They basically all gave two clusters of answers. 04:43 So, one of the, one of the clusters was time is a river, the other one was time is a weaver. 04:48 Um, essentially, you know, these are kind of, they’re like Pareto optimal responses, but there’s not, like, a full, um, distribution of, like, novel, uh, thoughts. 05:00 So, everyone’s kind of, like, pushed to this optimal mean. 05:05 Um, you can see this, you can see the same thing with stories. 05:09 So, this is a study done on short stories. 05:14 Some of them, uh, were studied with AI, kind of giving them ideas. 05:19 So, this is the rating, and basically people with AI generated ideas, they were, their stories were rated about 8% more novel. 05:29 But the really interesting thing is that, if you look at the similarity between those stories, you know, you, you get this collapse again. 05:36 Right? 05:37 All these, all these AI ideas kind of cluster together, they get similar to each other. 05:42 So, the stories, on average, they get better, but everything starts looking the same. 05:46 Um, this is, uh, so we’ve kind of experienced this similarly.

05｜谁被抬升得最多：客服与顾问

05:52 So, you know, not to, um, not to kind of, uh, call out anyone, but, like, in, in, uh, in last year, a lot of talks, you know, from, from students, like, a lot of presentations, they looked, like, there’s this kind of clawed style of presentation. 06:11 Um, and, as a, you know, as an assessor, you can really tell, like, even the choice of font, even the color, like, it all starts looking the same. 06:20 And it’s, like, I think, like, this year, it’s the first year, like, everybody’s talks start looking the same. 06:26 Um, so you can really tell, like, you know, on average, the presentations are better, but they kind of all start looking the same. 06:32 Um, so, I guess, to, to follow up on this, like, so, the reason you get this collapse of the bell curve, um, is that, so, on routine tasks, AI helps the weakest, the, the most. 06:51 So, if you, so, this was a study on, uh, customer support workers, and, basically, there were five groups of, uh, customer support workers. 07:00 Uh, the lowest, uh, cohort saw the biggest increase, um, in their, uh, skills, or their, kind of, uh, uh, productivity, uh, but the most skilled, um, saw the, the least, uh, improvement. 07:15 Right, so, you get this, kind of, like, squashing of this, uh, distribution. 07:20 And, um, this, this gap, kind of, exists because this tool is ultimately doing most of the work. 07:27 Uh, now, on more, um, complex tasks, so, this was a study with, uh, consultants, uh, with and without AI. 07:36 So, this was, like, I think they studied, like, uh, 750, uh, consultants on some tasks that AI can do. 07:45 And, so, on the task that AI can do well, you saw an increase in, uh, task quality. 07:51 But, on the task that, um, AI was, kind of, like, the task was, like, just beyond AI, um, actually, kind of, decreased their performance a little bit. 08:01 Uh, so, you see, it, kind of, pulls down the mean a little bit. 08:06 Um, I guess, all of this is to say that, if you are, kind of, using AI heavily, to, kind of, like, replace yourself, you, kind of, become average. 08:17 Because, everybody’s going to use it the same way. 08:19 So, you don’t really stand out anymore. 08:21 Uh, and I think, you know, I think you should try to stand out, try to, uh, be ambitious.

08｜往上尾部走：需要主观判断时，专家赢

08:27 So, aim for the upper tail. 08:30 So, I think you should try to aim in this region. 08:33 So, this is, like, default AI use. 08:35 You know, maybe you’re on this side of the bell curve, and AI comes along and increases the floor. 08:40 Um, I think you should try to be in this spot. 08:43 So, I’ll talk more about, like, how I think you could try to do that. 08:47 Um, so, we’ll talk about, like, how, I guess, how people can get past the spike. 08:53 So, one of the observations made in some of these studies is that when a task requires, like, subjective, uh, judgment. 09:07 So, you, kind of, need an expert to judge the output of an AI model. 09:13 Um, then the expert can exceed the novice’s performance by a lot. 09:18 So, this was, uh, I think this was, like, this was a software task. 09:24 Um, basically, the experts who are using AI have got 33%, and novices only 15%. 09:32 Um, furthermore, uh, when sessions, so, when, when these AI sessions, kind of, hit trouble, uh, the novices were much more likely to give up than the experts. 09:42 Because the experts can, kind of, you know, they provide this overall, um, trajectory for this AI model. 09:48 Okay. 09:49 So, I guess to give a personal anecdote about this.

09｜一次 14 小时的自动研究

09:54 So, I mean, I was curious about, like, AI, uh, doing automated research. 10:00 I wanted to see, like, okay, I’ve got this, you know, uh, project idea. 10:04 I really want to just, like, push it forward and see, like, how good is AI today at doing automated research. 10:10 So, I let it run for 14 hours autonomously. 10:13 Uh, it came back and it said, you know, you know, all the goals have been proved. 10:19 Like, every task criterion has been met. 10:21 Uh, you know, like, I accomplished this research task. 10:25 Um, and then I, you know, so I have some level of expertise in this research area. 10:31 And I studied the outputs. 10:33 And it’s just, like, it’s complete garbage. 10:35 Um, you know, because it’s, like, it’s, it’s a subjective, uh, test. 10:41 So, this was, like, a symbolic regression, uh, task. 10:45 And it’s, like, the agent is, like, oh, this is a clean equation. 10:48 This is a mangled equation. 10:49 It’s, like, that doesn’t make any sense. 10:51 Like, and, um, you know, this AI agent, it was just like, 10:55 it was so confident about this. 10:56 And it put so many tokens into it. 10:58 Um, and it’s just, like, once you get to a task where it’s very subjective. 11:02 Where you can’t do just reinforcement learning on, like, a verified reward. 11:08 Uh, like, even writing. 11:10 Like, if you’ve noticed, like, AI models have gotten worse at writing over time. 11:13 It’s, like, I feel like it’s because it’s subjective. 11:16 It’s really hard to, uh, train models for subjective tasks. 11:21 Um, now, I guess, in general. 11:26 So, I think we have this, this note in the, the course book as well.

11｜你 100% 对 AI 的产出负责

11:31 So, you are 100% responsible for anything an AI generates. 11:34 Um, I think this, so, people ask, uh, you know, is there anything AI can’t do? 11:41 The one thing AI can’t do is go to jail. 11:46 So, it can never be held responsible for its outputs, right? 11:49 Like, you can’t put an AI model in jail. 11:52 You, so, they’re, they’re not liable for the things they output. 11:55 It always goes back to a human. 11:57 So, you are 100% responsible for anything an AI generates. 12:00 Even if it’s mostly automated. 12:03 You know, this is, uh, if I use an output, and it contains a mistake, the mistake is now mine. 12:09 Right? 12:10 So, um, my advice is to use AI to accelerate yourself, but not to replace yourself. 12:16 Uh, and if you use, so, this is kind of like a slogan. 12:21 If you use it to replace yourself, you will get replaced. 12:24 Um, okay.

12｜能讲清楚，就可以交出去

12:26 So, you might say that, um, okay, like the AI model, uh, it can just, it can write tests now. 12:34 You know, this AI model can write unit tests, can write integration tests. 12:37 Uh, they’re quite competent at this. 12:40 Um, and in fact, in the, in the personal research project that I ran, this 14-hour run, 12:47 it had tons of tests and goals, and it hit every single one of them. 12:52 Um, but what it didn’t do is, like, have good judgment for what tests to run. 13:00 And, like, what goals, what sub-goals to hit. 13:03 Right? 13:04 So, those were things that were subjective, and were things that I ultimately had to decide for. 13:10 Now, of course, like, this will differ by subject area, but I think a lot of that, like, 13:15 it seems like a, a big problem in AI that I don’t know if we’ll be able to solve is just this subjective, 13:21 uh, like, like, taste in what questions to ask, um, how to judge something. 13:27 Uh, this seems really hard for AI to get right. 13:30 Um, so, the check that I do for, uh, using AI in work is whether I can explain the output without the AI. 13:44 So, like, could I defend the output of this AI model well? 13:49 Um, could I, say, find its mistakes? 13:53 Uh, if I can, so if I can explain it, if I can defend the outputs, uh, then I delegate freely. 14:00 So, so nowadays I do delegate almost all of my coding. 14:05 Um, but that is because I can explain it, defend it. 14:09 Now, of course, I mean, this is like, uh, there’s subtleties to this, because on one hand, like, nobody’s writing machine code. 14:16 Right? 14:17 Like, if you, if you’re using, uh, compiler, you’re not actually writing the assembly. 14:22 So, at some level, you know, you don’t have an understanding of all the parts. 14:27 So, I think it’s important to understand, uh, at a, at a certain level, kind of understand the, um, the outputs. 14:35 Um, one thing I like to do, so I’ll talk about this a bit more later. 14:40 It’s really important to explain it kind of out loud to yourself. 14:44 Or even, you know, type it up. 14:47 Because AI makes it really easy to, um, to feel like you learned something. 14:54 To kind of feel like you understand something. 14:56 So, I, I think it’s good to try to, uh, explain it back to, uh, the model or just to yourself. 15:03 So, with AI, you can really stop learning without noticing. 15:09 So, I’ll, I’ll show some studies on this.

15｜学习会悄悄停止：土耳其的高中数学课

15:11 It’s actually, like, it’s kind of interesting that this is a thing. 15:15 So, this was a study done in Turkey with, like, a thousand high school students in Turkey. 15:22 Uh, in a maths class. 15:25 They had three groups. 15:26 So, the control group, no AI. 15:28 Right? 15:29 They, they just solve practice problems. 15:31 Now, they do worse on the practice problems. 15:33 Right? 15:34 They get, like, 28%. 15:35 Then they had another group using ChatGPT. 15:38 They get 42% on the practice problems. 15:40 And another group that does, like, a mix. 15:42 So, the, they have this ChatGPT tutor. 15:45 But it only gives them hints. 15:46 Right? 15:47 It only tells them, uh, you know, maybe look over here or try this approach. 15:52 But it never, it never gives them the solution. 15:55 Then they did an exam. 15:57 And so, on the exam, the no AI group and the hint only group, they got about the same performance. 16:07 Uh, but the group that used ChatGPT to get the solutions, uh, they were the ones who suffered the most on the exam. 16:14 So, so, so this is kind of interesting because, um, the students actually, they did not think they learned less. 16:24 It’s, it’s really funny, like, when you read these studies because you’ll see that the, the students who are, like, doing practice problems, they, they feel like they’re doing worse. 16:33 Maybe because the problems are hard. 16:36 Um, but they’re actually, uh, learning more. 16:38 So, there’s kind of, like, this, this element of struggle that’s important. 16:42 Uh, another, so this was, uh, another example of, like, not high school math, but kind of more advanced material. 16:51 Uh, so learning, uh, this Python library trio. 16:55 Uh, and they, basically, they took a quiz, uh, with, you know, I switched off. 17:02 Um, and the group that learned without AI, uh, did better. 17:06 So, there’s, this kind of, like, extends past just, you know, uh, normal educational, uh, syllabus. 17:13 Um, this was, I think this is one of my favorite studies that I’ve read on this topic.

17｜要答案，还是要解释

17:19 So, this was, I think, a university study, um, where they were learning Python. 17:28 And they gave them access to the ChatGPT. 17:33 And they basically just recorded, um, uh, like, what the students did with ChatGPT. 17:39 Now, what they saw is that every time a student asked ChatGPT for a solution, their test score, uh, dropped. 17:47 Right? 17:48 Their test score went down. 17:50 Every time they asked ChatGPT for an explanation, it went up. 17:54 So, they actually saw, like, you can actually use AI to get, you know, just, like, improve your own understanding. 18:02 But just, like, asking it the right questions. 18:04 So, if you ask it for a solution, you know, and, like, um, like, I think solutions, like, the final answer. 18:12 Like, I think it’s something that you should really value. 18:15 Because it’s hard to write a good, uh, question. 18:18 Um, and kind of, you shouldn’t just ask ChatGPT for the answer. 18:23 So, you should really, um, kind of use it as, like, a guide. 18:26 Um, because this is, uh, yeah, this study was, like, it was really clear about this. 18:31 Um, just how big an effect it was. 18:33 Um, okay.

18｜先做题还是先听课：举重比喻

18:38 Another study, uh, I mean, you know, this, you know, this phenomenon is kind of observed in many different contexts. 18:44 Um, this was comparing, I think, uh, how students felt they learned with how much they scored. 18:52 Um, and so, this was, oh yeah, so they, they ran an experiment where the students studied the problems first. 19:02 Uh, and then came to a lecture and kind of, like, saw the solutions. 19:07 And they felt they learned less, but they actually learned more. 19:12 So, it’s something like, if you, if you try a problem that’s, like, really hard, maybe your confidence gets hurt. 19:17 But, you know, ultimately you learn more, um, which is interesting. 19:21 So, it’s, it’s, in some ways it’s kind of like lifting weights. 19:25 Like, if a machine lifts the weight for me, my muscles don’t get stronger. 19:29 Um, it’s not like, oh, I just, like, put the weights on the machine. 19:34 And that’s like, it’s like you’re writing a prompt for ChatGPT. 19:38 Like, you, you do need to struggle. 19:40 Like, I, I think in learning you do need to struggle a bit. 19:42 That doesn’t mean I’m going to torture you in M2, by the way. 19:45 Like, this is, like, I think, but, uh, you know, I, I think you need to, um, you need to put cognitive effort to, uh, to learn. 19:56 So, um, okay, so psychologists, so they kind of, uh, call these, like, this process of struggle, this, uh, I think it’s, uh, desirable difficulties. 20:08 So, this kind of, um, this struggle with, uh, being stuck and feeling stuck. 20:13 Like, it does, it does, um, it helps you learn more.

20｜多学四遍，还是自测三遍

20:17 This was another interesting experiment that I think is also really cool. 20:21 So, in this study, they looked at, um, is it better to study more or study less and test yourself more? 20:33 So, here, they, um, they had one group study four times. 20:40 Right, they, they studied the material four times. 20:43 And then they wrote a test five minutes later. 20:45 Uh, they got 83% on the, uh, recall for the test. 20:49 They had a different group study once and then test themselves three times. 20:54 So, five minutes later, uh, you know, it’s kind of comparable. 20:59 The study more group actually did better. 21:01 But a week later, uh, you get this crossover. 21:05 So, the group that studied once but test themselves three times, they did, uh, much better one week later in terms of recall. 21:14 So, this process of, of, like, struggling with the material, feeling stuck, uh, that’s actually, it turns out to be really important for learning. 21:22 Um, yeah. 21:24 Can you please specify a bit more what this study and testing mean here? 21:29 Because if you’re doing some practice test problems, it’s also kind of study. 21:34 Oh, this is, uh, so this is more like reading the material. 21:38 Oh, reading. 21:39 Like reading the, or like maybe reading the solutions or something. 21:42 Okay. 21:43 Yeah. 21:44 Um, yeah. 21:45 Um, yeah. 21:46 If people have questions during the talk, feel free to just ask. 21:49 Um. 21:50 Okay. 21:51 Okay.

21｜计算器没有腐蚀大脑：一张 2×2 决策表

21:52 So, one way that you might like to think about it is like, okay, calculators. 21:58 Calculators didn’t rot anybody’s brain. 22:01 Um, but we still learn how to do this by, like, we still learn how to multiply numbers by hand. 22:08 Um, so, I think this could be, like, a nice way to think about it. 22:14 So, if you have a skill, like, if you’re, if you’re kind of, you have to use some skill that you’re not learning. 22:21 And you can check it, so you, you feel confident to check the, the output. 22:27 Uh, I think this is something that you can maybe feel comfortable to hand it over to AI. 22:32 If you cannot check it, uh, then I think you should learn enough to check the outputs, generally. 22:40 Now, if it’s a skill you are supposed to learn, um, I think you should always, you know, this is something you should be doing yourself. 22:49 Like, if it’s something you can check, uh, then do it yourself first at least, at the very least. 22:54 Um, otherwise, uh, you know, just do it yourself.

23｜只给最小提示的 AI 家教

23:00 Okay. 23:01 Um, yet another study. 23:04 So, this was, uh, I think this was, uh, oh yeah, this was an AI tutor. 23:08 Okay, basically, they, they built this, like, uh, prompt for this AI. 23:13 And the AI forced students to, uh, it never gave them a solution. 23:19 So, they basically, they had students interacting with the chatbot. 23:22 The chatbot refused to give them the solutions. 23:24 It would give them hints, uh, but never the solution. 23:27 And you can see that, uh, this really improved performance. 23:30 Um, okay. 23:33 So, ultimately, like, all of these studies are showing you that, uh, the best way to use AI and not to rot your brain is to, uh, struggle with the material. 23:44 So, kind of use AI to quiz yourself, uh, to not just, like, give you the solution right away, but maybe provide hints if you’re stuck. 23:52 But I think this feeling of being stuck, uh, is really important for, uh, learning.

23｜让 AI 反过来教你：苏格拉底式循环

23:58 Um, so, one way that I like to learn. 24:02 So, I, I, I like to learn with chatbot. 24:04 So, like, if I’m trying to learn a new topic quickly, um, I like to have AI teach me with the Socratic method. 24:11 Uh, the way I do this is I’ll have the AI ask me something. 24:16 They’ll say, okay, ask me something. 24:17 I’m gonna try to explain it back to you. 24:19 I explain it back to the AI. 24:22 The AI, like, you know, realizes, like, I don’t know something about the subject. 24:27 Uh, so maybe it gives me a hint. 24:29 And then I go read more material about it. 24:31 And I try again. 24:32 And I kind of loop through this until I get it correct. 24:35 And then I move on. 24:36 Um, I think this process is, uh, I, I like this process a lot. 24:42 Um, now it is less fun than just, like, having the AI tell you. 24:47 Um, but, you know, all the research kind of supports this for, uh, having better long-term recall of the material. 24:55 Uh, just, like, learning it better. 24:57 Um, I feel like you get a more, uh, in, you get more intuition this way as well. 25:02 Um, so I made a couple of demos. 25:07 I mean, yeah. 25:08 In this case, AI, uh, is supposed to be a good teacher. 25:14 In the presentation before with agents, that was providing just garbage. 25:18 Yeah. 25:19 You couldn’t trust the AI in that case. 25:21 Yeah. 25:22 How do you know that the AI is teaching you the right things? 25:25 Yeah, I mean, this is, um, this is, uh, tricky. 25:29 Like, I think, so also, I think that study was a little bit, um, I think it was, like, uh, I think it was like, uh, I think it was like 20, 25 or something. 25:38 So the models were worse. 25:39 Like, I think now they’re better. 25:40 I think, depending on the material, you can, um, I don’t know, you can do more or less. 25:48 So you can feed. 25:49 Like, it really depends. 25:50 Yeah. 25:51 You can feed that one of these books. 25:52 Yeah, more like a research paper. 25:53 Like, I think they’re okay if you give, uh, ChatGPT, like, a research paper. 25:58 Um, and you have it, you know, teach you the material as, like, a private tutor. 26:04 Um, I think it’s okay at that. 26:06 Like, I think nowadays it’s, it’s, uh, decent at that. 26:09 Um, I think it’s, like, really niche material. 26:11 Uh, you might not want to trust everything it does. 26:14 And sometimes they do make a lot of mistakes. 26:17 Um, I, I think it depends. 26:19 Okay. 26:20 Like, I would use that as one of many ways of learning, I think. 26:25 Yeah. 26:26 Yeah. 26:27 Any other questions? 26:28 Okay. 26:29 I have a question. 26:30 Yeah. 26:31 So this is about . 26:33 Or researcher. 26:34 Like, like, I use this in research. 26:35 But this works if, if the material exists. 26:36 But, for example, if you want to do a new calculation, do you struggle, uh, doing the new calculation? 26:37 Or do you ask AI to transfer it? 26:38 Or do you ask AI to transfer it? 26:39 And then read all the possible solution, for example. 26:40 Oh, I see. 26:41 Uh, I don’t know. 26:42 Like, I feel like there’s no general answer to that. 26:43 Like, that’s hard. 26:44 Um, I think it’s good for learning material adjacent to the problem, definitely. 26:45 I think this, like, loop of having the AI model, uh, you know, teach you in a Socratic method 26:46 style. 26:47 I think it’s good for, like, learning material adjacent to the problem, definitely. 26:52 I think this, like, loop of having the AI model, uh, you know, teach you in a Socratic method 27:09 style. 27:10 I think it’s good for, like, learning material around the subject. 27:15 Uh, for pushing to new questions, like, I feel like, I don’t know. 27:23 I mean, it really depends on your area. 27:25 Like, I think AI models are still pretty bad at some things. 27:31 I think, like, once you pick, like, a really hard problem and you try to solve it, you start 27:35 seeing, like, all the problems with AI. 27:37 Um, and, uh, I don’t know. 27:41 But, like, at the same time, like, they’re really good at math. 27:44 Um, I don’t know. 27:48 It’s hard to say, yeah. 27:49 It’s not my own experience. 27:51 So, I wanted to write about something that I was learning. 27:56 Um, and I took all the relevant, it was kind of new, right? 28:02 So, I took all the relevant papers, and I put them together. 28:06 So, I had the film that, in this case, was brought. 28:10 So, it was kind of assembling a Lego piece, a Lego thingy. 28:14 So, he was taking, or he was taking things from, I don’t know, this paper, this paper, this paper, this paper, this paper, and put them together to do something. 28:23 So, in that case, I was kind of confident to follow what the reasoning, because I knew where the sources were coming from, instead of blind looking, I mean, asking things. 28:40 But, in that case, we’ll take material from these different other resources, and books, and so on, and so on, and kind of ensemble a Lego for me. 28:49 And put all the reasoning together, so I could follow it, if that makes sense. 28:53 I don’t know. 28:54 I don’t know. 28:55 One piece. 28:56 One piece. 28:57 One piece. 28:58 I need more statistics on that. 28:59 Yeah. 29:00 I don’t know. 29:01 Yeah. 29:02 That helps. 29:03 Yeah. 29:04 Okay. 29:05 Uh, cool.

29｜演示：高斯先验与 L2 的苏格拉底式推导

29:06 Uh, yeah. 29:07 So, um, I mean, this, okay. 29:10 So, this is kind of an example of a, uh, like how you might want to use AI. 29:15 So, say you are learning about the connection between L2 regularization and a Gaussian prior, you know, kind of Bayesian context. 29:25 I think this is like an exam question last year, actually. 29:28 Uh, in this setting, you know, we’re very clear, don’t tell me the answer. 29:34 That’s kind of the, the key thing you should remember. 29:37 Don’t tell me the answer. 29:38 Ask me a question at a time. 29:40 You know, give me a hint only when I’m stuck. 29:43 But, you know, like, like let me learn. 29:46 Right? 29:47 Um, and, so the user asks this. 29:50 And then Claude follows up with, what are you maximizing when you do a MAP estimation with a prior? 29:57 Uh, and the user, you know, maybe they, they do know this part or they, they kind of struggle and think about it. 30:03 Map maximize the posterior. 30:05 Um, and then Claude says, okay, what is, right. 30:09 So then next step, like what is the log, uh, prior? 30:13 Where does Gaussian this? 30:15 And then, uh, I think it’s this. 30:17 So maybe Claude helps them with the sign. 30:20 Like, okay, does Claude, does log increase or decrease? 30:24 Uh, user asks them for a hint. 30:27 Um, and then they kind of, they figure it out, uh, through that process. 30:32 So, I think this is like a much better way to use AI than just asking for the solutions. 30:38 Uh, and as the research shows, you know, doing this process, we’re kind of struggling through the material. 30:47 You might feel like you’re learning less, which is counterintuitive. 30:51 If you just get the solutions, you might feel like you’re learning more, which is really weird. 30:56 Um, but the research shows that, uh, when you kind of do this, uh, this active learning process, you, you do learn more material. 31:06 Um, so I think it’s important to avoid brain rot.

31｜让它搭一个你能动手玩的 demo

31:10 Um, uh, one other thing that I personally like to do is just like, you know, what I like to do. 31:16 I like to build, I like to have Claude or Codex or whatever, like, build stuff. 31:21 So I can visualize a concept. 31:23 Um, because Claude and Codex, like, they’re very good at writing code quickly. 31:29 So I think it’s easy to have them generate, like, a little demo so you can explore a concept. 31:35 So for example, I say, uh, I don’t get why the learning rate matters in gradient descent. 31:41 Uh, can you build a single file web page to help me visualize it? 31:47 Um, okay. 31:51 And then Claude, like, whips up this little demo, um, and I kind of explore it from there. 32:00 Okay. 32:01 Now, I do have to warn you, like, sometimes you’ll do this and the demo will just be, like, wrong. 32:06 So, you know, don’t rely on only this. 32:10 This is, like, one tool in your learning toolbox. 32:13 But it might be helpful sometimes for learning a concept. 32:17 But I think it’s important to, um, uh, really engage with the material in an active way to not, um, you know, even if it feels harder. 32:26 That’s important for learning, I think. 32:29 Um, and we see it on the coursework to exam as well. 32:32 Okay.

32｜让它画图，而不是写小作文

32:34 Another thing you can do, so, this is, these are kind of, like, tips that I like. 32:39 So, when I am running a coding agent session, they find all sorts of new ways to mess up. 32:48 Like, they’re so, they find, like, such creative ways to, like, completely mess up the solution. 32:54 Right? 32:55 They’re, they’re brilliant in finding the wrong solutions that, like, pass the tests. 32:59 Um, so, humans absorb information really easily through visualization. 33:05 So, I think you don’t want to just, like, have a language model generate, like, essays in, in explaining something to you. 33:15 I think it’s, it’s faster to kind of have a language model, uh, build plots for you. 33:21 So, you can kind of, like, analyze it quickly, uh, through the visualization. 33:26 So, one thing that I like to do is I’ll say, okay, like, you know, you did all this explanation. 33:31 You wrote all this code, but I don’t have time to read that code. 33:34 So, just, like, make a plot. 33:36 You know, pretend you’re, like, my student and you’re bringing a plot to a meeting. 33:40 And I’m going to ask you about this plot. 33:42 Uh, so, I’ll say, oops. 33:45 Say, blah, blah, blah. 33:49 I have 15 motion sensor measurements. 33:51 And then, uh, you know, it fits something. 33:55 So, it fits an equation or whatever. 33:59 Um, and then I say, okay, I’m skeptical. 34:03 You know, this is a sonnet model. 34:05 So, it’s pretty dumb. 34:07 Like, I, I want you to plot it. 34:09 And then it plots it. 34:10 And then, like, oh, yeah, it goes through the floor. 34:12 Right? 34:13 This, this, um, this, uh, this fitted me. 34:16 So, obviously, it wasn’t a good solution, even if the model might have said it was. 34:21 Um, and you can absorb this really quickly rather than having to read through the code 34:26 and, like, you know, slowly identify that in problems. 34:29 But that’s just, like, a personal, that’s how I like to use AI. 34:32 Um, I think, so, this is kind of, like, the, I think, 34:38 some of these skills, like, you, you kind of just pick up from working with AI a lot. 34:43 There’s still not, like, a fixed, uh, syllabus for, like, how to use AI because it changes so quickly.

34｜agent 的默认偏好与三条规矩

34:49 Um, I think, uh, the current suite of, um, approaches to using AI is, uh, to, to identify, like, what AI has been optimized for. 35:03 They’ve been optimized with, uh, reinforcement learning on this suite of, you know, internal code. 35:11 And a lot of the time, they write code that passes tests, like, whatever it takes. 35:17 So, this is how they’ll write, like, a try catch. 35:21 Like, everywhere. 35:22 Because that passes tests. 35:24 Um, or they’ll produce, like, a really complicated solution to something because it passes a test. 35:30 So, uh, one of the solutions that I like for working with coding agents is just tell them, like, write the least code that works. 35:37 You know, spend a lot of effort on a small amount of code. 35:41 Always reproduce bugs first before, uh, trying to solve it. 35:46 So, like, really try to identify, like, what the bug is from, what causes this bug. 35:52 Like, deeply understand that bug before writing the solution. 35:56 Uh, and then maybe always show me a plot. 35:59 Because I can read a plot faster than an essay that the AI loves to generate. 36:04 Uh, so, this is, like, my, uh, instructions. 36:09 Um, I’m gonna skip some of this. 36:13 So, these are two skills that I like. 36:16 Ponytail is a skill that I like a lot. 36:19 This is one of my favorite skill files. 36:21 So, it’s basically just, like, a big prompt that you feed to an AI agent. 36:26 And it behaves better. 36:27 So, Ponytail is, it kind of reduces the, the slop in code. 36:34 Um, another thing I like is, is pxstack. 36:37 Uh, so this is from someone at, um, xAI or something. 36:42 Um, it’s another good workflow. 36:44 I mean, these are just, like, the current suite of tricks to improve AI. 36:48 Um, okay.

36｜写下你自己的标准

36:51 So, I think it’s important when you’re working with AI to kind of, you know, build your own standards. 36:57 So, you could have in your system instructions. 37:00 Like, when you’re teaching me something, you know, don’t just tell me the solution. 37:05 Don’t just explain it to me. 37:06 Like, you should quiz me. 37:07 You should make sure I understand the material. 37:09 You should, you know, make me work. 37:11 Make me, uh, you know, make me answer questions. 37:15 Um, when you’re writing code. 37:17 You know, maybe you tell it, always start from a public example, um, for your, uh, system instructions. 37:26 Uh, and I think the, the good part about writing this, like, global system instructions is that it lets you define what good work is. 37:36 Um, and, uh, because at the end of the day, like, this curve where AI kind of brings people together and it kind of pushes past the current population mean. 37:48 You know, it’s going to keep advancing. 37:50 Right. 37:51 So, I think it’s important to kind of, like, ride the wave, so to say. 37:56 So, the AI tools are advancing. 37:58 So, you kind of need to be on this side of it and kind of, like, ride the wave, uh, and stay in front of it. 38:04 Because this AI is only going to get better and better. 38:06 Um, and, uh, you know, so I think, you know, having these, um, habits of working with AI are really important for this.

38｜默认的 AI 产出会越来越好，曲线还会继续前移

38:17 Um, now the other thing I wanted to say is that, um, you know, some people are, you know, uh, completely pessimistic. 38:30 They think AI will replace all of us. 38:33 I don’t think that at all. 38:34 Like, I think there’s always going to be, uh, work to do. 38:38 Um, and I think one of the most important skills that I think you can have is, uh, learning. 38:46 Like, I think, I think being able to adapt rapidly to changes and pick up new skills and learn things quickly. 38:55 Like, that is the most important skill. 38:57 Especially in the current age. 38:59 Because AI is advancing so fast. 39:01 You know, we don’t even really know what skills are going to be important in 10 years. 39:06 So, I think, uh, you know, being able to learn stuff quickly is, is really important. 39:11 Um, is there a question? 39:14 Um, one thing that, you know, is kind of, uh, is surprising nowadays is, like, I think it’s better nowadays to be, like, 50% comfortable with the newest tool than to be 90% comfortable with a tool from, like, two years ago.

39｜站在最新工具上，哪怕只有 50% 熟练度

39:35 So, I think it’s always good to be, you know, at the front lines of AI tooling, um, because it’s just moving so fast. 39:42 Like, being just half as comfortable with the latest tools, uh, is, is better than being a year behind. 39:50 Uh, and this is, like, you know, this takes some getting used to kind of working with tools that you don’t understand that well. 39:57 And kind of, like, uh, you, you haven’t, uh, gotten familiar with, but I, I think it’s really important. 40:03 So, and also the habit of, like, quickly adopting new tools is good. 40:08 Um, kind of, like, the only constant is really adaptation. 40:14 Okay, so, how quickly can you learn something new? 40:18 Uh, I think, by the way, like, I think taking this MPhil program is good for a variety of reasons. 40:24 Like, not only the, uh, the, the skill set you learn and the, the, um, ideas you learn are going to be really important. 40:32 But also, it’s just, like, it teaches you to learn. 40:36 Because we’re gonna cover a lot of material in all of these modules. 40:40 So, you’re gonna learn a lot quickly. 40:42 Uh, and that is a good thing to practice in this current age of AI. 40:48 Kind of quickly learning things. 40:50 Uh, so, you know, even if you learn something you never use again. 40:54 Just this, uh, this process of learning and learning quickly is, uh, really good.

40｜把目标抬高

40:59 Because the brain transfers. 41:01 Um, I guess, on a more optimistic note. 41:05 I think that, I mean, my general approach to AI automation. 41:11 Is that, um, you know, AI can automate kinds of things. 41:16 That’s great. 41:17 Now, let’s solve even harder problems. 41:21 Because there’s, like, a never-ending list of problems that we have to solve. 41:26 Scientific questions that are unanswered. 41:28 Um, and I am very excited about AI. 41:32 Because, you know, we can automate all of these, uh, pieces. 41:36 And, I guess I’m not really worried about automation. 41:40 Because, like, there’s so many problems that we need to solve. 41:44 And, I think it’s a good time to, like, raise your ambitions. 41:47 And, you know, really go after those harder problems. 41:50 Those bigger problems. 41:51 Uh, with, with AI accelerating. 41:54 Uh, so, raise your ambitions, basically. 41:57 Uh, yeah. 41:59 And, I think, you know, in terms of what to try. 42:01 Like, I think using AI to quiz you. 42:03 To make you work. 42:04 To make you, uh, you know, really struggle through material. 42:08 Kind of pick up stuff. 42:10 Uh, I think that’s good to, to mess around with.

42｜问答

42:13 Um, and, uh, yeah. 42:15 Uh, yeah. 42:16 I’ll stop there and take questions. 42:17 Thanks. 42:26 Yeah. 42:27 So, um, at the very beginning, you talked about the diversity of answers. 42:36 Yeah. 42:37 The river, the time. 42:39 Do you have any sense of how that specific metric is? 42:44 Is it evolving in time? 42:46 Is it, are they getting better at diversifying answers? 42:49 Or is it something that is, like, a fundamental property of how they’re trained? 42:56 Um, I think, like, I heard that, uh, the latest GPT model got a bit better. 43:02 Because they basically, like, explicitly identified this problem. 43:06 And, kind of, like, made it, kind of pushed it to the side. 43:09 So, it would, like, be forced to go out of distribution. 43:12 I mean, I don’t know if this is, like, I don’t know if this is going to get solved. 43:18 Like, I mean, a lot of things in AI, you know, some people think, oh, it’s only going to get better and better and better. 43:24 But then you look at, like, actual problems. 43:26 Like, writing. 43:27 It’s gotten worse at writing. 43:29 So, there’s some things that it’s not getting better and better at. 43:32 Um, I think coding and math, it’s kind of, you know, it’s been getting better and better. 43:37 But there’s some things that seem really hard to put into the training mix. 43:42 And, you know, maybe it’s also something that it doesn’t make them revenue. 43:47 So, there’s no incentive to, kind of, reward that. 43:50 So, maybe, like, giving, like, weird, out of distribution answers is against their incentives. 43:55 I don’t know. 43:56 So, it’s really hard to know. 43:59 Yeah. 44:00 Yeah. 44:01 Um, I, I just wonder, um, AI is not able to produce content, new knowledge, right? 44:13 I mean, like, what would you say about that? 44:14 Yeah. 44:15 I mean, like, some people say this. 44:16 I don’t, like, I wouldn’t say this because I think, you know, you can arguably, uh, find 44:23 new knowledge by just, like, generating random noise and feeding it into, like, a Lean checker 44:28 until it finds, like, a new math proof. 44:30 So, there’s, like, like, you can use it to find new ideas. 44:33 Um, I think, uh, like, there’s certain, there’s a certain subset of, like, novel ideas that 44:40 you can find. 44:41 Well, there’s one of the, like, what, first, like, part of the talk was about is that, as 44:46 he mentioned that, like, the answers converge on a certain, like, cluster of concerns. 44:51 Yeah. 44:52 And I just wonder if you have, like, a, like, if you have your own, you could just read the 44:55 ideas and maybe prior to the past, like, just pace up the process of testing your ideas, 45:01 right? 45:02 Yeah. 45:03 And I’ve just come through the revelation as probably a way to open, you need. 45:07 Hence, also, why I make this kind of brain rot that comes from your own skill set. 45:11 Sorry, what was the last sentence? 45:13 So, I’m just, I think I have, I find this way of working with it very effectively. 45:18 Yeah. 45:19 But, like, using your idea to test your ideas. 45:20 Yeah. 45:21 Yeah, just because it pace you up. 45:22 Yeah. 45:23 Yeah. 45:24 Yeah, I like using this too, like, like, using it for testing your ideas, but you are still 45:34 the idea generator. 45:35 Like, definitely because, like, at least currently, like, you know, they’re good at composing ideas, 45:42 but maybe not, like, in the things that I work on, like, they have a lot of dumb ideas. 45:52 There’s, there’s certain things that, I don’t know, like, like, they’re good at kind of taking, 45:58 like, small steps and kind of getting new ideas that way. 46:01 But, like, going really far out of distribution or, like, stepping back and looking at the big 46:05 picture and, like, taking a big lead, that is harder for them, at least in, like, an ML 46:11 research area. 46:13 So, I like this approach too of, like, you know, you are still generating ideas and kind of directing 46:21 traffic and they are testing and kind of rapidly coding stuff. 46:25 They’re so good at coding stuff. 46:26 Yeah. 46:27 Yeah. 46:28 You showed one slide where you were talking about agents.md and you mentioned that you 46:33 can ask, when you’re working with code, I imagine that slide is more pertinent to coding. 46:38 So, you asked to degenerate plots before claiming results as sort of a replacement for reading 46:46 the code outside, in a way, because if you read the code, you would know where the problem is coming from. 46:52 Yeah. 46:53 If you don’t, you’re asking for a plot to figure it out. 46:56 Would you not say flowcharts, perhaps, would be a better way? 46:58 Because this is something I use in my work. 47:00 There is a lot of functions that, I don’t know, defining flowcharts actually give you a… 47:06 What is a… 47:07 Cloud chart? 47:08 Just flowcharts on how the logic is flowing. 47:10 Yeah. 47:11 Basically, what’s related to what is happening. 47:13 That would be able to follow the logic, be more of the architect. 47:17 And that way, it’s also easier to identify where the editor might be coming from your data. 47:22 Yeah. 47:23 Because by doing the plot, my only concern is as a person… 47:27 Well, I think it’s like, it depends on the task. 47:29 I think flowchart, definitely, for coding. 47:31 Like, I think it’s good, because you can, you know, build a flowchart of it, like an interactive 47:35 flowchart, and then you’re like, “Wait, that’s so dumb. 47:37 Why did they do that?” 47:38 You can click in. 47:39 And like a plot is for like, you know, a data analysis task, where maybe a flowchart is 47:44 less informative. 47:45 It’s kind of like a, you know, like a higher level programming. 47:49 So, would you think a concern be something that develops more? 47:53 So, I mean, initially, it would just be, well, at least, you want to get the point there, 47:57 you want to get the skill from there. 48:00 But that’s the only thing you start with, and once you actually have a full base. 48:05 Because initially, you could be setting up the foundation of what you’re going to work 48:10 on for some time. 48:12 But the more you work, you keep defining the standards of coding in your agent. 48:17 Yeah. 48:18 So, you keep defining the routine tasks in there. 48:20 You take that cognitive load up and focus on the identity of your idea. 48:24 But still being in complete control of the code. 48:27 At the moment, you lose it. 48:29 But this could happen, and you would never know what happened. 48:32 Yeah. 48:33 I mean, I know, like, I think it depends on the project you’re working on. 48:38 So, for my coding projects like PySR and SymbolicRegression.jl, like, there, the AI writes 48:45 all the code, but I read all of it. 48:46 So, there’s, like, I think the one exception is tests. 48:49 Tests, I kind of skim rapidly, but the actual source code, I do still read it. 48:54 And that’s, like, that’s not true in a lot of areas. 48:57 Like, some people, they don’t read code. 48:58 But they still do this broad architecture planning. 49:01 Um, yeah, I mean, I think, like, it’s super dependent. 49:07 And, of course, it’s dependent on current models. 49:09 I guess, like, one interesting story is, uh, this step of, um, having this broad human oversight. 49:16 Like, it still seems like we need that, or at least the moment. 49:21 So, there was this, um, interesting talk on how lights off software factory. 49:27 So, a lights off factory is a factory where it’s just robots. 49:32 So, you don’t need lights. 49:33 So, they turn the lights off, and the robots are just, like, building cars and stuff. 49:36 Uh, a lights off software factory is where you just have, like, coding agents iterating on a code base. 49:43 And users, like, submit bug reports and suggest features, and the coding agents kind of do it. 49:49 People have tried that. 49:51 It doesn’t work right now. 49:53 And it, it, it does work on, like, a three to six month timeline. 49:58 Like, it’ll stay alive, the code base. 50:00 But after that point, it really starts to deteriorate. 50:03 And eventually, you have to go in and, like, manually read it and see what it did. 50:08 Uh, and it introduces, like, tons of technical debt. 50:10 Um, so, with current AI, like, I don’t know, maybe the way it’s trained, uh, without, like, this long-term, uh, this long-term vision for, like, architecting and code base. 50:24 Like, it seems really bad at that point. 50:26 So, I think the human oversight of that process is still really important. 50:30 And I think software is really instructive because software is, like, three years ahead of, say, where math was. 50:37 Uh, so, like, you can kind of look at the, like, how software has evolved with AI and kind of map that into other fields. 50:44 So, like, if I want to know what this field is going to do, I can kind of say, okay, so, like, at this point in the history of software plus AI, it was here. 50:54 So, this is what it’s going to look like in a few years. 50:56 Um, yeah. 50:57 Yeah. 50:58 Yeah. 50:59 Yeah. 51:00 Yeah. 51:01 What do you think about the second line ? 51:05 Uh, so this, so, I think this is really important because sometimes when you’re using a coding agent, they will be lazy. 51:12 And they’ll just, like, they’ll see the bug and they, like, just jump into your code base and just try to fix it and, like, patch it. 51:20 But they might not have truly understood the bug. 51:23 But they might not have truly understood the deep reason. 51:27 They’ve kind of fixed the symptom. 51:28 But they haven’t fixed the disease. 51:30 Uh, so, uh, this library, what is it, pxstack? 51:36 Well, I think both Ponytail and pxstack, they have instructions, um, that are basically, like, you know, figure out the disease first. 51:46 Like, figure out exactly what the bug is from. 51:49 And then go patch it. 51:50 Because coding agents, they just love to just jump in and go patch it. 51:54 But without understanding it. 51:55 Um, yeah. 51:56 Yeah. 51:57 So, uh, yeah. 51:58 Yeah. 51:59 Uh, I have, I’m just curious about the comment that you made about, uh, uh, writing getting worse, uh, and the code and math and improving. 52:10 Okay. 52:11 Sorry. 52:12 Can you say it louder? 52:13 Yeah. 52:14 Uh, I was just curious about, uh, uh, the comment that you made about writing and getting worse. 52:18 Yeah. 52:19 Uh, with the advancement of these models. 52:21 And coding and math and improving. 52:23 Yeah. 52:24 Um, do you think, uh, there may be competing objectives, uh, between these two tasks, in the sense that, uh, we can’t really make, uh, coding and math better without, uh, uh, making writing worse. 52:39 Um, I don’t know. 52:40 Um, I don’t know. 52:41 Like, my take is that, uh, uh, it’s really hard to build reinforcement learning for writing. 52:56 Because you don’t have a simulation of the human brain to, like, see if they like the writing or not. 53:01 Like, in math, you can run, uh, lean on this, like, formalized proof and check does it compile, therefore the proof is right. 53:11 You know, with software you can kind of do something similar with unit tests and just have the agents generating code and, like, getting a reward if the test passes. 53:19 But there’s no equivalent in writing because it’s subjective. 53:22 Uh, so the early models, I think, were better because they were closer to the human, uh, pre-training data. 53:29 Where they just, like, you know, trained on pirated books and they, like, you know, they have, uh, more human language. 53:36 But then there’s no, like, RL loop to just continue to improve the writing. 53:42 Unless you were to simulate a human brain, like, you know, really get that subjective, uh, like, taste in writing. 53:49 That’s my take. 53:50 Like, I don’t know, I don’t know how they’re gonna solve that. 53:53 Right. 53:54 Yeah. 53:55 Yeah. 53:56 Yeah. 53:57 Yeah. 53:58 Uh, you said that in your research, uh, agents struggle with big leaps and, uh, like, high up here. 54:10 Yeah. 54:11 Do you do anything to actively encourage that behavior? 54:15 Uh, actively encourage, like, an out-of-distribution? 54:20 Yeah. 54:21 Yeah. 54:22 I mean, like, I think, I think the problems that AI is worse at right now is just the problems 54:27 where it’s really hard to train them. 54:29 Because the problems they’re good at, it’s, it’s easy to get, like, a binary yes/no answer. 54:34 Like, does the code compile? 54:36 But, like, long-term research trajectory, uh, that is kind of, you know, years and years of 54:42 experience in kind of spotting the right research directions. 54:46 But I think they’re good at, like, making small steps. 54:49 Um, maybe from their pre-training data. 54:52 Like, they just know more research directions. 54:55 Yeah. 54:56 Yeah. 54:57 If you’re orchestrating a lab, right? 55:00 Yeah. 55:01 And you’ve got the AI as it is. 55:04 Do you actively, like, do anything to try and make that out-of-distribution leaps better? 55:11 Um, I guess, yeah, I don’t know. 55:14 Like, it’s, it’s hard to think of, like, a general solution to that. 55:17 Um, I guess just, like, me giving the idea seems to work better for the moment, I guess. 55:21 Yeah. 55:22 Uh, okay, I think we need to wrap up now. 55:24 But if anybody has questions, I’d love to chat after. 55:26 So, thanks. 55:27 Thank you. 55:28 So, thanks. 55:29 Thank you.