Building Codex with Tibo Sottiaux

主持人 A fun story that you recently shared on X as well is how you were part of the team that built this internal Google bot that was Chad GPT but a year before Chad GPT 你最近也在 X 上分享过一个有趣的故事：你曾参与打造 Google 内部的机器人，它很像 ChatGPT，但比 ChatGPT 早了一年。

Tibo Sottiaux that caught up like you know wildfires. It felt more than a research project 它像野火一样传开了。感觉已经不只是一个研究【音乐】项目。

主持人 for Codeex you built it in Rust and at the time the model was not on distribution for Rust. 你们用 Rust 来写 Codex，当时模型对 Rust 还不在分布之内。

Tibo Sottiaux Turns out it was quite clear that Rust as a language would actually be quite good for agents fairly quickly if we decided to put some effort into it. When someone joins the Codeex team, what do you tell them? How do things get done here? 结果很快就很清楚：如果我们愿意投入精力，Rust 作为一门语言其实会非常适合做 agent。决定投入精力。有人加入 Codex 团队时，你会跟他们说什么？这里的事情是怎么做成的？

主持人 The thing that they hear the most about when they have a question is like, "Have you asked Codex?" It still surprises new starters that you can basically ask it anything. 他们有问题时听到最多的一句话就是：「你问过 Codex 了吗？」新同事还是会惊讶，原来几乎什么都可以问它。

Tibo Sottiaux Building was the fun part, but then maintenance was the painful. 开发是有趣的部分，维护才是痛苦的。

主持人 So maintenance is really sort of like a tax that you pay over time just to keep things running. But where I think it changes is like a lot of it is just going to be automated. 所以维护其实就像一种税，你要持续缴纳，才能让系统继续跑。但我认为正在改变的地方是：其中很大一部分马上就会被自动化。

Tibo Sottiaux So you still have the concept of code review. 所以你们仍然有代码审查这个概念。

主持人 The role of code review is changing. And the role of code review now is like I think Codex is one of the most popular AI coding harnesses today. But how did it all start? Many of you will know today's guest Tibo from his generous and pretty frequent Codex usage resets. He was also there when Codex as a product started and has led the broader Codex team since. Today we cover how Codex started and why it was built in Rust and made open source. How coderies are changing inside the Codex 代码审查的角色正在改变。而且现在代码审查的角色是……我觉得 Codex 是今天最受欢迎的 AI 编程 harness 之一。但这一切是怎么开始的？很多人认识今天的嘉宾 Tibo，是因为他经常很大方地重置 Codex 用量。产品形态的 Codex 起步时他就在场，之后也一直带领更大范围的 Codex 团队。今天我们会讲 Codex 是怎么起步的，为什么用 Rust 写、又为什么开源。Codex 团队内部的代码审查正在如何变化，

主持人 team and open AAI. What it means when maintenance and rearchitecting are getting ridiculously cheap. what the merge of codeex into chat GPC looked like and the many underappreciated engineering challenges of this project. If you want to understand how teams inside of OpenAI plan review and ship software, this episode is for you. This episode is presented by Turbopuffer, a ridiculously scalable, fast and cheap hybrid search engine built on top of object storage by an engineering team that I've really grown to like after spending time with them. Turbopuffer is the tool that companies like Entropic, Notion Cognition, and Harvey all use to connect their AI products to massive amounts of unstructured data. When I've talked with engineers who use Turbopuffer, the theme that always comes up is reliability and performance at scale. The reasons for this have everything to do with Turbopuffer's architecture. Turbopuffer uses only object storage for state and MVME SSDs with memory cache for compute. Data in Turbopuffer is organized into 以及 OpenAI。当维护和重新架构变得便宜到离谱时，这意味着什么。把 Codex 并入 ChatGPT 是什么样子【音乐】，以及这个项目里许多被低估的工程挑战。如果你想了解 OpenAI 内部的团队如何规划、审查和交付软件，这一集就是为你准备的。本集由 Turbopuffer 呈现。这是一个可扩展性、速度和价格都离谱的混合搜索引擎，建在对象存储之上，背后的工程团队我接触过后真的越来越喜欢。他们。Anthropic、Notion、Cognition 和 Harvey 这类公司，都用 Turbopuffer 把 AI 产品接到海量非结构化数据上。我和用过 Turbopuffer 的工程师聊过，反复出现的主题就是大规模下的可靠性和性能。原因完全在于 Turbopuffer 的架构。Turbopuffer 只用对象存储来保存状态，计算则用 NVMe SSD 加内存缓存。Turbopuffer 里的数据按

主持人 namespaces. You can think of a namespace as a database table or a search index or an S3 prefix depending on the world you come from. When a namespace is not being queried, it stays on cheap object storage with no associated compute cost. When a namespace is active, Turbopuffer pulls it up into hot caching tiers, so queries are very fast. This design fundamentally makes it effortless to scale to hundreds of millions of namespaces. If you're building a multi-tenant AI product, every user and their agent can have their own dedicated search index without any overhead. And each namespace can hold hundreds of millions of documents without any special configuration. You can scale Turbopuffer virtually without limit. And the performance, reliability, and operating model all stay the same. If you need to connect AI to lots of data, Turbopuffer should be your first choice. Check it out at turbopuffer.com/pragmatic. Tibo, welcome to the podcast. So good to have you here. Thank you for having me. It's so good to see you again. namespace 组织。你可以把 namespace 想成数据库表、搜索索引，或 S3 prefix，取决于你来自哪个领域。某个 namespace 没有被查询时，它就待在便宜的对象存储上，没有对应的计算费用。namespace 活跃时，Turbopuffer 会把它拉进热缓存层，查询就会非常快。这种设计从根上让扩展到几亿个 namespace 变得毫不费力。如果你在做多租户 AI 产品，每个用户和他们的 agent 都可以拥有自己的专用搜索索引，没有任何额外开销。每个 namespace 都可以存放几亿份文档，不需要特殊配置。你可以几乎没有上限地扩展 Turbopuffer，性能、可靠性和运行模式都保持不变。如果你需要把 AI 接到大量数据上，Turbopuffer 应该是你的第一选择。去 turbopuffer.com/pragmatic 看看。Tibo，欢迎来到播客。非常高兴你来。谢谢邀请。再次见到你真好。

Tibo Sottiaux It's good to do this. Last time we did it in person. Now, now we're doing our video. First, I I wanted to ask you, how did you get into tech? When did you first know that you want to work with computers? 能做这期很好。上次我们是面对面做的。现在改成视频了。首先我想问你，你是怎么进入科技行业的？你最早是什么时候知道自己想跟计算机打交道的？

主持人 It's a good question. It was a long long time ago. Um my my parents actually decided to move out of Brussels where I was born and just just thought it was great to um just buy a small house and refurbish it. But it was in the middle of the middle of a village with not much going on. I think there was like roughly 200 people living there. Not many that I felt like I wanted to talk to or you know could make friends with. And so I kind of got stuck uh this is like very early like 8 8 years old. I kind of got stuck cuz like you know computers and you know it's like early days of like 这是个好问题。那是很久很久以前了。我父母其实决定搬出我出生的布鲁塞尔，当时就觉得买一栋小房子翻修一下会很好。但那是在一个村子正中间，周围没什么事可做。那里大概住着两百人。我觉得想说话、能交朋友的人并不多。所以我就有点被困住了。那是很早的时候，大概八岁。我被困住了，因为电脑，那还是

主持人 the the for me the internet and you know that was my way to learn about things and so just the rest is just like you know came from that. Um I sort of like I owe it to my parents to you know have moved into the middle of nowhere and then you know I had no choice but to get interested in computers. Once you you finished high school like you went on and went to university, right? Actually studying it properly. 对我来说是互联网的早期，那是我了解事物的方式，后面的一切都是从那里来的。某种程度上我要感谢父母搬到了荒郊野外，我别无选择，只能对计算机产生兴趣。高中毕业后你上了大学，对吧？开始正经学这个。

Tibo Sottiaux Yes. Uh I I studied mathematics, applied mathematics at university. I I went there quite quite early. Um and so I I graduated early as well. Like I thought for a long time um that I would actually not make it and that I would drop out. I had like small companies and small consulting business like uh like while I was studying uh I was like working for banks. I was working for I was like um very interested in supply chain and 是的。我在大学学的是数学，应用数学。我去得相当早，所以毕业也早。有很长一段时间我以为自己撑不下去、会退学。读书期间我开过小公司，也做过小型咨询，给银行干活。我当时对供应链和

Tibo Sottiaux applied mathematics problems and I sort of like selling that and learning a lot through that. Eventually ended up in the startup world in Belgium. I did that for a little while and then moved to London to work uh initially at Google and then Deep Mind and then now you know moved uh to be here at OpenAI like this California. I love the California weather. We can talk about that. Uh it's been very good. 应用数学问题很感兴趣，差不多是在卖这套东西，也从中学到很多。后来进了比利时的创业圈。做了一段时间，然后搬到伦敦工作，先是 Google，再是 DeepMind，现在又搬到加州这边的 OpenAI。我喜欢加州的天气。这个我们可以再聊。一直很好。

主持人 Right after university you started you you've founded a startup right? You had the startup bug in you or the entrepreneur entrepreneurial bug. 大学一毕业你就创办了创业公司，对吧？你身上有创业那股劲，或者说创业的瘾。

Tibo Sottiaux Yeah. So this this startup was all about uh pharmaceutical uh supply chain um looking at the supply chain for uh clinical trials and like try to optimize and decide like hey you know should you produce more medicine where should you send it where should dispatch it like how do you avoid waste and through that making clinical trials more efficient and this was using traditional like nonML techniques um like traditional like more optimization solving Monte Carlo simulations these kinds things stochastic multi-stage optimization problem really and we also applied it on steel industry and we applied it to electrical grid as well in Europe um it's like anything that sort of had the shape of like an optimization problem we sort of like get interested in and you know to this day like this this company still exists and I think they do some of the most interesting work uh still but it's changing a lot uh you know with with 对。那家创业公司做的是医药供应链，看临床试验的供应链，试图优化并决定：要不要多生产药、该送到哪里、该怎么调度、怎么避免浪费，从而让临床试验更高效。用的是传统的、非机器学习的技术，传统的更多是优化求解、蒙特卡洛模拟这类东西，其实是随机多阶段优化问题。我们也把它用到钢铁行业，还用到欧洲的电网。凡是长得像优化问题的东西，我们都会感兴趣。直到今天这家公司还在，我觉得他们仍然在做一些最有意思的工作，但变化很大，因为有了

Tibo Sottiaux modern AI for sure 现代 AI，这是肯定的。

主持人 but it's interesting because you kind of said like oh yeah that wasn't ML it was just the traditional stuff and then you go into like Monte Carlo simulation and optimization and this algorithm I get a sense that you kind of just went deep right that it was like okay like here's a problem space like how can I use mathematics stuff that I learned stuff that I didn't learn to just go deeper and deeper do I sense that correctly 但有意思的是，你差不多是说：哦对，那不是机器学习，只是传统的东西；然后你又讲到蒙特卡洛模拟、优化和这些算法。我感觉你就是往深处钻，对吧？面对一个问题空间，怎么用学过的、没学过的数学，一层一层往下挖。我这个感觉对吗？

Tibo Sottiaux yeah that's that's why I was obsessed with applied mathematics is just really this idea of you have theoretical mathematics or you have theoretical science and physics and like there you just you you do it because there's something to be discovered and something beautiful about it and it's all about patterns and pushing the frontier but you don't necessarily always know like how you're going to apply it and then there was like the real world right this like you know there's all these cool problems that just lie around and I was like very interested in seeing like you know how can I make the world better and so like how do I apply like you know sophisticated mathematics you know to 对，这就是我痴迷应用数学的原因：就是那种想法——一边是理论。数学，或者理论科学和物理。在那边你做它，是因为有东西可发现，有美在里面，全是关于模式、把边界往前推，但你不一定总知道以后怎么应用。然后还有真实世界，对吧？周围躺着各种很酷的问题。我非常想看自己能不能把世界变得更好，所以怎么把高深的数学用到

Tibo Sottiaux just optimize the world around me and that was like a lot of the thesis behind that startup 去优化身边的世界。那就是那家创业公司背后很大一部分论点。

主持人 yeah and then after a startup you ended up at Google and first at Google London it was in 2015 and I remember 2015 Google is a really really competitive place to get into like maybe as competitive as open AI is today in terms of the industry or terms of prestige. You worked on maps initially and then you moved over to deep mind. Can you talk a little bit what what you worked on and and then why did you move on from a already really interesting space that you clearly loved you know like optimization logistics and all these things? 对，创业之后你去了 Google，先是 Google 伦敦，那是 2015 年。我记得 2015 年进 Google 竞争非常非常激烈，放到行业或声望上，也许不亚于今天的 OpenAI。你一开始做地图，后来转到 DeepMind。能不能讲讲你当时做什么，以及你为什么离开一个你显然已经很喜欢的、已经很有意思的领域，就是优化、物流这些东西？

Tibo Sottiaux Yes. I I didn't I didn't start on Google Maps. I started on a project that was meant to make the web faster. uh and make to me to uh make websites faster especially on mobile. At the time you know Google was kind of so like seeing the transition from desktop to mobile and like more and more traffic going to like you know mobile phones and so like wanted to get ahead of that. So funded like a number of a number of initiatives and projects. Um I was working on one of them. This was like really really fun because it was a small group um within actually the ads organization. It was meant to sort of like you know offset the the loss uh for the ad revenue loss because of the shift of traffic to mobile and worked on it for roughly two years. Um and then it was cancelled and although it was like the most fun I've 是的。我并不是从 Google Maps 开始的。我一开始做的项目是为了让网页更快，尤其是让网站在手机上更快。当时 Google 差不多已经看到从桌面到移动的迁移，越来越多流量走到手机上，所以想抢在前面。于是资助了不少倡议和项目。我做的是其中之一。那真的很好玩，因为是一个小组，其实在广告组织里面。目的差不多是抵消流量转向移动带来的广告收入损失。我做了大约两年。然后项目被取消了。虽然那是我

Tibo Sottiaux had on, you know, solving hard technical uh challenges. I learned a lot from not having product market fit, not having the right users, not having the right feedback loop, not trusting your product manager when they say the project is going well when in fact it's not going well at all. And then you know one day it's just like this VP flew in uh from California and then it was just like oh yeah it's like you know we're canceling this project um you know unfortunately you only have you know hundreds of users and this is clearly not Google scale and then uh it's unbelievable but people were surprised um and I think there's a lesson there um that I that I carry with me of course is you know just always question always go to like always you know deeply think about the impact that you're having but also like the importance of the overall project that you're contributing. And then I moved into Google Maps. Google Maps was super fun. Worked on reviews. And then after roughly a year, I couldn't ignore like 在解决硬技术挑战上最开心的一段。我从中学到很多：没有产品市场契合、没有对的用户、没有对的反馈回路；产品经理说项目进展很好，其实一点都不好，你不该信。然后有一天，一位副总裁从加州飞过来，接着就是：对，我们要取消这个项目，很不幸你们只有几百个用户，这显然不是 Google 该有的规模。然后令人难以置信的是，大家还感到惊讶。我觉得这里有一个教训，我一直带着：永远要质疑，永远要深入想你正在产生的影响，也要想你参与的整个项目到底重不重要。然后我转到 Google Maps。Google Maps 超级好玩。我做评价。大约一年后，我没法再忽视

Tibo Sottiaux Deep Mind. It was just it was this special place. Uh headquartered in London. So many great things were happening. This was like really the early days, you know, with rumblings of um things like AlphaGo and they just seemed to be doing extraordinary things and you know, just really tackling the very very hardest problems that you can tackle. And like with my background I was obviously drawn to that started there like I worked on uh a lot of like the research infrastructure research tooling. This is a theme that I carried on for almost a decade and it's like this is very much also the like how I approach things is how can I build tooling and products that help make others more efficient and bring a lot of utility to them. Initially I was doing this for research and then like over time you know I got like into thinking about things in a much more like more general and general and general way you know eventually like you know ending up where where I am now. DeepMind。那是一个特别的地方，总部在伦敦。好多了不起的事正在发生。那真是很早期，已经有 AlphaGo 这类东西的传闻，他们好像在做非凡的事，真正去碰你能碰到的最难的问题。以我的背景，我当然被吸引过去。我在那里做了很多研究基础设施、研究工具。这是一条我带着走了将近十年的主线。这也非常像我做事的方式：怎样做出工具和产品，帮别人更高效，给他们带来大量效用。一开始我是为研究做这些，后来越想越往更通用、更通用、再更通用的方向走，最后就到了现在这个位置。

主持人 Yeah. And a fun story that you recently shared on X as well is how you were part of the team that built this internal Google bot that was you know if you want to say similar to Chad GPT but a year before Chad GPT can you can you talk about that? That's a that that is a new story. I haven't heard it before. 对。你最近还有一个有趣的故事，也在 X 上分享过：你参与过打造 Google 内部的机器人，如果你愿意说的话，它很像 ChatGPT，但比 ChatGPT 早一年。你能讲讲吗？这是我没听过的新故事。

Tibo Sottiaux This was part of deep mind. There were like multiple efforts as well. There was like brain as well that was separate at the time. They had their own efforts on large language models, but it was definitely something that was being explored. It was not the main thrust of of deep mind. Deep mind was like very much worried um and and busy like thinking about grand challenges and games and you know thinking about RL not in the language sense. And so there was like this group um that was pushing on large language models and you know thinking about you know think what what if what if large text corpuses are everything. What if uh you just pushed language to its maximum and you just scaled language models like you know 这是在 DeepMind 里做的。当时也有多条线。Brain 当时是分开的，他们自己也在做大语言模型，但这件事确实在被探索。它不是 DeepMind 的主攻方向。 DeepMind 当时非常操心、也非常忙于思考宏大挑战和游戏，想的是强化学习，不是语言意义上的。于是有一个小组在推大语言模型，在想：如果大规模文本语料就是一切呢？如果你把语言推到极限，只去扩展语言模型，

Tibo Sottiaux would that be enough to get to general intelligence? That was like a hot debate at the time and then one one group decided to just really push on that and then it it felt really natural like you know as I was building tooling you know with others for for research is like you know obviously you're like hey you know what can we do with this model like how do we present it you know to the researcher like how can they sort of like you know debug the inputs outputs and eventually you sort of like end up with you know like a chat system. So we built that internally. We had a lot of fun. Uh initially the models were like you know kind of like almost like a little bit absurd like you know not very coherent. Uh not super useful but it was a lot of fun um to sort of like tinker with them that caught up like you know like wildfires like you know this application is just sort of like you know everyone uh was kind of like u sharing little conversations within deep mind. It felt more than like a research project or like a research a project for researchers. And so then there was this desire over time to like launch it as an 那是否足以走到通用智能？当时这是一场热议。然后有一个小组决定真的往这个方向推。接着事情就显得很自然：我当时在和别人一起给研究做工具，你当然会想，这个模型能做什么、怎么呈现给研究员、他们怎么调试输入输出，最后差不多就做成了一个聊天系统。所以我们在内部把它做出来了。我们玩得很开心。一开始那些模型几乎有点荒谬，不太连贯，也不是特别有用，但拿来摆弄非常有趣。它像野火一样传开了，这个应用在 DeepMind 内部人人都在分享一小段对话。感觉已经不只是一个研究项目，也不只是给研究员用的项目。于是过了一段时间，就有了想把它做成

Tibo Sottiaux external product. But Deepine was just like not set up, you know, there was like the the right way to launch products at Google. There's like, you know, the whole machinery of like, you know, how you do that. Um, you know, the whole like blessed production stack. You obviously very very optimized over the years to do things well. Um, but also very very hard as an environment to truly innovate. And then I I I wanted to ask what made you you know look around or or or maybe even consider open AI but I feel you partially answered this question just just putting myself back into your shoes like you're you know if it's it's 2024 or 2023 you're inside of Google who are publishing amazing papers doing really good research you're doing super fun stuff right that pushing the limits of what what's been done before it's inside a company where you already moved you know for people who are feeling kind of comfortable or good about where they are right now which I imagine you must have been like what made you still explore all right like 对外产品的愿望。但 DeepMind 当时并没有为此准备好。在 Google 发布产品有一套正确做法，整套机制都在，还有那套被认可的生产技术栈。这些年来它当然被优化得非常擅长把事情做好，但作为环境也非常难真正创新。然后我想问，是什么让你开始张望，甚至考虑 OpenAI；不过我觉得你已经部分回答了这个问题。只是把自己放回你当时的位置：假如是 2024 或 2023 年，你在 Google 内部，他们在发很棒的论文、做很好的研究，你自己也在做超级有趣的事，对吧？，在把以前做过的事情往极限推。你已经在一家公司里挪过位置了。对那些现在觉得自己待得挺舒服、挺好的人来说——我猜你当时也是——是什么让你仍然去探索：好吧？，外面。

Tibo Sottiaux what else might be there? Yeah, I was I was very comfortable uh at my it's it's a good place, but really I had a I had a desire to, you know, meet great people, but also join a mission that I truly believed in and that, you know, I felt like the people were true to the mission and cared deeply about impacting the world in in a very in a deeply positive way, but also in a direct way, not not being like, oh yeah, it's just like, you know, we just do this work over here and then it's like it's the job of someone else to figure out you know how to how to make this useful. It's like I wanted to join a group where you know like all the parameters were sort of like considered together where you know research and product were like really co-designing. Open was just like crushing it. Uh I thought chatbt was like you know taking off. I was like I was uh I met a couple people from OpenAI and then I was like wait what you know 可能还有什么？对，我当时非常舒服，那是个好地方。但我真的渴望认识优秀的人，也渴望加入一个我真正相信的使命；我觉得那里的人对使命是认真的，非常在乎以一种深刻正面的方式影响世界，而且是直接的方式，而不是那种：哦对，我们在这边做这项工作，怎么把它变得有用是别人的事。我想加入的是这样一个群体：所有参数差不多是放在一起考虑的，研究和产品真正在共同设计。OpenAI 当时简直势如破竹。我觉得 ChatGPT 正在起飞。我见了几个 OpenAI 的人，然后我就想：等等，

Tibo Sottiaux you only have like 20 people working on Chad like that is that is an insanely small number that that must be like extremely empowering like you know how does that work? How do you manage to maintain you know a product with that level of scale um and with that level of autonomy with you know only only 20 engineers and then you know as I kind of dug and dug and dug and it's like it was just a an amazing group of people amazing mission you know super talented super driven and like it was it was like drew me in and then I joined pre-ereasoning uh efforts immediately like typical openi fashions like I joined it was like oh yeah you know like there's this thing going on like you know we're going to launch reasoning models like you know it's like some new paradigm and then you know start sprinting on that and like you know like a month later like the company launched 01 uh 01 01 preview and that was exhilarating to be part of I wanted to 你们做 ChatGPT 的只有大约 20 人？这个数字小得离谱，那一定极度赋能。这是怎么运作的？你们怎么用只有 20 个工程师，去维护那种规模、那种自主程度的产品？然后我越挖越深，发现那就是一群了不起的人、了不起的使命，超级有才华、超级有驱动力，把我吸进去了。然后我立刻加入了推理出现之前的工作，典型的 OpenAI 作风：我一加入就说，对，有件事在进行，我们要发布推理模型，这是某种新范式，然后开始为此冲刺。大约一个月后公司发布了 o1、o1 preview，能参与其中令人振奋。我想待在

Tibo Sottiaux be part of like a place that moves fast cares about impact would be in tune with the world and you know just really listen um and sort of like that's also to me like you know what I've carried with me like when when building codecs when building products is like having a community listen to the community just really focus on like a really intense feedback loop uh and then building something that is just like you know you just really want to care about it and like you know care about the utility of it that it provides to the world 一个动作快、在乎影响、跟世界合拍、真正会听的地方。对我来说，这也是我带着走的东西：做 Codex、做产品时，要有社区、听社区，真正抓住非常密集的反馈回路，然后做出一样你真心在乎的东西，也在乎它给世界提供的。效用。

主持人 and then of course you started to work pretty quickly on codeex. So you joined in 2024. Can you take us back what the thinking back there when you joined was about AI or LMS and and code? I I know there was this ASWE um effort back then. The we talked about it in the deep dive as well that we did in the pragmatic engineer the autonomous software engineer 当然，你很快就开始做 Codex。你是 2024 年加入的。能不能带我们回到当时：你加入时，大家对 AI、大语言模型和代码是怎么想的？我知道当时有一个 ASWE 的工作。我们在 The Pragmatic Engineer 的深访里也谈过，就是 autonomous software engineer。

Tibo Sottiaux AS3. Yeah, that's what it's it was pronounced internally. Um we we don't we don't AS3。对，内部是这么叫的。我们现在已经没有A3 这条线了，现在就是 Codex。但对我来说，我加入后开始给研究做基础设施。我以前做的很多是大规模数据存储分析，以及用来理解训练过程的工具。这些年我做过很多不同的事，但始终是为别人而建，让他们更快，并且

主持人 have an A3 effort anymore like you know it's it's it's codeex. Um but really for me it was I I joined I started building infrastructure for research like my a lot of what I did before was large scale um data storage analysis and then tools to understand training runs. I I I did a lot of different things over my years. Um, but it was always about building for others and making them faster and just really caring about, you know, fundamentally doing that well and then through tooling and infrastructure making new things possible. And so when I joined OpenAI, I was like with the same idea and then with the re with with one preview and like you know some some of some some of the later models, it was very clear that we had to use the models themselves to help us go faster. And so I just really got obsessed with this idea of what were the limitations, how 真正在乎把这件事从根本上做好，再通过工具和基础设施让新事物成为可能。所以我加入 OpenAI 时还是这个想法。然后有了 o1 preview，以及一些更后面的模型，事情就非常清楚：我们必须用模型本身来帮我们更快。于是我非常着迷于这个想法：限制是什么，我们要怎样

主持人 were we going to use those models for research itself. So got together with um other folks in research. We started training models. We started building little agents. Those were truly the precursor to to to Codex. And like this was like we were training internal models to be very uh proficient on the Python codebase of OpenAI and then uh very proficient with you know having like good good taste in architecture, good taste in you know like code style. It was like Python only and then the idea was like you know we would sort of like use that to build infrastructure very quickly and you know help researchers code faster as well. uh and then you know and then we would move faster and then over time when you just kind like push that and simplify it to its core you're making a lot of uh you know we found like you could make a lot of progress very quickly and then learn 用这些模型来做研究本身。于是我和研究里的其他人凑到一起。我们开始训练模型，开始做小的 agent。那些才真正是 Codex 的前身。当时我们在训练内部模型，让它们非常精通 OpenAI 的 Python 代码库，也非常精通于拥有好的架构品味、好的代码风格品味。当时只针对 Python。想法是用它来很快地建基础设施，也帮研究员写代码更快。然后我们就会更快。时间一长，你把这件事往下推、简化到核心，我们发现可以非常快地取得大量进展，然后非常快地

主持人 very quickly and then Greg and Sam are you know people with they're immensely supportive and also uh Greg was very adamant that you know we would we would not just focus on ourselves but we would also focus on benefiting uh the world and so he just sort of encouraged that we would be thinking about this not just as a tool for OpenAI itself but also as something that we would actually make into a product and this is when uh we merged this research effort with this AS3 effort um and we started building one thing and then that led to a sprint which was like the initial cloud codecs that we launched which didn't really have PMF because it was like a little bit too high friction and then we also launched the codeex CLI and we continued to push but it was always this idea of 学习。Greg 和 Sam 这些人给予了极大支持。Greg 还非常坚持：我们不能只盯着自己，也要盯着让世界受益。所以他差不多是在鼓励我们：不要只把它想成 OpenAI 自己的工具，也要把它做成真正的产品。就在这时，我们把这项研究工作和 AS3 那条线合并了，开始做成一件事。接着就进入一次冲刺，也就是我们发布的第一版云端 Codex。它其实没有 PMF，因为摩擦有点太高。然后我们也发布了 Codex CLI，继续往前推，但始终是这个想法：

Tibo Sottiaux hey how do we get models to really help here. You mentioned that first you started to build this model to train on the Python code and and actually help build in for better but then you made this interesting decision where for codeex you built it in Rust and at the time the the model was not on distribution for Rust right it wasn't as good as in Rust and it was in Python or Typescript why did you make that kind of a decision was was it kind of like did you expect that it'll catch up or or you figure that performance is more important or because it was very counterintuitive most of the other harnesses built were actually not built in Rust. They were built on distribution 嘿，我们怎样才能让模型在这里真正帮上忙。你提到一开始是训练这个模型去吃 Python 代码，实际上是为了帮内部建得更好。但你们做了一个有意思的决定：Codex 用 Rust 来写。当时模型对 Rust 还不在分布之内，对吧？它在 Rust 上不如在 Python 或 TypeScript 上好。你们为什么做这种决定？是预期它会赶上来，还是觉得性能更重要？因为这非常反直觉。当时大多数其他 harness 其实都不是用 Rust 写的。它们写在分布之内的

Tibo Sottiaux on TypeScript or Python or something else. TypeScript 或 Python 或其他东西上。

主持人 Yes. From first principles like we very early on we were thinking about the product interface and the agent as different things. So it was very important um to build the core of the agent in a way that was robust, that was secure as well, that was um you know engineered for efficiency and scale and having worked through projects over the years that go from hey this is a fun thing to like, hey, we need to scale this to the scale of like the largest data center um is the ear decisions early on are like really turn out to be quite important as long as you don't sacrifice too much of the velocity and so it's like it's a 是的。从第一性原理出发，我们很早就把产品界面和 agent 当成不同的东西来想。所以非常重要的是，要把 agent 的核心做成稳健的、也安全的，并且为效率和规模而工程化。这些年来做过不少项目，从「嘿这挺好玩」走到「嘿我们要把这扩展到最大数据中心那种规模」，早期决策最后会相当重要，只要你没有牺牲太多速度。所以这是一个

主持人 it's a trade-off but we had very prolific and amazing Rust developers our internal models were not bad at Rust um and then you get a lot of uh validation as well at compile time it's like you know statically verified and all these things and that is great for agents too so turns out you know it was quite clear that you know Rust as a language would actually be quite good for agents fairly quickly if we decided to put some effort into it. But primarily we were focused on correctness and we were focused on efficiency as well. 权衡，但我们有非常高产、非常出色的 Rust 开发者，内部模型在 Rust 上也不差。而且你在编译期就能得到大量校验，静态验证这类事情，这对 agent 也很好。所以结果相当清楚：如果我们愿意投入精力，Rust 作为一门语言其实会很快就非常适合做 agent。但我们主要关注的是正确性，也关注效率。

Tibo Sottiaux Interesting. So you're saying you know it's it's worth in your case it was worth thinking ahead of where you want this thing to be and for example thing like a language choice. Obviously with agents you can rewrite a bunch of stuff and easier than in the past, but it's still like you can say save yourself reworking by putting in the right I guess scaffolding or or or well the you know the the baseline of of what you're building on right 有意思。所以你是说，就你们的情况而言，值得提前想这个东西要去到哪里，比如语言选择这种事。当然有了 agent，你重写一堆东西会比过去容易，但你仍然可以说：把正确的脚手架，或者说你所依赖的基线放对，就能省掉日后返工，对吧？

主持人 I think we could have been successful if we had written it in Typescript or you know maybe even Python and then it would have fine and then you know we would have rewritten it at some point but having a very clean separation between the agent itself which can exist irrespective of the product. Um it was a very important principle and if you write everything in the same codebase in the same language it's like inevitably you're going to be a little bit sloppy and um you're going to intertwine things more than you should and then it's going to prevent further innovation after 我觉得如果我们用 TypeScript 写，或者。甚至 Python，我们本来也可以成功，然后会没事，某时再重写。但让 agent 本身和产品干净地分开，agent 可以不依赖产品而存在，这是非常重要的原则。如果你把所有东西写在同一个代码库、同一种语言里，你难免会有点马虎，把不该缠在一起的东西缠在一起，之后就会妨碍进一步创新。

主持人 that. And so that was that was very important like the rust boundary in a sense like was very useful for that. One interesting decision that you made which is unique across all of the major labs is having this built-in open source right the CLI is open source the SDK and the app server are all open source when and why did you decide that it's not a given especially you know there used to be jokes about open AI having things closed but this this is actually the opposite where like this is open whereas like some competitors would would ship closed source harnesses which again I I think it's very easy to understand why you would want something closed source why did you want it open source There was something really cool about the idea of having the code open source because fundamentally what you're building is you're building a coding agent. And so we were sort of like thinking about well if you have that you know you're obviously going to point it at itself 所以那一点非常重要。某种意义上，Rust 这条边界对此很有用。你们做的另一个有意思、在所有大实验室里都独特的决定，是从一开始就开源，对吧？CLI 开源，SDK 和 app server 也都开源。你们什么时候、为什么决定这样？这并不是理所当然的，尤其以前还有笑话笑 OpenAI 把东西都关上；而这里正好相反，这个是开的，而有些竞争对手会发布闭源的 harness。我同样很容易理解为什么有人想闭源。你们为什么想开源？把代码开源这件事本身就有一种很酷的感觉，因为你从根本上是在做一个编程 agent。所以我们差不多在想：既然有这个东西，你显然会让它指向自己，

主持人 and you know maybe you know you can build a community of you know contributors that use it to improve it and then you know you can learn a lot from that. Also, it felt at the time is like, you know, very clear to us that if we were going to be successful, open source itself would change and the role of code itself would change and so being part of that community seemed important instead of divorced from it. I think, you know, it's it's it's hard to solve problems if you don't sort of like witness them yourself. Uh, and then the other thing was just it still feels like early, but it was very early at the time. Um, it felt like we would have some ideas for how to solve things well. Um, and we were co-designing these, you know, with with with with the training and and and and the research and it's all about expressing like the capabilities of model in like the most flexible and the best way, but also we 也许还能建立一个贡献者社区，他们用它来改进它，然后你能从中学到很多。另外，当时对我们来说很清楚：如果我们要成功，开源本身会变，代码本身的角色也会变，所以成为那个社区的一部分显得很重要，而不是跟它脱节。我觉得，如果你自己不去亲眼看见问题，就很难解决它们。另一件事是：现在仍觉得早，但当时更早。我们觉得自己会有一些把事情做好的想法。我们在和训练、研究共同设计这些东西，核心是用最灵活、最好的方式把模型的能力表达出来，但我们

主持人 didn't have all the answers and sort of being very open about, hey, this is what a good harness looks like. This is how we think about it. We did like a couple of like very technical like deep dives and blog posts and we talked about it a lot and we thought you know hey it's just like the world is vast out there's like you know crazy smart people it's like you know we're we're going to get inspired by other open source project as well and so let's just make this a level playing field and sort of like encourage a lot of tinkering um and exploration at this stage. Now this has been now you know like a a year later a year and a half later which is a very long time in right now in this AI time frame but looking back or taking the experience what are the benefits you've seen the kind of engineering benefits the engineering team's benefits from being 并没有全部答案，所以也很开放地说：嘿，好的 harness 长这样，我们是这样想的。我们做了几篇非常技术向的深访和博客，也谈了很多。我们想：世界很大，外面有超级聪明的人，我们也会被其他开源项目启发。所以干脆做成一个公平场地，在这个阶段鼓励大量摆弄和探索。现在已经过了一年、一年半，以眼下这个 AI 时间尺度来说已经很长。回头看，或者就这段经历来说，你看到的好处是什么？工程上的好处，工程团队从开源得到的好处，

主持人 open source and just honestly what are things that are kind of hard about being open source right like there must be downsides like just try trying to get an honest take on both sides 以及老实说，开源有哪些难处，对吧？一定有坏处。就想听两边都诚实的说法。

Tibo Sottiaux yeah there there there are definitely downsides it it it comes at a cost Right. Um the the benefits are it's almost something to build in the open. Uh it's awesome to have like a small a small repo as well. Like whenever we hire uh someone and they join the Codex team, it's like they've seen the repo before. They've they've looked at PRs. They're like 对，肯定有坏处，是有代价的。好处是：这几乎就是在公开场合建造。有一个小仓库也很棒。每次我们招人加入 Codex 团队，他们都已经看过这个仓库，看过 PR。他们就像

主持人 onboarding is done. 入职已经完成了。

Tibo Sottiaux You know, it's it's it's done. Yeah. It's like in onboarding is just like you use Codex to look at the repo, you know, with you and you ask them questions, but it's like it's not it's not a secret issue that you can get productive right away. We get a lot of good contributions. Although we get like you know an a tsunami of like random stuff as well. 对，已经完成了。入职差不多就是：你用 Codex 和它一起看仓库，向它提问。这不是什么秘密，你马上就能产出。我们收到很多很好的贡献，虽然也会收到海啸一样的杂乱东西。

主持人 Obviously you and everyone else right open source is changing. I think this is one of the examples. 显然你和其他人一样，对吧，开源正在变化。我觉得这就是例子之一。

Tibo Sottiaux That's right. And then to to me it just and and to a lot of the team it just brings a lot of energy to just be part of the community and like be directly contributing um not just saying that we care about the community but actually doing things that you know you can see it's it's costing us effort right. U we don't have to do it. the the downsides are, you know, it's it's separate from the rest of our code. Um so, you know, sometimes we have to draw like artificial boundaries and, you know, work across multiple repos. Um when we're working on something particularly 没错。然后对我来说，对团队里很多人来说，成为社区的一部分、直接做贡献，会带来很多能量。不只是嘴上说在乎社区，而是真的在做你看得见、也确实花了我们力气的事，对吧？我们不是非做不可。坏处是：它和我们其余的代码是分开的。所以有时我们得划出人为边界，跨多个仓库工作。当我们在做特别

Tibo Sottiaux exciting, um and uh you know, we're building it in the open then, you know, at times we find that uh you know, others copy it, you know, before we have the time to release it. And it's like it's just a little bit sad. Um, but also it's like it's part of the game, you know. It's it's like you're building in the open. It's like, you know, that's sort of like the contract that you signed is like, you know, you can copy it. Uh, we have a very permissive license as well, but it does sting a little bit when you're working on something and you're like, you know, and then uh the the third thing is just just like everyone else is like, you know, we are overwhelmed with, you know, random contributions and, you know, we have to deal with that additional tax. Um, but then that pushes us to, you know, also like try and solve for it, right? which I think is good. 激动的事情、又在公开场合建造时，有时会发现别人在我们来得及发布之前就复制了。这有点难过。但这也是游戏的一部分。你在公开场合建造，差不多就是签了这份契约：你可以复制。我们的许可证也非常宽松。但你正在做一件事的时候，还是会有点刺痛，你就想……然后第三件事和其他人一样：我们被各种随机贡献淹没，还得应付这笔额外的税。但这也会逼着我们去解决它，对吧？我觉得这是好事。

主持人 And on top of the open source, one thing that surprised me about Codeex, and I didn't even know about it until recently, it's not tied to the OpenAI models, you can use other models with Codeex, you know, like putting myself if in a vendor's shoe, it it might not be very obvious because again, all the other vendors I look at when they do a a CLI, it's kind of use it with our models. again what made you decide to be this permissive about you know using or allowing to use your harness with with other models? 在开源之上，Codex 还有一件事让我惊讶，而且直到最近我才知道：它并不绑死 OpenAI 的模型，你可以用其他模型来跑。 Codex。站在厂商的位置上，这未必那么显而易见，因为我看过的其他厂商做 CLI 时，差不多都是搭配自家模型来用。又是什么让你们决定如此宽松，允许用你们的 harness 去跑其他模型？

Tibo Sottiaux It it felt quite natural if if you are part of this community and building an excellent coding harness is like why would you couple it to your model? That that felt like sort of like quite disappointing to make that decision. So it didn't it didn't feel right. Um and in general it's like I think you know it's like I kind of try to make decisions that I'm like yes you know just like I can just sort of like explain it you know it is correct. It's the same reasoning with you know it is open source in the first place. It would have been trivial for anyone to fork it and then add support for another thing. But then but then you're just encouraging people to just like you know go and use that fork and then now suddenly you have overhead and the only reason you have a fork is because you know you wanted to change like 10 lines of code to add support for like another model provider that feels very silly. So 这感觉相当自然。如果你是这个社区的一部分，又在做一个出色的编程 harness，为什么还要把它和自家模型绑死？做那个决定会让人有点失望。所以那感觉不对。总的来说，我尽量做那种我会觉得：对，我可以把它讲清楚、它是正确的那种决定。这和一开始就开源是同一套推理。任何人 fork 一份再加对另一家的支持都轻而易举。但那样你就是在鼓励大家去用那个 fork，然后突然多出额外负担；而 fork 存在的唯一原因，只是你想改大约十行代码来支持另一家模型提供商。这感觉非常傻。所以

Tibo Sottiaux like you know why not just support it in the first place. The other thing is we benefit a lot from like being able to just give optionality. So, you know, it's like maybe today, you know, you you love using OpenAI models um and you know, you're super productive with them, but like tomorrow there's a new model that comes out, you want to try that. Why force you to go and completely change your setup just to try a new model and then we benefit from the feedback that we didn't get uh which is like, you know, maybe there's something that you liked about that model. Maybe it actually didn't work well. But it's sort of like being nice to our users and to the community is like you know feels like the right thing to do here. Um and then you know we also you we also try like other models right so and you know we try them in the same harness and you know it's just all all good and then this is all also often like um this optionality is very important to companies that we work with. This is something that, you know, we we 为什么不一开始就支持呢。另一件事是：能给出可选项，对我们好处很大。也许今天你很喜欢用 OpenAI 的模型，用它们效率极高，但明天出了新模型，你想试试。为什么要逼你为了试一个新模型就彻底换掉整套环境？然后我们还能拿到本来拿不到的反馈：也许那个模型有些地方你喜欢。也许它其实不好用。但对用户和社区友善，在这里感觉就是该做的事。而且我们自己也会试其他模型，对吧，在同一个 harness 里试，一切都很好。这种可选项对我们合作的公司也往往非常重要。这是我们

Tibo Sottiaux absolutely lean into. 绝对会靠上去的东西。

主持人 This last point, I think, you know, as any serious company, you want to have optionality and you want to use a tool that gives you that optionality. 最后这一点，我觉得任何认真的公司都想要可选项，也想用能给你这种可选项的工具。

Tibo Sottiaux But I kind of appreciate it cuz I feels to me like it's kind of honest like look like it forces the whole company to be to compete the best in everywhere in the model layer and the harness layer with open source with with choosable models and it kind of like doesn't doesn't allow you to like kick back and say like, "All right, we're done. We we can we can hang back for a little bit for now." 但我有点欣赏它，因为它对我来说有点诚实：看，它逼着整个公司在每一层都做到最好，模型层、harness 层，还开源、还可选模型，差不多不让你往后靠、说「好了，我们做完了，可以先歇一会儿」。

主持人 Yeah. I want us to win users by having, you know, the best models, the most efficient models, the best product, and then, you know, if we do all of these things, it's like we're going to have a good time. If we sort of like force you to use the product because, you know, this one thing, it's just like then I don't think that will attract, you know, the the very best people to work on this product either. And like, you know, it's like we we're doing our best work here. We care a lot about the experience. It should feel delightful, you know, like it doesn't irrespect of the model that powers it, it should feel delightful. I love the idea of winning based on merit, not based on lockin. And this is a perfect time to mention our season sponsor, Entire, who also play by the same rules. Like it or not, Git is becoming a bottleneck for modern agent heavy software development. Devs are creating more code with agents. These agents are pushing more code. Many devs are running more parallel agents. These are pushing even more code. GitHub is clearly struggling to keep up and has frequent outages. So what's the 对。我希望我们靠拥有最好的模型、最高效的模型、最好的产品来赢得用户。如果我们把这些都做到，就会过得很痛快。如果我们因为「就这一样」而强迫你用这个产品，那我觉得也不会吸引最优秀的人来做这个产品。我们在这里做最好的工作。我们非常在乎体验。它应该让人感到愉悦，不管驱动它的模型是哪一个，都应该让人感到愉悦。我喜欢凭实力赢、而不是靠锁定来赢这个想法。现在正好提到本季赞助商 Entire，他们也按同一套规则来玩。不管你喜不喜欢，Git 正在成为现代、重度使用 agent 的软件开发的瓶颈。开发者用 agent 写出更多代码。这些 agent 在推送更多代码。许多开发者在跑更多并行 agent，它们又在推送更多代码。GitHub 显然在吃力跟上，而且经常宕机。那么

主持人 solution? Entire was founded by GitHub's last coza and he rebuilt git hosting for the agentic era from scratch. Entire was built to be very fast and to have your repos regionally close to you to reduce latency allowing for fleets of agents to push in parallel. Some numbers they published. Entire can handle 418 pushes per second. That's up to 89 times faster than every competitor on the market. When GitHub is down, you can still keep working and you don't even need to migrate from GitHub. You just sign up to entire and the platform mirrors your repo. And one more neat thing, have you ever wondered what prompt resulted in this specific code being generated? I find that the prompt and conversation with the agent carries more information than the PR itself, at least for me. Entire captures all the prompt history with your agent right in the repo easy to check back and has a pretty innovative UI to show all of this. If you're looking for Git hosting that works even when GitHub is down, head to 解决方案是什么？Entire 由 GitHub 上一任 CEO 创办，他从零重建了面向 agent 时代的 Git 托管。Entire 被做成非常快，并且让你的仓库在地理上靠近你，以降低延迟，从而让成群的 agent 可以并行推送。他们公布过一些数字。Entire 可以处理每秒 418 次推送，比市场上每一个竞争对手最多快 89 倍。GitHub 宕机时你仍能继续工作，甚至不需要从 GitHub 迁走。你只要注册 Entire，平台就会镜像你的仓库。还有一件漂亮的事：你有没有想过，这段具体代码是由哪条 prompt 生成的？至少对我来说，和 agent 的 prompt 与对话携带的信息比 PR 本身还多。Entire 把你和 agent 的全部 prompt 历史直接记在仓库里，方便回头查，还有一套相当创新的界面来展示这些。如果你在找即使 GitHub 宕机也能用的 Git 托管，去

主持人 entire.io/pragmatic, io/pragmatic, install the CLI, and mirror your repo with a click. I've already done it. Oh, and did I mention that it works with any agent and it's open source? I'd also like to mention our season sponsor, Anticys. Tibo talked about how the experience of the software you use should feel delightful. Delightful includes no annoying bugs. But when you're using agents to write your code, how do you avoid chipping bugs? Reviewing every line of code is becoming a challenge with the amount of code that agents generate, which is why antithesis goes well beyond code review. Antithesis runs your whole system in a hostile simulation. This simulation includes both targeted testing and fuss testing. By running this simulation, it finds every bug before your users do. And because the simulation is fully deterministic, it doesn't only find bugs, it gives you a perfect reproduction of every issue, which makes it much easier to fix issues. The first thing I thought when I heard about antithesis is that automated bug entire.io/pragmatic，安装 CLI，一键镜像你的仓库。我已经做过了。哦，我有没有提过：它能和任何 agent 一起用，而且是开源的？我还想提本季赞助商 Antithesis。Tibo 谈到你用的软件体验应该让人感到愉悦。愉悦包括没有烦人的缺陷。但当你用 agent 写代码时，怎么避免把缺陷交付出去？agent 生成的代码量让逐行审查正在变成挑战。这就是为什么 Antithesis 远远超出代码审查。Antithesis 在充满敌意的模拟里运行你的整套系统。这种模拟既包括定向测试，也包括模糊测试。通过跑这种模拟，它能在用户之前找到每一个缺陷。而且因为模拟是完全确定性的，它不只找到缺陷，还给你每个问题的完美复现，修起来容易得多。我第一次听说 Antithesis 时，第一个想法是：自动发现缺陷

主持人 discovery and fully deterministic testing sounds like science fiction, but it's actually hardcore engineering under the hood. Jane Street, Fly.io, and the Etscd community ship agent written code with full confidence because they know it's been verified by antithesis. To see more case studies and details, head to antithesis.com/pragmatic. And with this, let's get back to Tibo and why competition between tools is great. Yeah. And I think as as an engineer like I always see that whenever there's competition as someone who's using tools it's it's always amazing like I remember like when Microsoft had with Jet Brains the IDU wars and then there's the clouds battling with each other with all the features and now of course we have the harnesses we have the models and as a as a as a user it's great because now we have more more choice they just develop faster I guess our voice gets heard a bit better. So it's it's great to hear. Speaking of the harness, can you tell me how it works 和完全确定性的测试听起来像科幻，但罩子下面其实是硬核工程。Jane Street、Fly.io 和 etcd 社区能满怀信心地交付 agent 写的代码，因为他们知道这些代码已经过 Antithesis 验证。想看更多案例和细节，去 antithesis.com/pragmatic。说到这里，让我们回到 Tibo，以及为什么工具之间的竞争是好事。对。作为工程师，我总是看到，每当有竞争，对使用工具的人来说总是很棒。我记得微软和 JetBrains 打 IDE 大战，然后云厂商用各种功能互相较量。现在当然有 harness，有模型。作为用户这很好，因为现在选择更多，它们开发更快，我想我们的声音也被听得更清楚一点。所以听到这些很好。说到 harness，你能不能讲讲它今天是怎么工作的，

主持人 today in the sense of like when I start a codeex task, does it run always on my machine? Does it choose the cloud? Does it use a sandbox? And how do I control this or or know this or how much should I know about this as as an engineer? 意思是：我启动一个 Codex 任务时，它是否总在我的机器上跑？会不会选择云？会不会用沙箱？我怎样控制或知道这些，作为工程师又该知道多少？

Tibo Sottiaux Yes. So, by default, it runs uh in uh it runs sandboxed. Um it everything that if there is like an an a command that should run with additional permissions outside of the sandbox, it will ask uh you as a user for permission, but everything every tool execution happens within the sandbox by default and it runs entirely on the machine um your local machine. And this has been the case for you know more than a year now. But it is something that is evolving and 是的。默认情况下，它在沙箱里跑。所有事情，如果有一条命令需要沙箱外的额外权限才能跑，它会向你这个用户要许可。但默认情况下，每一次工具执行都发生在沙箱里，并且完全跑在你的本地机器上。这种情况已经持续一年多了。但这件事正在演化、

Tibo Sottiaux shifting where like you can select to run this um in the cloud which then runs in like a managed VM uh where it's the same VM that you get through chatk work um and you can you can sort of like inspect it but like it runs in a kata container it's like a secure environment and so everything runs inside of that VM and it doesn't run on your machine and then the only thing that happens on your machine is like the the your input and then the streaming back of the output and so that you know obviously then uh is like much nicer on your CPU and and your machine and you can scale much much um much more and this is like just a step um it's going to be much more seamless in the future um to like you know use cloud machines and then you know maybe have a combination of like 转变：你可以选择在云上跑，那就会跑在托管虚拟机里，和你通过 ChatGPT work 拿到的是同一类虚拟机。你可以去查看它，但它跑在 Kata container 里，是一个安全环境。所以所有东西都在那台虚拟机里跑，不在你的机器上跑。然后在你机器上发生的唯一事情就是你的输入，以及把输出流式传回来。那样对你的 CPU 和机器显然友好得多，你也可以扩展得更多。而这只是一步。将来用云机器会无缝得多，也许会有一种组合：

Tibo Sottiaux partial execution on your laptop partial execution on on on cloud machines and really The thing that we're thinking about that is very natural is as models just get better and uh more capable. They can leverage so much more compute and many more resources than are available on your local machine and so it would be a constraint at some point to just limit execution on your local machine. 部分在你的笔记本上执行，部分在云机器上执行。我们在想的、也很自然的一件事是：随着模型变得更好、更有能力，它们能用掉的算力和资源会远远超过你本地机器上有的，所以某时如果把执行限制在本地机器上，就会成为约束。

主持人 Yeah. One thing that is great about it running locally, and I think the reason I I love it when it runs locally. Of course, it's a pain because if I'm doing some work, it's like, you know, I have several agents, it's it's eating CPU. If I want to close my laptop, I I I cannot kind of leave it like half open, right? When I was in one of the offices of an AI company, I I had it half open and they're like, "Are you running agents?" I'm like, "Yeah, I have one running." He's like, "I get it." 对。本地跑有一件很好的事，也是我喜欢它本地跑的原因。当然也痛苦：如果我在干活，我有好几个 agent，它们在吃 CPU。如果我想合上笔记本，我没法半开着留在那里，对吧？有一次我在一家 AI 公司的办公室，笔记本半开着，他们就说：「你在跑 agent 吗？」我说：「对，有一个在跑。」他就说：「我懂。」

Tibo Sottiaux But the reason the reason I do it because I have my local tools, I have my local Postgress database. I have my my this this and that. How are you thinking about the cloud is amazing but it doesn't have this setup or it's just a pain to set it up. are you thinking or are you experimenting with you know making the these setups and I'm kind of reminded of a topic that we talked about prea which is cloud development environments and like 2022 23 they're hot and then we talked about AI more but 但我这么做的原因是：我有本地工具，我有我的。本地 PostgreSQL 数据库。我有这个那个。你们怎么看待云？云很棒，但它没有这套环境，或者搭起来就是痛苦。你们有没有在想、或在试验怎样把这些环境做出来？这让我想起我们以前谈过的一个话题，就是云开发环境。2022、2023 年它们很热，后来我们更多谈 AI，但是

主持人 yes I think outside of large tech companies like cloud dev boxes really never took off because there's a very big upfront cost uh and then you need to pay like a maintenance cost as well and you just you know don't benefit from it as like a a solo developer or like a small team with the level of capabilities that we have in agents now is like the setup almost is free, right? So like this the setup cost and this maintenance cost is like if if your agent is capable of doing it, you know, it should just do it for you. So for 是的。我觉得在大型科技公司之外，云开发机其实从未真正起飞，因为前期成本非常大，还要付维护成本，作为独立开发者或小团队，你根本受益不到。以我们现在 agent 的能力水平，搭建环境几乎是免费的，对吧？所以搭建成本和维护成本，如果你的 agent 有能力做，它就该替你做。所以举个例子，

主持人 example, if you're saying like hey, you know, I have like I have my local um SQLite or I have a local server and MCPS and whatnot. It's like how hard is it to actually configure exactly the same setup and keep it in sync on a cloud dev box? Well, maybe it's not that hard if the model just does it for you. And so I think we're going to see a resurgence of, you know, fully cloud orchestrated uh machines which then frees you from your laptop, right? It's like one thing that we've seen a ton of success with chat work is like it's just available on your mobile. I start my day just dictating a bunch of tasks into it. uh next to the coffee and it just does it. It has access to my calendar. It has access to my my email. Uh it has access to Slack and it's just so awesome to just be able to walk around and you know get stuff done without having to you know carry my laptop everywhere and I think it's the same things like you know 如果你说：嘿，我有本地 SQLite，或者有本地服务器和 MCP 之类的。要把完全相同的环境配到云开发机上并保持同步，到底有多难？如果模型替你做，也许没那么难。所以我觉得我们会看到完全由云编排的机器重新兴起，那就会把你从笔记本上解放出来，对吧？我们在 ChatGPT work 上看到大量成功的一件事就是：它在手机上就能用。我一天的开始就是对着它口述一堆任务，就在咖啡旁边，它就去做。它能访问我的日历、我的邮件、Slack。能走来走去就把事情做完、而不用到处带着笔记本，这太棒了。我觉得是同一类事。我们发布了

主持人 we shipped like Codex remote where you know execution is like still happening on your laptop but it would be wonderful if you know you didn't have to keep your laptop open. Can you tell me a bit on how in the past how did you improve codecs? Because I remember when I first used codeex this was one of the early versions you know like you could talk to it did stuff but for example I said like all right make this change and it did that change and I had unit tests and it didn't run it and then later a few months later I don't know exactly when it just started to run it automatically. were these things did you improve the you know the the script that runs you know the instructions I'm not sure how exactly you call the you know the the bootstrapping script or whatever that is it improving the model like as a dev how can I imagine you making each version better between the harness and then between the model and like what's the connection between the two Codex remote，执行仍然发生在你的笔记本上，但如果不用一直开着笔记本就太好了。你能不能讲讲过去你们是怎么改进 Codex 的？我记得第一次用 Codex 是很早期的版本，你可以跟它说话，它会做事，但比如说我说好，做这个改动，它做了那个改动，我有单元测试，它却没跑。后来过了几个月，我不确定具体什么时候，它就开始自动跑测试了。这些事情，你们是在改进那个运行的脚本、那些指令吗？我不太确定你们怎么称呼那个引导脚本或不管它叫什么；还是在改进模型？作为开发者，我该怎么想象你们让每个版本变得更好，在 harness 和模型之间，这两者的联系是什么？

Tibo Sottiaux yeah this is a good question so the uh the harness in a sense is always a little bit ahead of the model 对，这是个好问题。所以某种意义上，harness 总是比模型稍稍超前。

主持人 oh really how so? 哦真的？怎么会？

Tibo Sottiaux Oh, what what I mean by that is that you have the model, it's capable of certain things, but then you set it up with like a couple of crutches so that it can actually do the thing um to a level of reliability and um in in a way um that is like efficient and also with the behavior that you expect as a user. And sort of like that's the role of the harness, right? is like you know provide guard rails like safety, make it more efficient, make it more like steerable, controllable and then the harness usually is also responsible for you know what we call like the the developer um message which is sort of like infected in the context uh at the start of of each turn. And so that 我的意思是：你有模型，它能做某些事，但你再给它配上几根拐杖，好让它真正能把事做到某种可靠程度，并且以一种高效的、。也带有你作为用户所预期的行为。这差不多就是 harness 的角色，对吧？提供护栏，比如安全，让它更高效，也更可引导、可控制。然后 harness 通常还负责我们所说的 developer message，差不多是在每一轮开始时注入到上下文里。所以那会

Tibo Sottiaux affects obviously like the the purpose of that to affect like the behavior of the agent throughout uh throughout the turn. A lot of what you have is like the result of of the harness and the model initially like maybe you're like oh it doesn't run tests. So you know you have to remind it to run tests and then you know we train a battle model that is uh just you know capable of like better reflecting on what is it that you really want when you ask for something uh and then you know you don't actually have to tell it anymore. So over time what we see is like the system uh the the developer message shrinks and then the harness also shrinks 影响 agent 在整一轮中的行为，这显然就是它的目的。你看到的很多东西是 harness 和模型共同的结果。一开始也许你会觉得：哦，它不跑测试。所以你得提醒它跑测试。然后我们训练出一个更好的模型，它更有能力反思：你提出一个请求时，你真正想要的是什么。然后你其实就不用再告诉它了。所以随着时间，我们看到的是系统提示、developer message 在变短，harness 也在变短。

主持人 inside of the codeex team. Do you have specific goals? Do you say like all right now the the codeex as a hardness and and model to combine is not very good at this or it's kind of doing silly mistakes or here or how can I imagine how as the engineering team how you're working on the next you know version of codeex is because the thing that I don't really get as a as a dev is like okay there's a model which to me is this magical thing which will get better of course I'm sure you have some feedback channels but you also have the harness which is the tools that you're building like that's probably what the team is responsible for how do set even your goals right like in traditional software you'll be like we will build this feature and you build that feature because you know how to do it but it feels a bit more fuzzy to me this this development process. 在 Codex 团队内部，你们有具体目标吗？你们会不会说：好，现在 Codex 作为 harness 加上模型合在一起，这件事上不太行，或者在这里会犯蠢错误？作为工程团队，我该怎么想象你们是怎样做下一版 Codex 的？因为作为开发者我不太懂的是：好，有一个模型，对我来说是个魔法一样的东西，它当然会变好，我相信你们有一些反馈渠道；但你们也有 harness，也就是你们在建的那些工具，那大概才是这个团队负责的。你们甚至怎么设目标？在传统软件里你会说我们要做这个功能，然后你去做，因为你知道怎么做。但这个开发过程对我来说感觉更模糊。

Tibo Sottiaux Yeah it is and it's why we we co-design you know most things and it's it's a process where it's a collaboration between research and the engineering team like primarily building the the core the core agent harness. It's always uh it's always a question of like okay we see today that you know we are very good at this but we're not very good at this and you know we have a desire to do like another thing because it would be a very cool products feature and then you know we always like look at it it's like 对，确实如此，这也是为什么我们共同设计。大多数事情。这是一个过程，是研究和工程团队的协作，工程团队主要在做核心的 agent harness。问题总是：好，我们今天看到这件事我们很强，那件事我们不强；我们还想做另一件事，因为那会是很酷的产品功能。然后我们总会看着它，问：

Tibo Sottiaux okay this should this be like a harness change or should this be a model change and if it's a model change like how soon can we have it can we have it in a month can we have it in you know three months six months and we sort of like work through that and then depending on you know how soon we can just fix it in the model at which level of training. Then we might decide to not even do something in the harness at all and not and just wait for for for the model to to solve it. You know, it's it's agents all the way, right? So we use agents to analyze like a lot of the feedback to like, you know, come up with themes, you know, to just help us have these conversations and decide on priorities. But we analyze it across all of coding. We analyze it across like all of like know the other domains like finance, coms, marketing, you know, all the things where our users are using these agents nowadays and there's like you know subcategories within those and then we roughly know like you know how well we perform and then we're always pushing the frontier and there's a thing that is interesting 好，这该是 harness 的改动，还是模型的改动？如果是模型改动，我们多久能拿到？一个月？三个月？六个月？我们会把这过一遍。然后取决于我们能多快在模型里修好、在训练的哪一层修好，我们也许会决定 harness 里什么都不做，直接等模型来解决。你知道，一路到底都是 agent，对吧？所以我们用 agent 来分析大量反馈，找出主题，帮我们进行这些对话、决定优先级。但我们会在全部编程场景上分析，也会在其他领域分析，比如金融、通信、营销，以及用户现在用这些 agent 的所有地方。那些里面还有子类。然后我们大致知道自己表现如何，接着总是在把边界往前推。有一件有意思的事是：

Tibo Sottiaux is like as we make you know as our pre-training model gets better as we make the overall model better like the whole thing lifts up but then there are sometimes things that we pay a little bit more attention to. You mentioned you know he analyzes agents all all the way. Can we talk about the the software development life cycle on codeex in the sense of whenever a new engineer joins a team any team it's like okay how are things done here and you know back preai it would have been you join the company like Uber or Google and they would tell you that cool the way it works is we have an idea or the PM has an idea we make a plan we get together we do some estimations we break up the work we code the work we do tests we do code reviews we release we do feature flags and then you know we we were on call that that you know that's how it used to be when someone joins the codeex team you know they they've clearly been contributing to the open source part but what what do you tell them how do things get get done here if they're like a brand total 随着我们的预训练模型变好、整体模型变好，整件事会一起抬升，但有时有些事情我们会多花一点注意力。你提到一路都用 agent 来分析。我们能不能谈谈 Codex 上的软件开发生命周期？意思是：每当新工程师加入一个团队、任何团队，都会问这里事情是怎么做成的。在 AI 出现之前，你加入公司，比如 Uber 或 Google，他们会告诉你：酷，做法是我们有一个想法，或者产品经理有一个想法，我们做计划，聚在一起，做一些估算，拆分工作，写代码，做测试，做代码审查，发布，用功能开关，然后我们值班。以前就是这样。有人加入 Codex 团队时，他们显然已经给开源部分做过贡献，但你会告诉他们什么？如果他们是完全的新手，这里的事情是怎么做成的？

Tibo Sottiaux newbie 我把他们介绍给优秀的人。然后他们有问题时听到最多的就是：你问过 Codex 了吗？在 OpenAI，Codex 默认就插进了所有东西，所以它能访问 Slack、所有文档、所有代码。新同事还是会惊讶，原来几乎什么都可以问它。

主持人 I introduce them to great people um and then the thing that they hear the most about like when they have a question is like have you asked Codex um and Codex like is just by default at openi is like plugged into everything so it has access to slack it has access to all the documents, access to all the code, and it still surprises uh new starters that you can basically ask it anything. Uh and it will very often just like come up with like a really good response. Um and so the easiest way to understand the state of a project or who's working on something or why a decision was made is like critics knows about it all internally. 它常常会给出非常好的回答。所以要了解一个项目的状态、谁在做什么、某个决定为什么做出，最容易的办法就是：Codex 在内部知道这一切。

Tibo Sottiaux Um and so you just you just use all of that. We do a lot of work in you know for that reason we do a lot of work in public channels. Um we open up uh documents with like you know fairly broad uh permissions and so that you know everyone has access to this information as well and so that you know your agent can go through things and like you know reason through things and then you know we have a couple of other things that are just really uh very helpful for uh team productivity and team collaboration that we haven't released yet but are are going to come like some of it at def day. All of that just sort of like makes you very grounded and in tune with the rest of the team. Um, and allows you to like, you know, just very very quickly like understand the state of things and and 所以你就把这些都用上。正因为如此，我们大量工作放在公开频道里做。我们把文档打开，权限相当宽，这样你每个人也能拿到这些信息，你的 agent 才能把事情过一遍、推理一遍。然后我们还有几样对团队生产力和协作非常有帮助的东西，还没发布，但马上会来，有些会在 DevDay。所有这些差不多让你非常落地，也跟团队其他人合拍，让你能非常非常快地理解现状，并且

Tibo Sottiaux and produce things yourself. The general recommendation is just like, hey, care about the user, uh, care about the coherence of the product, care about the models and where they're going. Uh, if you're doing something and you know you're building like this 10,000 lines of code crutch to work around the model flaws, like you know, you're probably doing the wrong thing. So, we have a set of principles. Um but it's just really um sort of like a team culture and ethos at this point and you know it's just very much so like carries on you know when people join it's just like through the rest of the team just like you know sort like teaching the ropes and then when I have an idea I think it's a good idea I I talk it through with codeex maybe I talk it through with some my colleagues like here's a cool new feature I'm going to build as my first contribution or first major contribution at to codeex how do I go about that obviously I I code it down with codeex I obviously test it and make sure that it works from there on what's 自己产出东西。一般建议就是：嘿，在乎用户，在乎产品的连贯性，在乎模型以及它们要去哪里。如果你正在做的事是写一万行代码当拐杖，去绕过模型的缺陷，那你大概在做错事。所以我们有一套原则。但到这一步，它其实已经是团队文化和气质，当人们加入时，它就这样通过团队其他人传下去，教他们门道。然后，当我有一个想法，觉得它不错，我会跟 Codex 过一遍，也许也跟一些同事过一遍。比如说：这是一个很酷的新功能，我要把它作为我给 Codex 的第一次贡献，或第一次重大贡献。我该怎么做？显然我会用 Codex 把它写下来，显然会测试并确保它能用。从那之后，

Tibo Sottiaux the process do you still have the concept of code review or AI code review of verification of rolling out of verifying of stage rolled out you know the things because codex itself it goes out to millions of people like I just crossed a big 20 million active user mark but if it's chat GPT then it also goes out to like even a lot bigger number of people 流程是什么？你们还有代码审查这个概念吗？AI 代码审查、验证、滚动发布、分阶段验证发布，这些东西？因为 Codex 本身会到几百万人手里，我刚跨过两千万活跃用户这个大关；如果是 ChatGPT，还会到更大数量的人那里。

主持人 yes um but it's it's surprisingly like a similar process whether you ship on codeex or chat even though Chibd goes out to like a billion a billion you know active users and growing you can ship a PR um you know you can make a change and you know get it shipped like the next day or like even the same day um and it just goes out to a billion users and it's fine we just really instill a sense of ownership and care so you're like people are very empowered to make changes even large changes the general thing that is being asked is like sort of like evidence that 是的。但无论你是发到 Codex 还是 ChatGPT，流程出奇地相似。尽管 ChatGPT 会到大约十亿活跃用户那里，而且还在增长。你可以提交一个 PR，做一个改动，第二天甚至当天就发出去，然后它就到十亿用户那里，而且没问题。我们真正灌输的是主人翁意识和在乎，所以人们非常有权去做改动，哪怕是大改动。一般会被问到的是某种证据：

主持人 it's going to be wellreceived D evidence that is like a worthy addition uh evidence that you know it's like it is worth maintaining over time but also like the cost of maintenance is like just really as you know gotten done significantly as well. So we we we think about these things slightly differently than you know say like two years ago or 3 years ago. The other thing as well is like you know we automate as much as possible. So like a lot of like the process of like code review and deploys and you know catching regressions is like you know all of that is like pretty much automated. Uh and so like you know you get to just focus on just really the idea and you know how it's going to help our users and you care about you know the coherence of it all and so like the overall power of the agent um and making things better and like we don't we have a long long list of things that you know we sort of like aspire to do and haven't gotten to yet. And then there's like the sort of like the northstar direction um which is a delightful simple to use 它会受到欢迎；证据表明这是值得的增加；证据表明它值得长期维护。但维护成本也已经显著下降了。所以我们想这些事的方式和两三年前已经略有不同。另一件事是：我们尽可能自动化。所以代码审查、部署、捕捉回归，这些流程很大一部分都已经相当自动化了。所以你就能把注意力真正放在想法上，放在它会怎样帮助用户，你在乎整体的连贯性，也就是 agent 的整体能力，以及把事情做得更好。我们有一长串渴望去做、但还没做到的事。然后还有北极星方向：一个令人愉悦、简单好用的

主持人 personal AGI that you know knows everything about you like you know that it needs to know has access to the right resources can take like you know sometimes risky actions on your behalf but then you know you get like the push notification and then you know you can verify that and it's like a thing that you know you deeply understand as a user but also it knows about your schedule it knows about your goals it can be proactive and it should be like extremely natural it should be something that you can control through like natural language, voice, you know, like maybe it should understand, you know,your emotions like if it has like a camera feed, it should be the most natural thing on earth. It's like it should not be like a thing with 10, you know, different buttons and configurations. It's like AGI should be simple to use. Now, 个人 AGI。它知道关于你、它需要知道的一切，能访问正确的资源，有时能代表你采取有风险的行动，但你会收到推送通知，然后你可以去核实。作为用户你深刻理解这个东西，同时它也知道你的日程、你的目标，它能主动，而且应该极其自然。它应该是你能用自然语言、用语音来控制的东西，也许它还应该理解你的情绪；如果它有摄像头画面，它应该是世上最自然的东西。它不该是一个有十个不同按钮和配置的东西。AGI 应该简单好用。现在，

Tibo Sottiaux you kind of mentioned just briefly the review the code review, but I wanted to go back to it. you you worked at Google on a on a product used by you know like hundreds of millions which is Google maps and Google is very well known for their culture of very strict code reviews. They have I think two layers of code reviews. There's a language correctness review and they they've taken I think they've really perfected it across the industry for for a long time and they do believe that it it works and they they use it. How do you think that part is changing specifically the human review? Because for a very long time until maybe a year or two ago, I would have said you code review has all these benefits that knowledge sharing the second pair of eyes removing the bus factor because now someone else understands and when that person is out that they can jump in conversations are 你刚才只是简短提到了审查、代码审查，但我想回去谈这个。你在 Google 做过一个有几亿人在用的产品，也就是 Google Maps。Google 以非常严格的代码审查文化著称。我记得他们有两层代码审查。有一层是语言正确性审查。我觉得他们在行业里把这件事打磨了很久，他们也相信这套有效，并且在用。你觉得这部分、尤其是人的审查，正在怎么变？因为很长一段时间，直到大概一两年前，我都会说代码审查有这些好处：知识分享、第二双眼睛、去掉巴士因子，因为现在另有人懂了，那个人不在时别人能接上。对话会

Tibo Sottiaux happening about architecture not just the code but now there's you know there's a lot more code uh and what was the value of code review? What in what cases? And so on your team, because you guys are so ahead of this, where do you see humans still being or developers being involved in the review stage valuable? And and where is it fine? Did you find it find it fine to uh hand it off to an agent? 发生在架构上，不只是代码。但现在代码多了很多。代码审查的价值曾经是什么？在哪些情况下？所以在你们团队，因为你们在这件事上走得很前，你在哪些地方仍然觉得人、或者说开发者参与审查阶段是有价值的？哪些地方可以放手？你们有没有发现，把审查交给 agent 也可以？

主持人 Yeah, the the the role of code review is changing. One one of the early projects that I did on Codex was like working with research on developing a code review model um that was going to be to a level where it can spot mistakes in logic and reasoning. uh to a degree where it would require humans like you know multiple like potentially multiple hours to capture the same level of mistake because it requires like really digging like you know three four levels deep into like the dependencies and like you know understand that maybe the documentation actually was wrong and like the implementation of like this third party dependencies like different from what you expected and so therefore your invarants are not upheld um and these things it's just like you know unless you're an expert in that library you wouldn't know uh and therefore you have a bug and so we developed like these uh code review models and you know we we we released them and now they're 对，代码审查的角色正在改变。我在 Codex 上做的早期项目之一，就是和研究方向一起开发代码审查模型，要达到能发现逻辑和推理错误的程度。那种程度需要人、可能要花好几个小时才能抓到同等程度的错误，因为它需要真的往下挖，挖到依赖的三四层深，还要理解也许文档其实是错的，这个第三方依赖的实现和你预期的不一样，因此你的不变量没有被守住。这类事情除非你是那个库的专家，否则你不会知道，于是你就有了一个缺陷。所以我们开发了这些代码审查模型，也发布了。现在它们

主持人 like the same level of like capability and like ability to spot these mistakes by doing like you know deep verification are like just part of the mainline models like when we benchmark them it's like they're like super human in code review and this is not just true for correctness. This is also true for security for example where you're capable of like reasoning across like you know very very complex things and then you know coming up with like hey you know you have a critical security vulnerability here which is now mandatory across like all of OpenAI pull requests like we block pull requests from merging if you know we flag them with like a a security issue and this is like all automatic and the role of code review now is like I think it was always about correctness it was always about you know ensuring that things worked but it was also sort of like a little ritual for information exchange and you know bringing people on the same page and like you know encouraging like a discussion which ideally would have happened before but sometimes it just only happens like around the code because once it merged it just actually 已经具备同等水平的能力，能通过深度验证发现这些错误，并且已经成为主线模型的一部分。我们做基准测试时，它们在代码审查上已经超人类。这不只对正确性成立，对安全也成立。例如你能跨过非常非常复杂的东西去推理，然后得出：嘿，这里有一个严重安全漏洞。现在这在 OpenAI 的所有 pull request 上是强制的：如果我们标出安全问题，就会阻止 pull request 合并。这全部是自动的。现在代码审查的角色，我觉得它一直关乎正确性，一直关乎确保东西能用，但它也差不多是一种信息交换的小仪式，把人拉到同一页上，鼓励讨论。理想情况下讨论本该更早发生，但有时它只发生在代码周围，因为一旦合并，它就真的

主持人 runs in production it's doing stuff 在生产环境里跑、在做事。

Tibo Sottiaux and then you have to maintain it so there's like this social aspect to it as well it's I think all of it is changing like the correctness the cyber the the security is like I think that will be automated really what we see and I see is there's a sort of um really discussion around the intent that takes place around the poll request. It's like what are you even trying to do? Um and is that a right thing to attempt to do? I think you can have that discussion outside of the poll request. It doesn't have to be around code. 然后你还得维护它，所以这里也有社会层面。我觉得这一切都在变。正确性、安全，我认为那些会被自动化。我们真正看到的、我也看到的是：围绕 pull request 发生的，是关于意图的讨论。你到底想做什么？这件事该不该尝试去做？我觉得那场讨论可以发生在 pull request 之外。它不必围着代码。

主持人 So, so maybe this helps crystallize the you know like where a discussion needs to happen versus versus where we we did it because maybe we didn't have the type of tooling that we have right now. Yeah, I think this is going to change and it was like a forcing function because you know you have to have that discussion where like it's good to have that discussion before you merge it and it becomes production code but I I think there are other ways to have these discussions and you know design things together and make sure that the intent is good uh and then the code doesn't matter as much 所以这也许能帮我们把这件事说清楚：哪些地方讨论必须发生，哪些地方我们以前那样做，只是因为当时没有现在这类工具。对，我觉得这会改变。它曾经像一种强制函数，因为你必须在合并、变成生产代码之前进行那场讨论，那场讨论是好的。但我认为还有其他方式来进行这些讨论，一起设计东西，并确保意图是对的，然后代码就不那么重要了。

Tibo Sottiaux and it's interesting because when I think back of all my code reviews like of course I have like memories where like it was great we we had a good discussion or I learned something really interesting but a bunch of times honestly It was such a pain in the ass. Like I I was trying to get my stuff. You're paying. Hey, could you remove my code? And like, no, right now I'm busy. No, I really need this to unblock me. And then you context switch. And then I feel it's always been like good and bad, right? So I I feel whatever we do there will be always upsides and and downside, but there now they're just moving. So I guess one upside is as an engineer you might have to not give your attention to just kind of basic stuff that doesn't need your input per se. Yes, it saves time and I think progressively what we're going to see is also like you have an agreement on you know the box and the 有意思的是，回想我所有的代码审查，当然有些记忆很好，我们有过很好的讨论，或者我学到非常有意思的东西。但老实说，很多次真的很痛苦。我想把我的东西弄进去。你在求：嘿，能不能帮我看一下代码？然后：不行，我现在忙。不行，我真的需要这个来给我解阻塞。然后你切换上下文。然后我觉得这事一直有好有坏，对吧？所以我觉得无论我们怎么做，总会有好处和坏处，只是现在它们在移动。所以一个好处大概是：作为工程师，你也许不必再把注意力给那些本身并不需要你介入的基础事情。对，这能省时间。而且我认为我们会逐渐看到的是：你们对这个盒子、对

Tibo Sottiaux overall contract of what it's supposed to do and then you know what is inside the box as long as you have like strict guarantees in terms of resource utilization, data access, um security, these kinds of things. It's like what happens inside the box is you know it could be literally anything. it's like don't really need to care and like really what you need to agree on is like what does the box actually do and what are the invariants that must be satisfied and I think that is then worthy you know having like a really good conversation on you know maybe assisted by by your favorite uh agent but then once you have that and you have that understanding it's just like changing anything within the box is like you know doesn't require for discussion and it's like you know just really preserves your attention 它应该做什么的整体契约达成一致；然后盒子里面是什么，只要你在资源占用、数据访问、安全这类事情上有严格保证，盒子里面发生的事 literally 可以是任何东西。你其实不必在乎。你真正需要达成一致的是：这个盒子实际做什么，哪些不变量必须被满足。我觉得那才值得好好谈一场，也许由你最喜欢的 agent 来协助。但一旦你有了那个、有了那种理解，改盒子里面的任何东西就不需要再讨论，那才真正保住你的注意力。

主持人 the cost of maintenance has gone down you know maintenance is always such a hot topic whenever we build thing uh inside of all these companies like Google, Uber, even startups like building was the fun part but then maintenance was the painful and that's when we learned like okay it was not we're building it etc inside of codeex and open AAI what do you see maintenance becoming cheaper changing in terms of instead of what you're building what the ambition is the I guess custom tooling those kind of things 维护成本已经下降了。维护一直是热门话题：每当我们在 Google、Uber 甚至创业公司里做东西，开发是有趣的部分，维护才是痛苦的。那就是我们学到：好，问题不在于我们正在建造它，等等。在 Codex 和 OpenAI 内部，你看到维护变便宜、正在如何改变？这会怎样影响你们建造的东西、野心、以及定制工具这类事情？

Tibo Sottiaux maintenance is really like sort of like a tax that you pay over time just to keep things running and it's It's it's always been necessary. It will continue to be necessary, but where I think it changes is like a lot of it is just going to be automated. So, you know, it's like okay, you have you have this third party dependencies like you need to upgrade the version number. It's like a like you can fully automate this you know um if you have good change log and you know and the code is well documented and like you know and and the model can just like reason through it like you know you can just like blast through your codebase do it uh in a couple of hours and you know previously you would have like punted on 维护真的就像一种税，你要持续缴纳，才能让系统继续跑。它一直必要，也会继续必要。但我认为正在改变的地方是：其中很大一部分马上就会被自动化。所以，好，你有这些第三方依赖，需要升级版本号。【笑声】这件事你可以完全自动化。如果你有好的变更日志，代码文档也写得好，模型又能把它推理清楚，你就可以扫过整个代码库，几个小时做完。而以前你会把这件事

Tibo Sottiaux it because it's not the most fun thing to do but it's actually really important for your business. So it's like really important for your project, you know, especially for security vulnerabilities. You want to stay up to date, right? You want to apply, you know, all these patches. Um I think that's just going to be fully automated. So a large part of like maintenance, it just kind of comes for free, right? And then um I think it's awesome to also think about before like you know when you wanted to just completely react re you have to do like a new architecture because you're trying to make space for like a new you know different kind of trade-offs or you have a new understanding of like the workload or you're trying to fit a new feature and like suddenly you realize like your current system is just very limiting and you need to completely rearchitecture it that was like a really really costly endeavor right so you know like sometimes like multiple years and I think this is also like super super accelerated now. So you like the cost of mistakes uh you know I would say like 往后推，因为那不是最有趣的事，但它对你的业务其实非常重要。对你的项目非常重要，尤其是安全漏洞。你想保持更新，对吧？你想打上所有这些补丁。我认为那会完全自动化。所以维护的很大一部分差不多是白送的，对吧？然后我觉得也很棒的是再想以前：当你想彻底重做、你必须做一次新架构，因为你想为一种新的、不同的权衡腾出空间，或者你对负载有了新的理解，或者你想塞进一个新功能，突然发现当前系统非常受限，必须彻底重新架构。那曾经是代价非常非常高的事情，对吧？有时要好几年。我认为这件事现在也被超级超级加速了。所以我会说，犯错的成本

Tibo Sottiaux you know is going down but then at the same time the good old rules I would say of software engineer of like you know having good abstractions like really help like you know is going back to this like having the box with invariance like you know if you sort of like draw the right shape you're going to be able to change things much more quickly within the box and like not affect the rest of the of the services or the rest of your infrastructure and I think it's important it's important to design for very quick iteration and I remember when I talked with Peter Shamberger that was before he joined OpenAI but about open claw and how he thinks about it like you know he told me that he doesn't read the code but he kept thinking about like I could see that he's holding the architecture in his head and he was telling me how he rearchitects a lot and he thinks about how to make it modular how to allow 100 contributors to each build their thing without stepping on each other's toes. So I'm hearing what you're saying that 正在下降。但与此同时，软件工程那些老规矩，比如拥有好的抽象，仍然非常有帮助。这又回到那个带着不变量的盒子：如果你把形状画对，就能在盒子里面更快地改东西，并且不影响其余服务或其余基础设施。我认为为非常快的迭代来设计很重要。我记得我和Peter Steinberger 谈过，那是他加入 OpenAI 之前，谈的是 OpenClaw，以及他怎么看待它。他告诉我他不读代码，但他一直在想。我能看出他把架构托在脑子里。他跟我讲他会大量重新架构，想怎样做成模块化，怎样让 100 个贡献者各自做自己的东西、又互不踩脚。所以我听到你说的是：

Tibo Sottiaux this this care this this planning this structuring has become maybe just a lot more important to like which which was which which was something back in the day you know it was like the architect or the staff engineer or experienced folks were doing this thing and other engineers around them were kind of building this you know smaller parts but it sounds like now all engineers need to be aware of when you're building your software right and plan for it 这种在乎、这种规划、这种结构，也许已经重要得多了。以前这是架构师、staff engineer 或有经验的人在做的事，周围其他工程师差不多在建那些更小的部分。但听起来现在所有工程师在建造软件时都需要意识到这一点，并为此做规划，对吧？

主持人 yeah and and the GP models are getting better and better at this as well of like you know thinking about long-term maintenance and like good architecture and like this is like a natural sort of like next step right. It's like not just about code quality in the sense of like oh is this code clean within this file but like you know is this like is the architecture actually correct to reduce maintenance burden over time and like you know make space for like future u product or feature extensions or changes and just really this act of like you know engineering over time that's uh kind of like something that models are starting to become capable of like thinking about very well I think it's 对。而且 GPT 模型在这件事上也越来越好，去想长期维护和好的架构。这差不多是自然的下一步，对吧？不只是代码质量那种「这个文件里的代码干不干净」，而是架构是否真正正确，从而随时间降低维护负担，并为未来的产品或功能扩展、改动腾出空间。这种跨越时间的工程行为，模型开始能想得非常好。我觉得这

主持人 just kind of fascinating to understand that the software that we're building is just going through the life cycle much faster. Right? Like you know before you had you know you were you were scaling you were starting it you know maybe as like a small team of you know yourself maybe a couple of engineers and then you would add engineers like slowly and then you know maybe after a year you know it's like if it's very very successful you would have 50 engineers on it or like a 100 engineers on it. You would have time to see it coming. you would have time to see like you know the humans on board and you sort of like you know you can think about the documentation all of that stuff but now it's just sort of like that explosion of like you know suddenly you have like a 100 agents contributing to this thing is like you know that can happen like you know in a weekend and so you know you're just going through it at you know major speed compared to before 真的很奇妙：要理解我们正在建造的软件，生命周期转得快得多了。对吧？以前你在扩展，也许从你自己、也许几个工程师的小团队起步，然后慢慢加人；也许一年后，如果非常非常成功，会有 50 个或 100 个工程师在上面。你会有时间看见它到来。你会有时间看见人加入，你也可以去想文档这类事情。但现在差不多是那种爆发：突然有 100 个 agent 在给这个东西做贡献，这可以发生在一个周末里。所以相比以前，你只是以快得多的速度把周期走完。

Tibo Sottiaux okay but how do you and and the folks at OpenAI like deal with this like does it not mess with your mind like you know what I mean in the sense of like you you you've been in this business for quite some time now like like decades or or well over and there was a pace that we kind of got used to and obviously is it's now a lot faster but how do you get your head around the fact that a it's faster b the stuff that you've been doing a year ago right now you're not doing because now the model is is good at it and you know like how do you kind of reconcile that because there I'm sure there's stuff that you've been really good at uh related to software that now you can hand off to the agent do you not 好，但你和 OpenAI 的这些人怎么面对这个？这不会把脑子搞乱吗？我的意思是，你已经在这行很久了，几十年，或者说远不止。我们曾经习惯某种节奏，现在显然快得多。但你怎么接受这个事实：第一，它更快了；第二，你一年前还在做的事，现在你不做了，因为模型已经擅长了。你怎么调和这个？我相信有些你曾经非常擅长的、和软件相关的事，现在可以交给 agent。你难道不

Tibo Sottiaux get a little bit of sting you know we talked about it's stinging for your features to be implemented open source. But it can also sting that I've been really good at like I don't know refactoring or or or right now it might be architecture but maybe the model will be really good at that and now I'm like uh okay damn like I'm glad but also like uh it would have been nice for me to do that. 会有一点刺痛吗？我们谈过，你的功能在开源里被别人先做出来会刺痛。但也可能刺痛的是：我曾经非常擅长重构，或者现在也许是架构，但也许模型会非常擅长那个，然后我就会：好吧该死，我高兴，但要是能由我来做也挺好。

主持人 Yeah, I think there's like a craft aspect to it. Um which occasionally I still you know pull up an editor and like write some code. Um, and it's just like it it it it feels nice and and and it's sort of like I have fond memories of like late nights sitting in uh in Vim and you know, just like 对，我觉得这里有手艺的一面。偶尔我仍然会打开编辑器，写一些代码。那感觉很好。我也有美好的记忆：深夜坐在 Vim 里【笑声】，就是

Tibo Sottiaux cranking it out, 往外敲，

主持人 you know, drinking um Coke Zero and uh yeah, just not having to think about anything else other than like the problem in front of me. But really, I think it's um it's all about being in the flow and and solving problems. And what I find is like you know folks here and also like everyone I talk to is just like adapting very quickly and I think if you if you have a mindset where it's all about code is a tool to solve problems and you can solve so many more problems. It's like before you wanted to benchmark something and you weren't quite sure where you were going to net out. It's like you can just do it. it's it's going to take you like no more than 30 seconds, you know, to 喝着 Coke Zero，不用想眼前这个问题以外的任何事。但说真的，我觉得这全部关乎进入心流、解决问题。我发现这里的人，以及我交谈过的每个人，都适应得非常快。如果你的心态是：代码是解决问题的工具，那你就能解决多得多的问题。以前你想给某件事做基准测试，还不太确定结果会落在哪里。现在你可以直接做。最多 30 秒，你就能

主持人 launch something in the background and, you know, get proper numbers and be able to do like a better trade-off. It makes you it should make you a better engineer if you just really care about, you know, the outcome and the system working well. And so what it allows us to do at OpenAI, it allows us to run, you know, our inference much more efficiently. It allows us to, you know, get like much more like effective compute and, you know, deploy that to the world. And so like everyone's just like very focused on that and solving important problems at the speed that was not possible before. And like I I haven't yet, you know, encountered someone who's like, "Oh, that's not that's not good. Um that's not fun." 在后台启动某件事，拿到像样的数字，从而做出更好的权衡。如果你真正在乎结果和系统运转良好，这应该让你成为更好的工程师。所以在 OpenAI，这让我们能更高效地跑推理，让我们拿到多得多的有效算力，并把它部署到世界上。于是每个人都非常聚焦于此，去解决重要问题以以前不可能的速度。而且我还没遇到过有人说：「哦，那不好。那不好玩。」

Tibo Sottiaux Do I understand correctly that it sounds like if you have ambitious problems, if you have way more problems than what you can solve today or tomorrow or the next week, sounds like this is not really a problem because when you know you get more efficient somewhere, you keep going. Which which is a lot of startups, right? like startups are always way more ambitious than than what they're able to do. 我理解得对吗？听起来如果你有雄心勃勃的问题，如果你的问题远远超过今天、明天或下周能解决的，这就不算真问题，因为你在某处更高效了，就会继续往前走。很多创业公司就是这样，对吧？创业公司的野心总是远远超过他们能做的。

主持人 I don't we're not we're not out of problems for sure, right? So, and and I don't think we will be for a while. Um we have a long long road ahead of us in terms of like mathematical breakthroughs, scientific breakthroughs, you know, making the world a better place like just really building for humans and solving the most important problems that everyone is is facing and just doing it in a a deeply human way. that's that's what we're here for. Also, just going back to coding and like you know these late nights, it's like I think there's like it's also like maybe like a glamorous version of it. Just like I also had very a lot of late nights where I was trying to refactor something. Um and you know just like I would be like three hours deep into the refactor and then realize like actually this is a dead end. Uh and I must restart from scratch and it was like very frustrating. Um, and so it's like there was like there are like these very fun times, but there's also the time where it's like it doesn't compile 我们肯定还没把问题用完，对吧？而且我认为一时半会儿也不会。在数学突破、科学突破、让世界变得更好这方面，我们前面还有很长很长的路：真正为人类而建造，解决每个人都在面对的最重要问题，并且以一种深深人性的方式去做。我们在这里就是为了这个。另外，回到写代码，以及那些深夜，我觉得那也有一种也许被美化过的版本。我也有过很多深夜，想重构某样东西。我会在重构里陷进去三个小时，然后发现这其实是死路，必须从头再来，非常沮丧。所以有那些非常有趣的时刻，也有编译不过去的时刻，

主持人 and you're just like, why is this not compiling yet? Like 你就在想：为什么这还编译不过？

Tibo Sottiaux I'm sure you had the time where you you go you go later, it's now super late, you need to go to bed cuz you need to get some sleep and then you can't really sleep and you have this thing where like you you have some some task that is halfway and it upsets you sometimes. I remember dreaming about the code as well. And I guess one thing I don't really have these days when I'm working on my uh software for my business is I don't really have something that is halfway cuz I can just tell it do this and then I can leave it at a state where it's kind of like you know done either finished it's either working or it's I have proof that it's failed. But it's interesting because you know everything's sped up right. 我相信你有过那种时候：你越搞越晚，已经非常晚了，你得去睡觉、得睡一会儿，然后又睡不着，因为有个任务做到一半，这会让你心烦。有时我还梦见代码。我想这些天我给自己的生意做软件时，有一件事我其实不再有了：我不再有做到一半的东西，因为我可以直接告诉它做这个，然后把它留在一种差不多完成的状态：要么做完了能用，要么我有证据证明它失败了。但有意思的是，一切都加速了，对吧？

主持人 Yeah. I may maybe like I I do have like what what a lot of people do and I do myself is like you know I have like sometimes like bigger questions that I'm asking myself like and I you know from conversations I've had during the day or like I haven't yet you know just like had the time to just look into it and so I you know I will send off codecs to just like look at it overnight and then I'm very excited to then wake up and look at the results and so you know it's always like an exciting morning. Well, I feel there's an art to doing longunning tasks. And of course, you can use the slashgoal which will go and and and run. You know, that's also something that was recently added like a few months ago, right? The /go goal command to codeex. 对。也许我确实有很多人也在做、我自己也在做的事：有时我会问自己一些更大的问题，来自白天的对话，或者我还没来得及去查。于是我会把 Codex 派出去，让它连夜去看，然后我很兴奋地醒来看结果。所以那总是一个令人兴奋的早晨。我觉得做长时间运行的任务是有门道的。当然你可以用 slash goal，它会去跑。那也是几个月前刚加上的，对吧？Codex 的 /goal 命令。

Tibo Sottiaux Yeah. And back to, you know, maybe like the harness is a crutch, right? Is uh uh slash goal was like necessary to allow like no to keep the the model like on track on like a singular goal for a very long uh period of time. And it's like it allows the model to literally run for days or or weeks if it's like a really hard problem. But with the new generation of models like what we're seeing is like you know you don't need SL goal anymore. You don't need a harness around it. You can just tell the model like you know 对。再说回，也许harness 就是一根拐杖，对吧？slash goal 曾经是必要的，好让模型在很长一段时间里盯住单一目标不跑偏。它能让模型真正跑上几天或几周，如果那是很难的问题。但有了新一代模型，我们看到的是：你已经不【清嗓】再需要 /goal 了。你不需要在它外面再套一层 harness。你可以直接告诉模型：

Tibo Sottiaux hey go and work for a week and you know it will actually do it. 嘿，去工作一个星期，它真的会去做。

主持人 Speaking of hard problems and the fact that you're not out of them. One of the interesting things that you shipped from the outside it I would say it it was you know as an engineer it was moderately interesting uh is the what you call the merge which is codeex appeared inside of chat GPC and the reason I say that as engineers it's kind of moderately interesting because we've been using codeex like yeah it's there you can now open it in the chat GPT app great like I just went there and I just immediately went to Codex cuz I don't I don't really use Chad GBT in the app per se but I talk with uh folks at Open AAI and people in your team and you know they were telling me like there was a lot of preparation going on a lot of engineering challenges. Can you give a sense of how big this project was, what you needed to do and why was it difficult to pull off and and how you know how did codeex and other tools help you get it done in in ways that would 说到难题，以及你们还没把问题用完。你们交付过一件从外面看、作为工程师我会说只是中等有意思的事，就是你们所说的合并：Codex 出现在 ChatGPT 里面。我之所以说作为工程师这只是中等有意思，是因为我们一直在用 Codex，对，它在那儿，现在可以在 ChatGPT 应用里打开，很好。我只是进去，然后马上就去了 Codex，因为我本身不太在应用里用 ChatGPT。但我和 OpenAI 的人、和你团队里的人聊过，他们告诉我有大量准备工作，大量工程挑战。你能不能让人感觉一下这个项目有多大，你们需要做什么，为什么很难做成，以及 Codex 和其他工具怎样以以前会很难的方式帮你们做成。

主持人 have been hard before cuz since you've launched the merge the the numbers that you keep sharing of how many people use codeex it's like it's going up way faster than before. So I assume I assume there's a big scale uh problem you've solved here. 因为自从你们发布这次合并，你不断分享的 Codex 使用人数，上升速度比以前快得多。所以我猜这里你们解决了一个很大的规模问题。

Tibo Sottiaux A lot of things were the challenging with the merge is first of all completely different stacks. Chbt is like fully uh managed cloud-based like you know you run everything on our uh on our systems. We we store things like traditional way of like building things. Um built for scale, built for for efficiency. Codex fully local. And so the merge is just really like how do you get the same uh the same benefits and the same capabilities from this local coding agent and then build a product 合并有很多挑战。首先是完全不同的技术栈。ChatGPT 是完全托管的、基于云的，所有东西跑在我们的系统上。我们按传统方式存储、建造东西。为规模而建，为效率而建。Codex 则完全本地。所以合并真正要解决的是：你怎样从这个本地编程 agent 拿到同样的好处和同样的能力，然后围绕它做成一个产品，

Tibo Sottiaux around it and build it in a way where it can benefit like a much much broader pool uh of people which is which is also why you know all of us joined OpenAI is like to benefit like this very very broad population across the world and so it was like a very exciting journey of like figuring out like how do we build a cloud version of this that in essence is capable of like very very much the same things um but is also built in a way where you know we can serve it to like tens and hundred millions of hundreds of millions of users in a way that is still like you know efficient so that we can include it all the way into the plus plan work is essentially like running uh the full codeex harness in uh a cloud like together with like a a cloud computer it's It's a very powerful machine actually like people have sort of picked up on it and showed like you know what you can do like you know if if 并且做成能让宽得多的人群受益的样子。这也是我们所有人加入 OpenAI 的原因：让全世界非常非常广的人群受益。所以那是一段非常激动人心的旅程：搞清楚怎样做出这个东西的云版本，本质上能做非常非常相同的事，但同时也用一种方式来建造，让我们能以仍然高效的方式服务几千万、几亿用户，从而能一路做到把它放进 Plus 套餐。work 本质上就是在云上跑完整的 Codex harness，再配上一台云电脑。那其实是一台非常强的机器。人们已经有点意识到了，并展示了你能做什么：如果你

Tibo Sottiaux you are creative with the prompt is that you know you can get um you can get chatbt to like you know train another model in there you know 对 prompt 有创意，你甚至可以让 ChatGPT 在里面训练另一个模型。

主持人 wow 哇【笑声】

Tibo Sottiaux there are some pretty wild things uh you know you can get it to install blender and you know do like 3D modeling it's like very permissive it has like internet access it's like a powerful machine and then codex just works on it and this is like what we ship through cas work. a lot of system uh challenges. 有一些相当疯狂的事：你可以让它安装 Blender，做三维建模。它非常宽松，有互联网访问，是一台强力机器，然后 Codex 就在上面工作。这就是我们通过 ChatGPT work 交付的东西。有大量系统层面的挑战。

Tibo Sottiaux We the team did it very quickly. Obviously like Codex helped you know to make it more efficient look at and and build a lot of the infrastructure and then you know help resolve a lot of the little differences as well that you know had been occurring between between Codex and CHBT like merging plugins architecture um you know merging library and like so like really really working towards like unified system which is really the goal is like you shouldn't feel like you know you can do something in Codex that you can't do in TAB or vice versa like what we're trying to build is like one unified product that gives you access uh to the same intelligence but in the way that you want to use it. And so it was very fun as well because Codex throughout the whole journey also acted as a journalist to sort of like document all the steps and the debates and the discussions that the teams were having. And it was very animated debate you know of how we should do it and how we should name the 团队做得非常快。显然 Codex 帮了忙，让它更高效，去查看并建造大量基础设施，也帮着解决 Codex 和 ChatGPT 之间已经出现的许多小差异，比如合并插件架构、合并库，真正朝着统一系统努力。真正的目标是：你不应该觉得有些事能在 Codex 里做、却不能在 ChatGPT 里做，反过来也不该。我们想建造的是一个统一产品，让你以自己想用的方式，访问同一种智能。所以这也非常有趣，因为在整段旅程里，Codex 还充当了记者，记录各个步骤，以及团队之间的辩论和讨论。辩论非常热烈：该怎么做、该怎么命名这个

Tibo Sottiaux thing and you know when to introduce it in what way and like what to merge into what. there were like many different permutations considered and so there's like a very fun journalistic element to it where we like we have a full recounting that Codex did over time uh and uh yeah it's just it's kind of become known as well as like the toggle arc of uh OpenAI where you know we introduced like the work toggle um which there was also a lot of debate around of like you know whether this was like the right thing and then you know just like we kind of grew to to just really like it um but over time we're going to merge things further. So it's like we're really headed into this direction of like full unification and you know we kind of view this as like um a temporary state uh where you know you have like you have better stronger capabilities when you're in work mode but over time we're bringing this you know all all the way to like you know everyone that uses 东西、什么时候以什么方式引入、把什么并进什么。考虑过许多不同的排列组合。所以这里有一种很好玩的新闻记录成分：我们有 Codex 随时间写下的完整记述。对，这后来也有点被称为 OpenAI 的开关弧线：我们引入了 work 开关，围绕这是不是正确的事也有很多辩论，后来我们渐渐真的喜欢上它。但随着时间我们会进一步合并。所以我们真正朝向的是完全统一。我们差不多把现在看成一种暂时状态：你在 work 模式里会有更好、更强的能力，但随着时间我们会把这些一路带到每一个使用

Tibo Sottiaux CHP. And how do you personally use codeex? Like what's your what's what's your working setup in terms of agents, in terms of task, in terms of what you what what you manage uh with it. And related to this, I asked Peter Stainberger what I should ask about you and he said like you I I you need to ask him how do you deal with the fact that you're involved in all these projects? Uh your calendar is like Tetris, but usually you show up pretty cheerful. ChatGPT 的人那里。你个人怎么用 Codex？就 agent、任务、你用它管理什么而言，你的工作环境是怎样的？和这相关，我问过 Peter Steinberger 该问你什么，他说：你得问他，你怎么面对自己卷入这么多项目这件事？你的日历像俄罗斯方块，但你出现时通常还挺开心。

主持人 My calendar is fine. Um um and it's just I am capable of doing so many more things nowadays because I have the technology like CEX and I actually shifted a lot of like my my my work on uh on mobile using tab work where whenever I have something that I want to take note of I just like fire that off. I use dictation a lot. Um, whenever I have a question, instead of like writing it down to look into later or delegating to someone, I just like fire it off in charge of your work and I get like a report. It has like a whole bunch of like custom skills and um custom uh instructions where it's now like very tailored to like you know produce the kinds of reports and slide decks and uh code explorations you know in the 我的日历没问题。只是我现在能做的事多得多，因为我有像 Codex 这样的技术。我其实把大量工作挪到了手机上，用 ChatGPT work：每当我有想记下的东西，我就直接发出去。我大量使用口述。每当我有问题，我不再写下来以后再查，或委派给别人，而是直接在 ChatGPT work 里发出去，然后拿到一份报告。它有一堆自定义 skills 和自定义指令，现在已经非常贴合我，能产出我能有效消化的那种报告、幻灯片和代码探索，用的是

主持人 style that I can consume effectively. And so every time I'm like between meetings or like you know you'll kind of like see me like you know I was just like dictating to my phone. As I said before it's just like we do a lot of work in public channels. We have like a lot in Slack. We have a lot in in in notion and Google Docs as well. And so there's pretty much like there's no question really that I feel I cannot ask that you know Codex will be able to sort of like do at least a first pass of thinking through whether it is like public sentiment on a feature um looking at production logs for you know how much usage we have on a certain thing making a list of things that we should deprecate because they're not getting traction uh understanding what a certain team is up to. It's like any question I have I can get an answer to like you know within 30 minutes. And so that's how I use it. I use it for everything. It's like my personal uh agent in in like all the ways. And then oftentimes 那种风格。所以每次我在会议之间，你都会看见我在对着手机口述。像我之前说的，我们大量工作放在公开频道。Slack 里很多，Notion 和 Google Docs 里也很多。所以几乎没有什么问题我觉得不能问，Codex 至少能做第一轮思考：无论是某个功能的公开舆情，看生产日志了解某样东西有多少用量，列出该废弃的、没有起势的东西，理解某个团队在忙什么。任何我有的问题，我都能在大约 30 分钟内拿到回答。所以我就是这样用它的。我用它做所有事。它在所有意义上都像我的个人 agent。然后常常在

主持人 on on weekends as well, I do some like code explorations or like I build some prototypes and I have fun like sort of like imagining the future of the product in some ways. And I do that with others on on on on the teams. It's not always the same team. And it's just like in one day I can build things that I sort of like I had it in my system, right? It's like it's like I woke up one day I was just like we should explore what it means to build this and then I can just sort of express all of that and get like something in front of people in a day so that they can think through it and criticize it and hopefully get inspired by it. It's like by no means you know we need to ship it but it's more like okay I flush it out of my system and then you know I go on and like you know do other things. So it's just like so I know it's such a magical time and it's like so empowering. And as closing, what would your advice be for a software engineer/ AAI engineer, someone who builds 周末，我也会做一些代码探索，或做一些原型，好玩地想象产品未来的某些样子。我和其他团队的人一起做，并不总是同一个团队。一天之内我就能把已经在我系统里的东西做出来，对吧？就像有一天我醒来就想：我们应该探索建造这个意味着什么，然后我就能把那一切表达出来，一天之内把东西放到人们面前，让他们想清楚、批评它，也希望被它启发。绝不是说我们必须把它发出去，更多是：好，我把它从系统里冲出来，然后继续做其他事。所以我知道这是如此神奇的时刻，也如此赋能。作为结尾，你对软件工程师、AI 工程师，也就是建造

主持人 software who would want to get the skill set and the experience to have the opportunity to work at a place like the Codeex team, like OpenAI or like an AI startup. So like you know just become this really great builder with with these tools because the question that comes up is often like should I start with the theory? How important are the basics? Should I just get really good at using the tools? 软件、想获得技能和经验、从而有机会在 Codex 团队、OpenAI 或 AI 创业公司这类地方工作的人，会给什么建议？也就是怎样用这些工具成为真正出色的建造者。因为经常出现的问题是：我该从理论开始吗？基础有多重要？我该不该先把工具用得非常好？

Tibo Sottiaux Yeah. I think there are two things that are important is a deep deep curiosity for how things work um and an ability to like you know train yourself to understand things very quickly and so it's it's it is the case that things will continue to change but people that do extremely well at OpenAI are like you know people that just sort of like are able to like gro uh a system quickly and like you know also dive into like a new code base and sort of like know make sense of it. Um but obviously like all 对。我觉得有两件事很重要：对事物如何运作有深深的好奇心，以及训练自己非常快地理解事物的能力。事情会继续变，这是事实；但在 OpenAI 做得极好的人，是那种能很快抓住一个系统、也能扎进新代码库并把它弄明白的人。但显然现在所有这些

Tibo Sottiaux of that is helped with agents um nowadays, right? So just like there's so much information that you need to absorb and like you know being able to understand and reason through it and a lot of that is asking good questions really about you know how do things work and just like going into like the five W's which I think you know you can just kind of keep digging and digging and you know you're learning very very fast through that. The other thing is being in tune with the community or you know the people that you're trying to solve a problem for. It's like not everything is like solving a direct problem. Sometimes you're solving a problem that will be useful, you know, to like another group of people in the pursuit of like solving a problem for humans. But just being crisp about the taste or the needs or the requirements uh and being able to think clearly and like you know exercising through this clarity of thought feels really important to me. Like if you can't explain what you're trying to achieve, if you can't explain your intent, if you don't have a tie to a community, if you don't have the taste, it's it's um it's going to be much harder to do great work. 都有 agent 帮忙，对吧？你需要吸收的信息如此之多，要能理解并推理过去，其中很大一部分其实是问出好问题：事情是怎么运作的，进入那五个 W。我觉得你可以一直挖、一直挖，通过那样学得非常非常快。另一件事是跟社区合拍，或者说跟你想为之解决问题的那些人合拍。并不是每件事都是在直接解决问题。有时你解决的问题，会对另一群人有用，而那是在为人类解决问题的路上。但要把品味、需求或要求说清楚，能想清楚，并练习这种思想的清晰，这对我来说真的很重要。如果你解释不了自己想达成什么，解释不了意图，跟社区没有联结，没有品味，要做出出色的工作会难得多。

主持人 Awesome, TB. Well, thanks a bunch for this conversation. This was awesome. 太棒了，Tibo。非常感谢这次对话。这太棒了。

Tibo Sottiaux Thanks for having me. 谢谢邀请。

主持人 I've always wanted to get together with Tibo, and I'm glad that we finally made it happen. I appreciated how Tibo talked about not just the upsides of open source, but also the downsides. most notably how competitors can copy features you are just working on in the open right now and then ship it right before release and just how much this stings. Plus, you get a lot of low-quality contributions that you still need to somehow deal with. Another interesting one was Tibo was saying how the harness is always a step ahead of the model. From the inside, the Codex team see their job as building clutches for the model with the harness, the tools, and the setup instruction. And then the next version of the model will be trained to need fewer of these clutches. I'll be honest, as a dev, this sounds a little demotivating that the stuff I build in the next version of the model, it'll just know and we can get rid of it. Plus, I do suspect that it's not 我一直想和 Tibo 坐下来聊，很高兴我们终于做成了。我很欣赏 Tibo 不只谈开源的好处，也谈坏处。最显著的是：竞争对手可以复制你【音乐】此刻正在公开场合做的功能，然后赶在发布前交出去，以及这有【音乐】多刺痛。另外，你会收到大量低质量贡献，仍然得设法处理。另一个有意思的是，Tibo 说harness【音乐】总是比模型超前一步。从内部看，Codex 团队把自己的工作看成：用 harness、【音乐】工具和启动指令，给模型做拐杖。然后下一版模型会被训练成更少需要这些拐杖。老实说，作为开发者，这听起来有点打击积极性：我建造的东西，到了下一版模型，它【音乐】就会知道，我们就可以扔掉。另外我确实怀疑，这并不

主持人 just about building these clutches, but also building tools that models will use. And it's not like the next version of the model will reinvent an MCB protocol or scales or plugins. At least I hope not. I also enjoyed hearing what the merge merging chat GPC and codeex look like from the inside. It was merging a previously fully local coding agent, Codeex, into a managed cloud-based stack and doing it efficient enough so that it can be included in OpenAX $20 per month plan when $20 is not all that much in terms of compute purchase. It was pretty amusing to hear how CODC itself acted as a journalist of the whole project as it was present in all the Slack conversations and all the documents and so it could capture all the important debates and decisions. I'm not going to lie, this part felt a little bit of a big brother feel to it where the AI is always watching, but it could well become the new normal in startups in the future. I've not yet decided how I feel about this. And finally, I 只是在做这些拐杖，也是在做模型会使用的工具。下一版模型并不会去重新发明 MCP 协议、skills 或插件。至少我希望不会。我也喜欢听这次把 ChatGPT 和 Codex 合并从内部看起来是什么样。那是把一个原先完全本地的编程 agent，也就是 Codex，【音乐】并进一套托管的云技术栈，并且做得足够高效，以至于能放进 OpenAI 每月 20 美元的套餐。而 20 美元就算力购买来说并不算多。听起来挺有趣：Codex 自己充当了整个项目的记者【音乐】，因为它出现在所有 Slack 对话和所有文档里，所以能捕捉所有重要的辩论和决定。我不骗你，【音乐】这部分有一点老大哥的感觉，AI 一直在看。但这很可能成为创业公司未来的新常态。【音乐】我还没决定自己对此是什么感觉。最后，我

主持人 appreciated Tibo's advice for engineers to succeed. Be curious, understand symptoms quickly, and be in tune with the group you are building for. It's reassuring to hear from Tibo as well how much the fundamentals still matter. Do check out the show notes below for deep dives on how codecs, clock code, and cursor were built and other related topics. If you like what you heard, please hit a rating on a podcast player that you're using. It means a lot to me and to the show. Thanks, and I'll see you in the next 很欣赏 Tibo 给工程师如何成功的建议：保持好奇，【音乐】快速理解系统，并跟你为之建造的那群人合拍。听 Tibo 说基础【音乐】仍然如此重要，也让人安心。请查看下方节目注释，里面有关于 Codex、Claude Code 和 Cursor 如何被建造的深访，以及其他相关话题。如果你喜欢听到的内容，请在你正在用的播客播放器上【音乐】给个评分。这对我和这个节目意义很大。谢谢，我们下期见。