Context Engineering 与软件工厂：为什么不能把灯彻底关掉

主持人 So what is context engineering? It's kind of like deabstracting the abstractions that have been layered on top of rag memory, agentic history. At the end of the day, they're all different ways to pass tokens into a model. What is a smart zone and what is a dumb zone? The less context window you use, the better outcomes you'll get always. A new paradm that is spreading up is loop engineering. What do you think is bad about it? 主持人：什么是 context engineering？ Dex：它有点像把叠加在 RAG、memory、agentic history 之上的抽象重新拆开。归根结底，它们都是把 tokens 传进 model 的不同方式。主持人：什么是 smart zone 和 dumb zone？ Dex：context window 用得越少，结果永远越好。主持人：一种正在流行的新 paradigm 是 loop engineering。

Problem with loops is like at a certain point, you're going to generate so much code that you can't read it anymore. We built a lights off software factory in July of 2025 and by November we had shut it down. Can we talk about what you mean by token harder and token smarter? I'm in a group chat called hyperengineering and it's all like people trying to max out their cloud subs. 你认为它有什么问题？ Dex：loop 的问题是，到某个时点会生成多到无法阅读的 code。我们在 2025 年 7 月构建了 lights-out software factory，到 11 月就关闭了。主持人：可以谈谈 token harder 和 token smarter 各自指什么吗？ Dex：我在一个叫 Hyperengineering 的 group chat 里，里面都是尝试把 Claude subscription 用到极限的人，这就是我说的 token harder。

That's my idea of token harder and the goal is what happens when you let AI agent ship code for months and no developer reads a single line. Today's guest tried exactly that. He built a lights off software factory and four months later he had no choice but to shut it down as things just stopped working. Dexory is the founder of human layer and the person who coined the term context engineering days before Andre Carpathy and Tubiluska made it famous. 主持人：如果让 AI agent 连续几个月发布 code，没有 developer 看过哪怕一行，会发生什么？今天的嘉宾真的做过这个实验。他构建了一座 lights-out… … software factory，四个月后不得不关闭，因为系统逐渐停止正常工作。 Dex Horthy 是 HumanLayer founder，也是最早提出 context engineering 一词的人之一，比 Andrej Karpathy 和 Tobi Lütke 让这个词广为人知还早几天。

He spent the last two years talking to hundreds of AI engineers about what actually works when you build with LMS and is testing the most extreme ideas with his own team. 过去两年，他与数百名 AI engineers 交流哪些 LLM 构建方法真正有效，并在自己的 team 中测试最极端的 ideas。

In today's conversation we discuss context engineering, what it is and the physics of context windows, including what the dump zone is. loop engineering from the Ralph Wim technique to the slow loops that Dex's team runs every night to wake up to code cleanup PRs the rise of software factories from a NATO conference in 1968 through DevOps to today's agentic factories specdriven development and why specs always drift from the code itself and many more. 本期会讨论 context engineering 的含义、context window 的物理约束，包括 dumb zone；还会讨论 loop engineering。从 Ralph Wiggum technique，到 Dex 团队每天夜间运行、醒来便收到 code cleanup PR 的 slow loops；software factories 的发展则从 1968 年 NATO conference、DevOps 一路讲到今天的 agentic factories；还包括 spec-driven development，以及 specs 为什么总会与 code 漂移等。

If you want to understand increasingly important concepts like concept engineering and harness engineering or want to know how far you can push the let agents build everything idea from someone who pushed it further than almost anyone then this episode is for you. This episode is presented by antithesis. If you work with agents your job is no longer just writing code. It's specifying and testing it and antithesis is the most effective method of verifying agenda code today. 如果你想理解日益重要的 context engineering、harness engineering，或者想知道“让 agents 构建一切”究竟能推到多远，这期节目正适合你。本期由 Antithesis 呈现。和 agents 合作后，你的工作不再只是写 code，而是 specification 与 testing；Antithesis 是当下验证 agentic code 的有效方法。

Today's episode is brought to you by Buildkite, the CIOS platform trusted by OpenAI, Entropic, Cursor, Nvidia, Uber, Canva, and more. Today, we're talking about pushing the right context into models so that they write better code. Right after that starts working, your agents will write more code, a lot more. Trusting that code avalanche is where many teams face a challenge today. Every change that an agent makes still has to be built, tested, and proven safe before it ships. 本期也由 Buildkite 赞助，它是 OpenAI、Anthropic、Cursor、Nvidia、Uber、Canva 等信赖的 CI/CD platform。今天我们讨论怎样把正确 context 传给 models，让它们写出更好 code；一旦成功，agents 会写出更多 code，而且多得多。信任这场 code avalanche，是许多 teams 面临的挑战。

Worked on my machine is not enough. So, you obviously need CI. But when agents are pushing five, 10, or 50 times the commit volume into your pipelines, faster CI runners won't save you. Shaving 30 seconds off a single build is meaningless when the queue is 100 plus jobs deep. What you really want is a CI system that gets faster as the volume grows and CI that offers instant parallelization to give you unlimited concurrency and to intelligently route changes at runtime. agent 的每次 change 在发布前仍必须 build、test 并证明安全，“在我的机器上能跑”不够。所以需要 CI。但当 agents 把 commit volume 提高 5、10 或 50 倍时，单纯更快的 CI runner 无法拯救你。 queue 超过 100 个 jobs 时，单次 build 缩短 30 秒没有意义。真正需要的是… …一种随着 volume 增长反而更快的 CI system：提供即时 parallelization、unlimited concurrency，并在 runtime intelligent routing changes。

This is what Bill Kite does and why global software leaders continue to rely on it. The same architecture that observed the scale of Shopify and Uber a decade ago now runs about 1.4 billion job minutes a week across cursor meta and snowflake. While the rest of the CI world are cracking under the weight of rearchitecting your platform, build kite continues to reliably grow. Agents running on your infrastructure or build kites. 这就是 Buildkite，也是全球 software leaders 持续依赖它的原因。十年前支撑 Shopify、Uber 规模的 architecture，如今在 Cursor、Meta、Snowflake 等环境每周运行约 14 亿 job minutes。其他 CI 产品还在被平台重构压力压垮，Buildkite 则持续可靠扩展。

Any cloud, any chip, your secrets, your scale, every artifact and log is captured. So when something fails, either you or your agents have immediate insight for why. As you're ensuring the context you'll give to your agents, think about how you'll verify what they hand back. If your system is buckling under the increased volume, head to buildkite.com/pragmatic. 30-day all access trial, no credit card, and an actual human engineer on standby. agents 可以运行在你的 infrastructure，也可以运行在 Buildkite；支持任何 cloud、任何 chip，保留你的 secrets 和 scale，并捕获每个 artifact 与 log。出现 failure 时，你或 agents 都能立即知道原因。在设计给 agents 的 context 时，也要想清楚怎样验证它们交回的结果。如果系统正在被 volume 压垮，请访问 buildkite.com/pragmatic，获取 30 天全功能 trial，无需 credit card，还有一位真人 engineer Ola 随时待命。

His name's Ola, and he's very helpful.

So, Dex, welcome to the podcast. Super stoked to be here, dude. Before we get into some of the context engineering and some of some of the the more spicy stuff as well, how did you get into tech? How did you fall in love with computers? Oh, man. So, I uh I was doing undergrad as a as a physics major. Um, and I realized that uh I didn't like academia. 主持人：Dex，欢迎来到节目。 Dex：非常兴奋能来。主持人：进入 context engineering 和更尖锐的话题之前，你怎样进入 tech，又怎样爱上 computers？ Dex：我本科主修 physics，后来意识到不喜欢 academia。

And there's like basically like two or three paths out of physics is basically you go get a PhD or you go into finance or you go do programming. At that time, this was, you know, 2012, 2011 when it was like in the middle of undergrad and deciding what to do. And I had done an internship when I was in high school. I was working with NASA researchers to a jet propulsion lab in California. 离开 physics 基本有两三条路。要么读 PhD，要么进入 finance，要么做 programming。那是 2011、2012 年左右，我正在本科中期决定未来。高中时曾在 California 的 Jet Propulsion Laboratory 与 NASA researchers 实习。

They had just gotten this really highfidelity like the most uh you know fine grain data set of altitudes like the heights of very at very like top topographical map of the south pole of the moon. And the south pole of the moon is really interesting because some of the craters there are so deep because of the angle it has. It got hit by meteor storms like no other part of the moon. So there's very deep craters that have never seen sunlight. 他们刚获得一份高 fidelity、极细粒度的数据集… …描述月球南极高度，也就是非常精细的 topographical map。月球南极很有意思：由于撞击角度，某些 craters 极深，经历过月球其他区域没有的 meteor storms。

And so there's frozen liquid water in there from the formation of the moon. And so scientists were really interested in getting down there and exploring. And uh so we had this really fine grain map and it's like okay cool. Let's build software so that I I have point A to point B. I know the limitations of my rover can you know max incline up is this, max incline down is that find a path from point A to point B that doesn't like break those rules of the incline. 这些深坑从未见过阳光，因此从月球形成时起就保存着 frozen water，scientists 很希望下去探索。我们手里有这张高精度地图。任务是构建 software：已知 point A、point B 和 rover limitations，比如最大上坡与下坡 incline，找到一条不违反坡度限制的路径。

So I was you know 17. I had never cracked a CS textbook. So I wrote I basically like wrote a really naive bad version of Dystra's algorithm for pathf finding. Uh so I was in college. I was like, I don't know if I want to do the academics thing, but I really enjoyed programming back in the day. And so, uh, so I decided to go I got like half of a CS minor and then started working on a API platform team at a software company in Chicago and Sprout Social, right? 我当时 17 岁，从未翻过 CS textbook，于是写了一个非常 naive、很糟糕的 Dijkstra algorithm 版本做 pathfinding。到了 college，我不确定是否想走 academia，但记得自己很喜欢 programming。

Yes. And uh, basically never went back. Yeah. And then then where did you go from there? Where did you pick up like the parts of the trade? Because very early on, your first job that's not really common. you were doing platform engineering back in you know more than a decade ago. 于是我修了大约一半 CS minor，随后加入 Chicago 一家 software company 的 API platform team。主持人：就是 Sprout Social？ Dex：对，从此再也没回头。主持人：之后怎样发展？在哪里学会这门手艺？

From that point, it took me about two or three months to notice that like the most valuable work that was being done in the company was being done by like of course it's obvious like the first couple engineers who know everything and understand where everything was and like you spend a day on a support ticket from a customer and they solve it in 5 minutes but like you have to solve it so you learn and whatever. 十多年前第一份工作就做 platform engineering，并不常见。 Dex：两三个月后我就注意到，公司最有价值的工作由最初几位、了解所有事情的 engineers 完成。你花一天处理 customer support ticket，他们五分钟就解决；当然你仍得自己解决才能学习。

And I realized like the most valuable people in the company were the people that were building the developer platform CI/CD sandbox environments preview stuff. And so I kind of like that was my first step into the journey and I've basically been obsessed with software factories since that like three or six months into my first job. We talk about software factories now but you're you're talking about software factories back then. 我意识到，公司最有价值的人正是在构建 developer… … platform、CI/CD、sandbox environments 和 preview 等。这是我走上这条路的第一步；从第一份工作三到六个月后，就开始痴迷 software factories。

So like you were you're you were starting to already think that this is how we can produce better software inside this is pre AI world right? Well and I'm always surprised like there's a huge class of developers that say I don't want to work on CI/CD. I hate CI/CD. I'm like really because building the thing that builds the thing and building the thing that builds the thing is like as software engineers we're lazy. 主持人：如今常谈 software factories，但你在 pre-AI 时代已经思考怎样更好地生产 software。 Dex：我总惊讶于大量 developers 说不想做 CI/CD、讨厌 CI/CD。我会想，真的？ “构建用于构建事物的事物”，再构建它背后的系统，正是 software engineers 追求高 leverage 的表现。

We want to do the most high lever thing that makes our job easier. So how do we if we can build a thing that helps us build a thing that helps us move faster then that's the best use of my time as a as a lazy engineer. And then you went to another startup uh as aspiration. Aspiration. Yeah. Aspiration also platform engineering. 我们很懒，希望做最能让工作变轻松的事情；如果能构建一个帮助我们构建、加速工作的东西，那就是 lazy engineer 最好的时间投入。主持人：之后你去了另一家 startup Aspiration。

嘉宾 Yeah. I was brought in and then like three 3 months into the job, the VP of engineering who hired me quit or got fired. I don't know. There was some drama about it. I probably shouldn't talk about it. And then I was there for about a year uh and was kind of like acting CTO for a while like hired a couple people, helped hire the new VP of engineering, but I was out of there. I don't think I'll ever do consumer again. I think I'm actually a B2B guy. Good to know. Dex：对，Aspiration，也是 platform engineering。入职大约三个月后，聘请我的 VP of Engineering 辞职或被解雇，其中有些 drama，不太方便谈。我在那里约一年，一度像 acting CTO，招了几个人，也帮助招聘新的 VP of Engineering，之后离开。

And then you went to replicator where you spent like a good like like solid like four years and went from engineer for deployed engineer to product manager. Yeah, I did core engineering for like two years. We were building a container orchestrator like before Kubernetes, before Docker Swarm was really a thing. We built our own orchestrator. The founders had this vision that like, oh, Docker is going to make it much easier to ship on-prem software. 我大概再也不会做 consumer，自己其实是 B2B 类型。主持人：然后你去了 Replicated，扎实做了约四年，从 engineer、forward-deployed engineer 转为… … product manager。 Dex：我做了大约两年 core engineering。那时 Kubernetes 和 Docker Swarm 都还没真正成熟，我们自己构建 container orchestrator。

And when I say onrem, I don't mean literally like a a rack in a colo. It's more like, hey, look, bring the app to where the data is rather than sending the data up to some cloud vendor. And Docker makes it much more much easier to package up apps and and and move them around. founders 的 vision 是 Docker 会让 on-prem software 更容易发布。

And so they had this thesis that like basically you could build a platform that the experience that you get when you use GitHub enterprise which is like you install it and it has this admin panel but then you just get GitHub running in your data center and your code never has to leave your your data center. Suddenly you could build a generic SAS where everybody could have that. So I did two years as an engineer there and then our head of sales. 这里的 on-prem 不一定是 colo 机架，而是把 app 带到 data 所在的地方，不把 data 发给 cloud vendor。 Docker 使 apps 更容易 packaging 和移动。他们的 thesis 是构建一个通用 platform，让每个 SaaS 都获得类似 GitHub Enterprise 的体验：安装后有 admin panel，GitHub 运行在自己的 data center，code 永远无需离开。

We parted ways with our head of sales and uh honestly I was having a lot of arguments about the software factory with our CTO and it's kind of like almost like a too many cooks in the kitchen kind of thing. I'm sure many listeners listen listeners have had this experience of like well yeah I know I have these tickets to build but like CI sucks. 我在那里做了两年 engineer。后来公司与 head of sales 分手，而我也一直与 CTO 围绕 software factory 争论，有点像厨房里厨师太多。

I got to fix CI because it's too slow or it's like there's too many different builds and it's always breaking. like I'm going to fix that and then I'm going to do the end is just like Dex, I need you to stop fixing the build pipeline and like do the tickets I gave you. I'm sure you've had this experience perhaps. Yeah. And then and was this what led you to either forward deploy engineering? Yeah. So I like I really loved our customers. 许多听众大概经历过：明明有 tickets 要做，却觉得 CI 太慢、build 太多且经常坏，所以想先修 CI，再完成任务。最后领导会说：“Dex，别再修 build pipeline，先做我给你的 tickets。 ”主持人：这促使你转向 forward-deployed engineering 吗？ Dex：是的，我非常喜欢 customers。

Our customer our customers are Hashi Corp, Data Stacks, Puppet, all these really cool engineering brands. TravisCI, CircleCI. I was like yeah I actually love working with our customers. Our customers are awesome. customers 包括 HashiCorp、DataStax、Puppet、Travis CI、CircleCI 等优秀 engineering brands。

And uh it was a great way to like get in the trenches. a lot of really good engineers who were solving the hardest problem at the company which is like how do we take this 3 to 5year-old SAS platform and package it all up so that someone who knows nothing about our architecture can run it reliably in their own AWS VPC in their own on-prem data center whatever it was and so I spent I was our first kind of customerf facing engineer and it was in about three months I we closed I met with like every company customer that was like kind of in the pipeline but wasn't moving saleswise is and we closed like 12 deals in 3 months and the CEO was like, "Holy crap, Dex. 我喜欢和他们一起深入一线。他们有很多优秀 engineers，在解决公司最困难的问题：怎样把已有三到五年的 SaaS platform 全部 package，让不了解 architecture 的人也能在自己的 AWS VPC 或 on-prem data center 中可靠运行。我成为第一位 customer-facing engineer。大约三个月里，我见了 pipeline 中 sales 一直无法推进的每个 customer，最终三个月签下约 12 单。

Like the the investors are taking my calls again. Like I don't I know you want to get back to coding, but like I need you to go hire three people and like build this team out cuz I think you might have been like born for this." CEO 对我说：“天啊，Dex，investors 又开始接我电话了。我知道你想回去 coding，但需要你再招聘三个人、建立 team，因为你也许天生适合这件事。 ”

主持人 Wow. Yeah. So I did that for about four years, built that or to like 25 people and then Zer happened and uh it got a lot smaller and we kind of realized like, hey, we have a product that's like pretty good uh and we've been solving what lots of early startups do is like, okay, there's some usability issues. is we'll throw we'll get a bunch of smart people, throw them in the trenches with our customers, great for sales, great for retention, all this stuff. 主持人：哇。 Dex：我做了约四年，把 organization 建到约 25 人。后来 ZIRP 时代结束，团队大幅缩小。我们意识到 product 本身其实不错。很多 early startups 会用一群聪明人深入客户一线，补偿 usability issues；这对 sales、retention 都很好，但 margins 不够。

And it was like, oh, we actually like the margins on that aren't aren't good enough. And so we basically were like, cool, we actually just need to make the product way more usable, do a more PLG-shaped thing, make it productled growth,product led growth. Make it a little more self-service so you don't need an expert to teach you how to use it. And I was like, cool. If that's the most important thing, then I want to go be a product manager because I have tons of opinions. 于是必须让 product 易用得多，转向更像 PLG 的方式。主持人：product-led growth。 Dex：对，让它更 self-service，不需要 expert 教你怎么用。我想，如果这是最重要的事，那我就去做 product manager，因为已经有很多观点。

I've now spent four years in the trenches with our customers. I have a laundry list of roadmap things that I think would make the product way easier to use and adopt and implement and deploy and and now you went the full ar you went towards a dark side. Exactly. Yeah, I did. I was like this is going to kill my street cred isn't it? 四年一线 customer experience 让我积累了一长串 roadmap ideas，知道怎样让 product 更容易使用、采用、实现和部署。主持人：于是你彻底投入 dark side。

But uh I was really glad you know I think a lot of engineers are afraid that if they go do a customerf facing thing they lose all their credibility and like yes I wasn't coding for 10 hours a day. I was coding for like three or four hours on a Saturday for fun. Not uh but I mean we were helping people build YAML we were building CLIs. We owned a lot of the tooling that customers use, but it was like the last mile delivery side of it, not the core platform. Dex：没错，我还想这会不会毁掉自己的街头信誉。我很庆幸这样做。许多 engineers 害怕做 customer-facing 工作会失去技术 credibility。确实，我不再每天写十小时 code，但会周六为了兴趣写三四小时。

And like on a more personal note, I had spent the last like most of my 20s feeling like okay, a little bit introverted, a little bit like socially awkward. What I what a lot of engineers I'm sure experience and uh I had talked to my uncle's a music producer. So he used to work with like Randy Newman and a bunch of like really famous musicians. Oh wow. Yeah. 我们的工作仍帮助 customers 写 YAML、构建 CLIs，也拥有很多 customer tooling，只是负责 last-mile delivery，而非 core platform。个人层面，我二十多岁大部分时间有点 introverted，也有些 socially awkward。

This guy Mitchell F. And he he I was sitting with dinner with him at some point and when I was I think it was when I was still in undergrad, but he gave me this lecture. He was basically like if you want to be really good at something, you have to make it the only thing you do. The guy playing guitar nights and weekends trying to get his band off off the ground will probably never achieve greatness. 很多 engineers 大概都有这种体验。我叔叔是 music producer，曾与 Randy Newman 等著名 musicians 合作。主持人：哇。 Dex：有次一起吃饭，大概还在本科，他给我上了一课：想真正擅长某件事，就必须让它成为你唯一做的事情。

The people who become great are the people who basically make it like if I don't play guitar, I don't eat. And you go and you sit on the street all day and you play for 14 hours a day or whatever it is. That's the only way to become great. 那个只在夜晚和周末弹 guitar、试图让 band 起步的人，大概永远无法达到 greatness。

So, I said, "Okay, instead of trying to like read self-help books about how to be less introverted and less socially awkward, like what if I just made it my freaking job to just talk to people and make friends and like help people and solve their problems and uh I think it worked out. I recommend it. 真正伟大的人会让它变成“不弹 guitar 就没饭吃”，整天坐在街头，一天弹十四小时，这才是成为优秀者的唯一方式。

I think everyone should spend a year or two at least doing something really like customerf facing." Did you do this because you felt that it was holding you back be being introverted or or like what what what and I I know you got the motivation from the whole musician motivation. I I get it on one part, but what was it that you said like is a customerf facing thing that I'm I'm going to be doing it because clearly you were pretty great at like writing code by that point. 于是我想，与其读 self-help books 学习如何不再 introverted、socially awkward，为什么不直接让与人交谈、交朋友、帮助别人、解决问题成为自己的工作？我认为奏效了，也推荐大家至少花一两年做真正 customer-facing 的工作。主持人：你这样做，是因为 introversion 在阻碍你吗？我理解 musician 的激励，但你当时已经很擅长写 code，可以说日夜都在做。

You could argue you were doing it night and day. So where where did you find that like I actually I think like customerf facing or like getting this introvert off of me? Did you feel that I was holding you back or you just wanted to be good at it? It was just kind of a thing that was like interfering with my like general life satisfaction. And it was also like I'm not a very type A person. 为什么觉得应该做 customer-facing 工作、摆脱 introvert 状态？ Dex：它影响了整体 life satisfaction。此外我不是很 Type A，十分 disorganized。

I'm very disorganized. is I don't know if people call it like okay I'm like ADHD now that's why I can run 30 quads in parallel or whatever it is but it was like I was really bad at email and calendars and spreadsheets I just like didn't care about these didn't understand them and so like another side effect of this was like it just forced me to be organized and keep a lot of things going and so like I don't know there's like weird benefits you get from like stepping outside your comfort zone and learning like industrial disciplines that are separate from what you've been doing and so the opportunity presented itself and I was like oh I like working I'll try this for a little bit started going really well I'm like cool let's keep let's see let's see how far this thread goes and then afterwards you're now in your second startup. 也许现在有人会说是 ADHD，所以才能并行跑 30 个 squads。但当时我很不擅长 email、calendar 和 spreadsheet，既不在乎，也不理解。另一个副作用是，这份工作迫使我变得 organized，同时维持很多事项。走出 comfort zone、学习与过去领域不同的工业化 discipline，会带来一些意外收益。机会出现后，我想自己喜欢与人合作，可以先试一阵；进展很好，于是继续看看这条路能走多远。主持人：后来你成为 founder，如今已经是第二家 startup。

You you became a founder and you also got involved in in AI pretty early as I as it was even before it was so obvious that it would change how it would change how we develop software, right? Well, I would say I was I was later than I could have been because we started the company uh me and a buddy in Chicago started a company in the data engineering space in about 2020 November 20. We decided in like August of 2015,this is Metalytics. 你也很早参与 AI，甚至早于大家明确看到它会怎样改变 software development。 Dex：其实我本可以更早。 2020 年 11 月，我和 Chicago 的朋友创办一家 data engineering 公司。主持人：Metalytics。 Dex：对。从技术上说它与 HumanLayer 仍是同一家公司，只是 pivot 了 mission。

Metalytics. Um, technically still the same company as human layer, we just like pivoted the the the mission. But,uh, yeah, the the the advice I got from every angel investor that, you know, people who just knew CTOs I'd worked for before and stuff, they were just like, look, hitting a lot of heads, wins. 我从 angel investors、以前共事的 CTO 等所有人那里得到的建议都是“有很多 headwinds”。

I don't know if you know like the whole DBT data engineering fiverr that whole arc where it was like this huge party and tons of investor money going into all these different companies and then within by like 2021 2022 there was kind of the zer thing and just this general realization that the TAM for those sorts of tools is not as big as everyone market 你大概知道 DBT、data engineering、Fivetran 那段故事：大量 investor money 进入各种公司，像一场盛大 party；到 2021、2022 年，ZIRP 结束，大家普遍意识到这类 tools 的 TAM 没有市场想象得那么大。

嘉宾 yes the total addressable market for those sort of tools was was not as quite as big as uh as we all thought it was. Um so it was it was a hard place to raise money. It was a hard place to get customers. Yeah. And then I I met you at while you were at Human Layer NSF at an event. We you actually talked and we chatted afterwards. But by by that, this was about a year ago, you were already you started to have some really strong opinions on using AI. 主持人：也就是 total addressable market。 Dex：对，没有大家以为的那么大，所以融资、获客都很困难。主持人：大约一年前，我在你做 HumanLayer 时于 SF 的活动上见过你。那时你已经对 AI 使用形成一些强观点，其中之一就是后来出名的 12 Factor Agents manifesto。 Dex：现在叫 manifesto 了吗？

And one of them was this now famous 12 factor agents manifesto. Is are we calling it a manifesto now? I'm I'm calling it a manifesto. It's a manifesto. I'm calling it. Let's talk about this. This was 12 engineuring principles to build reliable production ready apps. uh how did you come up with this and maybe we can also talk about some of them. 主持人：我称它为 manifesto。它是构建可靠、production-ready apps 的 12 条 engineering principles。你怎样提出它们？我们也可以谈其中几条。

Yeah. So um I'll I'll kind of like go to like around August the co-founder I was working with kind of burned out and left and it was very we were on good terms. It was very mutual. Um and I decided to start messing with AI stuff and I was building a AI agents and what was really in fog right then was like the lang chain the crew AI these like agent frameworks. Um and it seemed like there was a ton of you go you go in the crew AI discord there's 10,000 people. Dex：大约 8 月，与我合作的 co-founder burnout 后离开，双方关系良好，是共同决定。我开始尝试 AI。我在构建 AI agents，当时 LangChain、CrewAI 等 agent frameworks 很热门。进入 CrewAI Discord 能看到一万人，感觉这种形态大概是正确方向，也有清晰 ecosystem。

It's like, okay, this feels like the right shape and this there's clearly this eco. You go in every single one of those projects, they have a Chroma DB plugin. They have like a Composeio plugin. There's like clearly like this is the shared interface that everybody is building for. I say, okay,what's missing from all of this? The agents can call tools, but it's really hard to like control which tools they call. 每个 project 都有 ChromaDB plugin、Composio plugin，明显是大家共同构建的 interface。于是我问… …这里缺什么？ agents 会调用 tools，却很难控制它们调用哪个。 chatbot 可以在 application UI 中显示 approve/deny，但我痴迷所谓 outer-loop agents 或 proactive agents：它们在 background 运行，由 events 触发。

And if it's a chatbot, obviously, you can show approved deny in the UI of your application. But I kind of was obsessed with what I would call like outer loop agents or proactive agent. Agents that would run in the background, get triggered by events. I mean, OpenClaw is basically like the biggest manifestation of this of like you have a heartbeat, it wakes up, it sees if there's any work to do, it tries to do stuff. OpenClaw 是最大的体现——heartbeat 唤醒后检查是否有工作，再尝试执行。我的想法是… …如果 agent 想做某件事时，系统不能通过 Slack、iMessage 等通知，并确定性保证我可以 approve、deny，或带 feedback 拒绝并要求“按这种方式重做”，我就不会信任它做任何有意义的事。

And my thought was like, I'm not going to trust that agent to do anything meaningful. if I can't get like a Slack message or an iMessage or something when it wants to do something and kind of guarantee deterministically that I can approve or deny that or deny it with feedback and say actually no do it like this. So we played in that space for a while and talked to a lot of founders and founding engineers and builders. We came into YC in the fall of 2024 with this idea. 我们在这个方向探索一段时间，与许多 founders、founding engineers 和 builders 交流。 2024 年秋季带着这个 idea 进入 YC，开始构建 API platform。

We're building out this API platform and it was sort of like pedag duty but like it wasn't who's on call to fix the servers. 它有点像 PagerDuty，但不是决定谁 on-call 修 server，而是 routing mechanism：谁需要 approve 这个 agent，能否 escalate、delegate 或 defer。

嘉宾 It was like who's on call to this like routing mechanism for like who needs to approve this agent and can they like escalate it or delegate it or defer it all this stuff and we built it for this ecosystem crew AI link chain fi there's so many grip tape there was so many in that in that time and then I talked to tons of AI engineers who were actually building really interesting things and like actually making money doing six figure contracts shipping AI to the enterprise and all of them had tried that stuff for like a month or two and then they had thrown it out and they were just writing all a API calls by hand and they were building more things that look more like pipelines and workflows than these sort of like hands-off call tools in a loop kind of thing. 我们为 CrewAI、LangChain、Phidata、Griptape 等 ecosystem 构建。后来我与许多真正构建有趣产品、通过六位数 contracts 向 enterprise 交付 AI 的 engineers 交流。他们全都尝试这些 frameworks 一两个月后丢掉，改成手写 API calls，构建的东西更像 pipelines 和 workflows，而不是无人控制、持续 tool calls 的 loop。我与约一百人交流，也花了很多时间和好朋友 Boundary 的 Vib 相处。

And so I talked to a hundred people and I spent a lot of time a lot a lot of time hanging out with one of my best friends uh Vib from uh Boundary. So they build a programming they built like this like protobuffs for AI thing and they're I think they're about to launch their like full fat like programming language touring complete thing. 他们在构建类似“AI 领域 protobuf”的 programming system，似乎很快会发布完整、Turing-complete 的 programming language。他思考 agents、models 和 inference 的方式，更关注 structured output 底层究竟是什么。

But he had this way of thinking about agents and building with models and building with inference where it was a lot more about understanding what structured output really is under the hood. And every single step in your AI workflow is just tokens in tokens out. AI workflow 的每一步都只是 tokens in、tokens out。 engineer 的工作是弄清楚：需要输入哪些 tokens，才能最大化输出 tokens 质量良好的概率。

And your job as an engineer is figure out, okay, what tokens do I need to put in to maximize the chance that the tokens out are going to be good. and kind of distilled all these ideas into about 12 principles and wrote about it on GitHub, posted just like this like 12-page GitHub repo, threw it on HackerNews, got like five, it was on the front page for like two days and it I think it really resonated with a lot of people. 我把这些 ideas 提炼成约 12 条 principles。我在 GitHub 写成一个约 12 页的 repo，发到 Hacker News，在首页待了约两天，引发很多共鸣。

Yeah. So, I I'll just quickly read the 12 principles and and then let's talk about like one or two that resonate. Sub 12 are natural language of tool calls, own your prompts, own your context window. Tools are just structured outputs. Unify execution state and business state. Launch pause resume with simple APIs. Contact humans with tool calls. Own your control flow. Compact errors into context window. Small focused agents. Trigger from anywhere. Meet users where they are. 主持人：我快速读一下 12 条：natural language to tool calls；own your prompts；own your context window；tools are just structured outputs；unify execution state and business state；launch/pause/resume with simple APIs；contact humans with tool calls；own your control flow；compact errors into context window；small, focused agents；trigger from anywhere；meet users where they are；make your agent a stateless reducer。

Make your agent a stateless reducer. The stateless. Yeah, the stateless reducer one was a little actually someone hit me up on Twitter and uh corrected me. It's actually it's actually a transducer because there's technically multiple steps in the workflow, but there we go. But but but of this one, this this was a year ago,so like which is like forever in uh in how the tooling is is evolving.

Which ones still stick with you where you're like, "All right, these were good that still seem to hold off." Yeah, I think I'm going spent most of March writing it, published this in April. Uh, and then Swix hit me up from AI.engineer and he said, "Hey, can you come? You want to come talk about this." So, I gave this talk 12 factor agents in like June 6th, I think. And, uh, small room maybe like it was packed, but it was like maybe a hundred people. Dex：最后那个 stateless reducer 后来有人在 Twitter 纠正我，严格说是 transducer，因为 workflow 有多个 steps。主持人：不过这是约一年前提出的，按 tooling 演进速度已经很久。哪些仍然有效？ Dex：我花了三月大部分时间写，四月发布。 Swyx 从 AI Engineer 找到我，邀请我去讲。

That was the year at AI engineer where like the lower physically like on on the on the second basement floor was all the super corporate stuff and you go up a level is a little bit more and then like on the top floor is all the like weird cutting edge like startup stuff that like you probably shouldn't care about yet kind of thing. So we were up there on the top of this like weird way of thinking about agents. 大概 6 月 6 日，我做了 12 Factor Agents talk，房间很小但坐满，大约一百人。那年的 AI Engineer conference 很有意思：地下二层是最 corporate 的内容，越往上越前沿。顶层全是古怪、cutting-edge 的 startup 话题，当时可能还不值得关注。

Uh and then about a week later or two weeks later uh Tobi Lütke from Shopify says I really like this idea of like context engineering. And I'm like I I wrote about this two months ago. This is great. Toby gets it and then a week later Andrej Karpathy is like well I really like I think what we should think about is not prompt engineering but context engineering. And I was like, "Yes, that's my." Anyways, I don't know. 我们就在顶层讲这种奇怪的 agents 思维方式。一两周后，Shopify 的 Tobi Lütke 说很喜欢 context engineering 这个 idea。我想：“这是我两个月前写的，太好了，Tobi 懂了。 ”再过一周，Andrej Karpathy 说不该只思考 prompt engineering，而应思考 context engineering。

If you ask Gemini, depends what day it is, they will tell you either me or Toby or Andre came up with context engineering. You can't really own a word. Like I don't no one remembers who invented the word prompt engineering. But of all the factors, factor three of own your context window. 我想：“对，那就是我的… … ”不过你无法拥有一个词。根据日期不同，Gemini 会说是我、Tobi 或 Andrej 提出 context engineering。

And basically the only way you can whether it's agentic or a single step at a pipeline, the only way you can impact the quality of your output from AI is by caring a lot about what the inputs and crafting them. Let's talk about context engineering, which I am going to credit you that you coined it. I I did some research and like I think you were earlier by a few days. So there we go. You you coined it. We're adding we're adding to the we're adding to SEO juice. 也没人记得 prompt engineering 一词是谁发明的。但所有 factors 中，第三条“own your context window”最重要。无论是 agentic system 还是 pipeline 中的 single step，影响 AI output quality 的唯一方式… …就是认真对待 inputs 并精心构造它们。

We'll have it in a transcript. Dex coined context engineering. Well, and and like a asterisk on that is basically like I learned about context engineering from talking to these hundred engineers and founders. I just kind of like what was the same about what they were all doing and I put a name on it. So like I didn't invent doing it. 主持人：那就谈 context engineering。我愿意把这个词归功于你，我查过，你似乎早了几天。 Dex：需要加个注脚：我从与那一百位 engineers 和 founders 的交流中学到 context engineering，只是发现他们实践中的共同点，再给它起名。

I was just like I think we I think there's this thing and like vocabulary and names are really important and having like clean ways to talk about the problem especially when like a lot of the content about AI right now is so much hype and jargon that is like meaningless. I was like okay I think there's a word here that is useful to builders that explains how they should be thinking about building their software. So what is context engineering? 我没有发明这种做法，只是觉得这里存在一个值得命名的概念。 vocabulary 与 names 很重要，尤其当下许多 AI 内容充满 hype 和没有意义的 jargon。一个干净的词能帮助 builders 讨论问题，也解释他们应怎样思考 software construction。主持人：所以什么是 context engineering？

It's kind of like deabstracting a lot of the abstractions that have been layered on top. So you have rag, you have memory, you have agentic history, you have structured output, you have all these things that are like different ideas in the frame of agentic programming. And at the end of the day, they're all like different ways to pass tokens into a model and ask it to produce usually some structured output. Dex：它有点像把叠加的大量 abstractions 拆开。 RAG、memory、agentic history、structured output 等，在 agentic programming 中被当作不同 ideas；归根结底，它们都是把 tokens 传入 model，并通常要求生成 structured output 的不同方式。

And understanding that is a lot more powerful than trying to learn memory and trying to pick some agent framework off the shelf and some memory framework off the shelf. I mean, those are these things are all really good. If you want to get to like 80%, you want to get a really good demo. But when you have to go from 80% to 95% or 99%. You need to go down a level and think about what's everything we're putting into the context window. 理解这一点… …比单独学习 memory、随便选择现成 agent framework 和 memory framework 更有力量。这些工具在达到 80%、做出优秀 demo 时都很好；但要从 80% 提升到 95% 或 99%，就必须往下一层，思考放进 context window 的全部内容、排列顺序，以及面对不同 model 时如何调整。

What order is it going in depending on which model we're doing? And all of this stuff matters. You have all of these levers that you can pull. 所有细节都重要，你拥有许多 levers。

And it just felt like the right abstraction for thinking about how do I get AI to do the thing I want as accurately as possible. Why is context engineering started to become more more talked about? It it was about a year ago. Was it did it have to do with the context the the context window that we could pass on to LLMs pretty much. 这是思考“怎样让 AI 尽可能准确地做我想要的事”的正确 abstraction。主持人：为什么 context engineering 在大约一年前开始受到更多讨论？

Did it start to expand or did did we just start to realize that we can do a lot more by passing on from you know the easiest one is of course system prompts but of course whenever you build an LLM behind the scenes you will pass additional context as well not just to prompt the user you will add a bunch of stuff that's I guess a dirty secret of any any LM but why do you think the focus is moving on to like all right context is important I think it always was important I think what had to happen is a ton of smart people again like all these builders I talked to a ton of smart people had to like focus really hard on producing like I want to make software that I can sell. 是因为能传给 LLM 的 context window 扩大，还是大家意识到除了 system prompt，还能通过额外 context 做更多？任何 LLM 在幕后都会传入很多额外 context，不只有 user prompt。为什么现在焦点转向“context 很重要”？ Dex：它一直都重要，只是需要大量聪明 builders 真正专注于生产 software：“我要做出值得骄傲、可以卖给 enterprise，并让客户满意的产品。

I want to make something that is accurate enough that I'm proud of and I can sell to an enterprise and they're going to be happy with it. And there's just like the the the the easiest way to get to really high quality AI applications is by thinking at that token level. ”想做出高质量 AI application，最直接的方法就是在 token level 思考：把系统理解成不同 LLM calls 的组合，而不是只有一个开放、灵活却不可靠的 tools-in-a-loop。

Thinking about a string of different LLM calls like rather than just tools in the loop and it's kind of open-ended and very flexible but not that reliable. thinking of agents as as workflows, as pipelines, as some mix between maybe a couple tools in a loop versus just, hey, I have my tools and I have my model and I have my system prompt and these are the only levers I have. And it's actually no, you have way more levers. 把 agents 看作 workflows、pipelines，或其中掺杂少量 tool loop。不是只有 tools、model、system prompt 三个 levers，而是还有很多。虽然需要更多工作，也必须对 LLM 建立更深 intuition，但这一直是必要层级；只是 builders 花了一段时间才意识到，它能突破 quality ceiling。

It's going to take more work and you're going to have to like understand the LLM with a deeper intuition. But it was a thing that we always needed and it just took time for people to build with this technology to figure out that like this is the layer of abstraction that allows you to break through the quality ceiling. And how are cost and context engineering connected? Yeah. Um I don't know. 主持人：cost 与 context engineering 怎样连接？ Dex：和 LLM 合作时，我喜欢说“make it run, make it right, make it fast”。先看当时世界上最好的 LLM 能否解决问题。

I was I was talking about this with uh someone this morning um about like when you're working with LM, one of the things I I like to say is kind of like make it run, make it right, make it fast. see if the world's best LLM at the time I think we did a podcast episode that at the time it was like 03 see if 03 can solve your problem and then give it to people and see if they want that and then if people want it and you use it a lot then go do a bunch of context engineering because your engineering time is always the bottleneck like humans trying to figure out and solve problems and build evals and improve and try different dimensions or set up jeep or whatever it is is always going to be more expensive than just using a smarter model until you have millions of requests a A and then it's like, okay, we're going to do a bunch of context engineering, break this up into three calls, and get it to work on GPT40. And then we're going to take two of those and make those two work on GPT40 and using old model names. 我们之前录节目时可能是 o3。先让 o3 尝试，再给 users 使用，看他们是否需要。如果用户确实想要并大量使用，再投入 context engineering。因为 engineering time 始终是 bottleneck：humans 分析、解决问题、构建 evals、尝试不同 dimensions 或做设置，通常都比直接使用更聪明 model 昂贵，直到 requests 达到数百万级。那时才值得大量 context engineering，把任务拆成三次 calls，设法让 GPT-4o 完成，再把其中两次降到更低成本 model。 model 名称可能过时，但重点是，对于某项 task…

But the point is like for a certain task in your workflow, can you get GPT OSS 12B, which is like 1/ 1,000th of the cost of Opus, can you get it to solve parts of the problem so that the tokens and the things you're using the smartest frontier models for are just the things that you really need, that level of intelligence? …能否让成本只有 Opus 千分之一的 GPT-OSS 12B 解决其中部分问题，把最聪明 frontier models 的 tokens 只用在真正需要这种 intelligence 的地方？但在证明需求和价值之前，不应构建和过度设计所有这些。

But you shouldn't go build all of that and overengineer it until you've proved that you need it that it's valuable that it's like okay this is now I mean we get to Eli Goldrat and like what is the the he had this book the goal right it was about how to model your factory and I'm sure we'll get to that when we talk about software factories it was like what is the bottleneck in your system and one day it will be latency and cost but it's probably not that when you first start out and context engineering is how you move from the you you add human effort to the equation to improve the efficiency the speed the price the cost efficiency of your system. Eliyahu Goldratt 的《The Goal》谈到怎样建模 factory，software factories 部分大概还会回来：你的 system bottleneck 是什么？某一天会是 latency 和 cost，但刚起步时大概不是。 context engineering 就是把 human effort 加入 equation，改善 system 的 efficiency、speed、price 与 cost efficiency。

Interesting. And then one thing that came up more recently and a lot later uh recently is harness engineering. What is harness engineering? So I made a post in like October I think about or maybe November of of like hey there's this new thing that I see is like I'm calling it harness engineering. My definition that I had at the time is not what actually this guy Viv who's at lang chain now does a lot of really good writing on agents and how to think about harness. 主持人：有意思。最近又出现 harness engineering，它是什么？ Dex：大概 10 月或 11 月，我发 post 说看到一种新东西，称为 harness engineering。我当时的 definition 与 LangChain 的 Viv 后来写法不同；他很擅长写 agents 与 harness，实际上比我早几周写过 harness engineering，但当时我还没读。

He had written something called harness engineering like a couple weeks before me but I hadn't read it at that point. And my take was basically like okay when you build an agent you use use context engineering. When you use an agent because we gave this talk in August of 2025 about like how to apply context engineering to how you use coding agents. 我的想法是，构建 agent 时使用 context engineering。使用 agent 时，则可以把 context engineering 应用到 coding agent。

嘉宾 And that kind of evolved into this idea of like how do you take a harness like cloud code like codeex how do you engineer against the integration points of that harness. So commands, MCPs, skills, how you organize your codebase. How do you kind of optimize the environment that the coding agent runs in to like get the best results? The same way with context engineer, how do you optimize the inputs to every single prompt? 我们在 2025 年 8 月做过相关 talk，后来演变成：怎样面对 Claude Code、Codex 这类 harness 的 integration points 做 engineering，包括 commands、MCPs、skills、codebase organization；怎样优化 coding agent 运行的 environment…

Well, harness engineering just is like how do I raise the floor so that every single turn of this thing, the results are as good as possible. And the term got super blurry and some people think harness engineering means building a harness. And some people think hard harness engineering means building around a harness. …以获得最佳结果。 context engineering 优化每个 prompt 的 inputs；harness engineering 则提高 floor，让每一 turn 的结果尽可能好。

I actually like what Martin Fowler came up with uh as usual he's very good at naming things and he kind of defined the you have the LLM and then you have the inner harness which is like the thing the the tool definitions and the integration points that like say like a cloud code or a codeex or a amp actually exposes that's your inner harness and then you have the outer harness which is the stuff that you the human do to customize that for your specific needs your codebase your languages etc that's the best definition I think we have for harness engineering it's interesting how naming is still so important, isn't it? 这个词后来变得模糊，有人认为它指构建 harness，也有人认为指围绕 harness 构建。我喜欢 Martin Fowler 的命名。你有 LLM，也有 inner harness，也就是 Claude Code、Codex、Amp 暴露的 tool definitions 与 integration points；再有 outer harness，也就是 human 针对自身 needs、codebase、languages 等做的 customization。

Well, it's like as soon as you name anything, people are most people are I'm actually surprised that context engineering still means the same thing to most people that it did a year ago and that it's even still relevant. Like that's honestly the the craziest thing to me is like you wrote how many things that were written about AI 15 months ago still matter or still interesting um or are still like have good advice baked into them. 这是我认为目前对 harness engineering 最好的定义。主持人：命名仍然如此重要。 Dex：一旦给东西命名，含义通常会变。我很惊讶 context engineering 一年后对多数人仍表示同一件事，而且依然 relevant。

Stuff changes a I think context engineering has been so long lived because it's it's grounded in the fundamentals of how transformer attention works and until we have post transformer models or linear attention or whatever it is which who knows when that's going to happen context engineering will be interesting and important to anyone building on AI and can we talk about the physics of of context uh you you you had a you had a tweet uh this this one the the context reality check this is a graph of uh as you get to 1 million context just the quality just drops it. 15 个月前写的 AI 内容还有多少今天仍重要、仍有有用建议？变化太快。 context engineering 能够长寿，是因为它扎根于… … transformer attention 的 fundamentals。在 post-transformer models、linear attention 等技术真正出现前——没人知道何时——context engineering 都会对 AI builders 保持重要。

It goes down. What do we need to know about like the context? Again, we we now have models that do have a 1 million context window. Maybe we'll have even longer ones, but when you start to just put in more stuff into the context, it starts to become less efficient. Like what what do we know so far in terms of from a practical perspective of like someone who is using the context window to add on a bunch of stuff? 主持人：可以谈谈 context 的 physics 吗？你发过 context reality check 图，显示接近 1 million context 时 quality 明显下降。

May that be MCP, may that be tools, may that be scales, may that be all of these things. Yeah. I mean, so the longer context windows are good. You can talk to it for longer. Like they're doing a good job. But at the end of the day, like especially when you had like Opus, it was like Opus 4.5 and then Opus 4.51 mil or 4.6 and 4.61 mil. 现在有 models 支持 1 million context，未来也许更长；但加入更多内容后效率会下降。实际使用 context window 加入 MCP、tools、skills 等内容的人，应知道什么？

You're not actually getting a like smarter model. like the intelligence of the model is is what drives its ability to attend to all of the tokens in the context window to figure out on the next turn which parts of this 100k or 200k context window are the most relevant to making the decision of like what is the next tool we call and doing that over and over again in a loop. Dex：更长 context window 是好事，可以对话更久，models 也做得不错。但以 Opus 4.5、Opus 4.5 1M 或 4.6、4.6 1M 为例，你得到的并不是更聪明的 model。 model intelligence 决定它能否关注 context window 中全部 tokens。

So I don't know there was some study that came out in 2025 which found that and again these are old models so like inflate your numbers but it was like frontier LLMs can follow about 150 to 250 instructions before it starts to drop off. Their ability to follow all the instructions just like drops off pretty quickly. 它必须在每一 turn 从 100K 或 200K context 中找出最相关内容，决定下一个 tool call，再在 loop 中反复执行。

And I think Lori Vos I haven't actually looked at the data but they did a study with like the next generation models a year later and it looks like it's like much better the number of instructions you can get in. In any case, you have like I split context engineering into like two categories. 2025 年有研究发现——虽然 models 已旧，数字应上调——frontier LLMs 大概能同时遵循 150 到 250 条 instructions，之后表现迅速下降。

You have like the the most people think about like the information budget of like okay I can do rag and I can pull out chunks of this document rather than putting the entire book into my context window. I can just go grab the pages that matter. 一年后 Lori Voss 等人用 next-generation models 做的 study 似乎显示数量提高很多，虽然我没仔细看 data。无论如何，我把 context engineering 分成两类。

But it's also your instruction budget is like if you give the model too many instructions and especially too many conflicting instructions and that's in your initial prompt and also like if you have a conversation you start going down a path and then you change your mind and you start going down a different you actually I don't want to do any of that I want to do this. It's like a it's a lot of computation the model has to do to notice that it has to ignore that whole thing. 多数人想到的是 information budget：用 RAG 提取 document chunks，不把整本书塞进 context，而只取相关 pages。另一类是 instruction budget。给 model 太多 instructions，尤其互相冲突，会出问题。

And when both of those things are kind of far back enough in the context window that they're only half getting attended to, your likelihood that it's like actually going to like remember the exact instructions you gave it 100,000 tokens ago is like it goes down quite significantly. conversation 中先沿一条路走，后来改变主意说“不做前面那些，改做这个”，model 必须花大量 computation 识别并忽略之前全部内容。

This is all very interesting because as engineers there we are expected when you know when we're AI engineers which now a lot of software engineer meaning you just like use LMS to to build software like underneath there's an LLM layer somewhere you're an AI engineer congratulations but it sounds like the expectation is to be you know to be a good to be a good software engineer preI you need to understand you know how to write good code and it helps when you understand a little bit of the underlying we didn't need to do that much over time but it it it never hurts but sounds Like right now we're in this phase that to be an engineer who can write an efficient AI system that use LMS. 当两组内容都远在 context 前部、只能得到部分 attention 时，model 准确记住 100,000 tokens 前 instructions 的概率会显著下降。主持人：这很有意思。如今许多 software engineer 使用 LLM 构建 software，底层有 LLM layer，就已是 AI engineer。看来他们需要理解 context dynamics。 pre-AI 时代，优秀 software engineer 需要写好 code，了解一些 underlying mechanism 也有帮助。

You need to understand the dynamics of the context you need to understand why stuffing your context one way or the other can be compute can introduce latency and all of these. It sounds like it's kind of more of an intuition and of course there's some understanding but from talking to you you're like well it it it does this computation like I know you know cuz you tried it out right? 如今要写高效 AI system，就必须理解 context：不同 stuffing 方式怎样增加 compute 与 latency。听起来更多依靠 intuition。你会说“它进行了这种 computation”，是因为反复试过。

Yeah like I'm not I'm not a PhD in machine learning like I couldn't actually go like draw a mathematical proof of how this works but we know attention is quadratic and the more stuff you put in the more it has to spread this attention out over everything. Dex：我不是 machine learning PhD，无法画出 mathematical proof，但知道 attention 是 quadratic；放入内容越多，attention 就必须分散到所有内容上。

This just feels like an absolute new area and like a little bit very different to like what we're used to like software engineering which is like pretty kind of like black and white, right? That compiles or doesn't compile. That's true. I mean there's a different kind of intuition. 主持人：这是全新领域，与 black-and-white 的 traditional software engineering 很不同，后者要么 compile，要么不 compile。

I was talking about this earlier as well is like there's a different kind of intuition that you develop over years as a software engineer and uh there's many categories of it but the one I'll I'll call attention to that is like a thing that you cannot teach you cannot do you cannot learn in a textbook. The only way to learn it is like I know bad patterns in software because I have debugged them at three in the morning. Dex：确实是另一种 intuition。 software engineer 多年形成很多无法从 textbook 学会的 intuition。我知道 software 中哪些 patterns 很糟，因为凌晨三点 debug 过它们。

嘉宾 This is my buddy Jake from Netflix said this in his talk at AI engineer code. It's just like there's no better way to learn what is good and what is bad and what works and what doesn't than suffering through the thing that doesn't work. Well, speaking of suffering through the things that that doesn't work, uh a new paradigm uh that is spreading up is loops. Loop engineering. The idea that instead of writing prompts, just write loops. Set up your loops. Netflix 的朋友 Jake 在 AI Engineer talk 中说，没有比亲自承受 failure 更好的方式来学习什么好、什么坏、什么有效。主持人：说到承受 failure，一种正在兴起的新 paradigm 是 loop engineering：不再写 prompts，而是写 loops。这从 Ralph Wiggum technique 开始，可以看作早期 loop。

And this all started with the Ralph Wiggum technique where it it will just well it I I guess that's an early version of loops that were just loops around and now we're hearing with some of the big biggest labs talking about that they're actually just doing looping. What is your take on have have you done some looping yourself? Have you set up some loops? And what do you think is good about it and what do you think is bad about it? 现在一些大型 labs 也说自己在 looping。你做过吗？它的好处和坏处是什么？ Dex：这个话题我能讲十分钟，足够做整场 talk。我先概括高层内容。一年前零四天，我第一次看到 Ralph Wiggum demo。

Yeah. So I think of loops as I mean this could I could ramble on this for 10 minutes. This is an entire talk, but I'll I'll try to I'll try to lay out some highle stuff and then we can dig in wherever you think is most interesting. We had Ralph Wickham. Jeff Hunley 来 SF，展示自己让 Sonnet 日夜运行，六周花 6,000 美元，构建完整 Gen Z programming language；它能 compile，还有 stage-two compiler，也就是用语言本身编写 compiler。

It was actually a year and four days ago was the first time I saw the Ralph Wickham demo and like Jeff Hunley was just like visiting SF and he just like came through and like dropped everybody's jaws with his like, "Yeah, I just ran Sonnet around the clock and spent six grand in six weeks and like I built an entire Gen Z programming lang." Look at it compiles and it has a stage two compiler where the compiler for the language is written in the language itself and all the insane. 核心 lesson 是 backpressure。很多人早已实践：怎样让 model 检查自己的工作？怎样自动把 feedback 传回 model？可以有许多形式。可以用 deterministic linters、unit tests。 programming language 适合 Ralph，是因为可无限验证：写 code、compile；compiler failure 就修；运行 program，失败再修 compiler。

And the core lesson from all of that I think was the idea of back pressure which is basically and I think a lot of people were doing this for a very have been doing this for a long time which is how do I let the model check its own work? How do I automate the process of getting feedback into the model? And there's lots and lots of different flavors of this. You can have deterministic llinters. You can have unit tests. 它高度 verifiable。 loop engineering 的 lesson 是，如果 problem 可验证，就能… …把它当作 black box 并持续 loop，因为 verification loop 会推动自我改善。

Like part of what made the programming language easy to build with Ralph is a programming language can be infinitely verified. You write you write the code in the language, you compile it. If the compiler fails, you go fix the compiler. You run the program. If the program fails, you go fix the compiler. Like it's like it's very very verifiable. 比如改善 CI/CD：研究 codebase、做 change、创建 PR、跑 test、看是否更快，再重复。它可以在 loop 中验证自己的工作。

And I think the lesson in loops engineering is like if you can make a problem very verifiable, you can kind of like treat it like a black box and then have it loop because it will keep improving itself because of the verification loop is already there. Exactly. And so like you can do this with CI/CD is like I I do this every time I'm doing a release. I'm like I'm tired. The CI/CD is slow. 不必来回讨论 design 或不同 approach，只需设定“让 CI 更快”的 goal，并告诉 model 五个步骤：写 code、commit、push、启动 sub-agent 观察 job 完成、返回结果，再决定下一步。这就是最简单的 loop design。主持人：Claude Code 和 Codex 都有 goal，只需设定目标，它就会迭代直到达到或持续取得进展。

Cool. Go research the codebase, make a change, make a pull request, run the test, see if it's faster, try again.

Run the p run the test. push push to the branch check again see if it's faster and so it's like if it can verify its own work in a loop instead of design instead of saying let's try this approach or let's try that approach or suggest and being really back and forth you just say like my goal is to make CI faster and you tell the model here's the steps here's the five here's the five steps you're going to write some code you're going to commit it you're going to push it you're going to launch a sub agent to watch the job until it's finished it's going to tell you what happened then you're going to decide what to do next and so that's like the very simplest example I have of like designing loops and you just set the goal which is cloud code and and I think codecs have both chip/go goal which is you just set the goal and it iterates until it reaches it or or as long as it makes progress towards it. Dex：对。 Auto Research 也是这样：“让 model 快两倍”，prompt 让 model 一遍遍尝试直到得到好结果。这就是 loop engineering。我们还做一种很有意思的 loops。挑战在于，人们很容易对“构建用于构建事物的事物”感到兴奋，甚至再多嵌套一层。于是有人说需要把一切重做成庞大的 agentic-first factory，甚至 dark factory，并按未来五年 infrastructure 重新设计全部系统。 engineering，尤其 pragmatic engineering 教给我们的，是如何更 incremental、更 continuous。

Exactly. And so it's like if it's verifiable if you can measure this is auto research too. Auto research is like hey go make this model twice as fast and like it's just a prompt that tells the model to like go to it over and over again and try things until it actually has good results. So that's what I think of loops engineering. I don't know. 很多人不能让 Ralph loop 跑三天、修复 codebase 每个 lint error，再拿出 60,000 行 PR 问谁愿意 review、批准 merge 和 deploy，并保证没有 bug。没人愿意。所以我最兴奋的是 iterated loops 或 slow loops：用 cron job，每晚在 GitHub Actions 中运行一个简单 loop。

We do a very interesting kind of loops engineering where like the the challenge is like I think it's very easy to get very excited about building the thing that builds the thing or building the thing that builds the thing that builds the thing we talked about. 结构就是运行 linter、只修一件事、commit 并 push。每天早上醒来看到一个让 codebase 稍微变好的 PR。

Uh and so people say, "Oh, we need to like redo everything as this big like aentic first factory, maybe even a dark factory." And they're like redesigning their entire thing to be their infrastructure for the next 5 years. And I'm sure one thing we know of in engineering uh and especially uh pragmatic engineering is uh how can you make this more incremental? How can you make it more continuous? 主持人：我喜欢 slow loops。 Dex：它有两个 dimensions。现在我们有 blueprint，Kyle 也刚发布 skill，大家可以自己构建。可以添加更多 feedback mechanisms，比如 frontend 用 React Doctor。

Uh and a lot of people don't have the option to just hey I ran a Ralph loop for 3 days and it fixed every line error in our codebase. Here's a 60,000line PR. Who wants to review it and who wants to sign off on merging and deploying it and uh that there's not going to be any bugs? Nobody. So I think the the thing I'm most excited is actually like what we call like iterated loops or like slow loops where we basically have a cron job. 另一个 anti-pattern 没有 deterministic tooling，Kyle 只是说明“good 是什么、bad 是什么，修一件再回来”。例如 prop narrowing：很多 optional props 实际无需 optional，于是明确怎样改为 required，让 code 更干净、更容易 reasoning。

We have the loop the the the structure of the loop is really easy. It's like run this llinter fix one thing commit and push and then we run that every night in our GitHub actions and we wake up every morning to one PR that makes the codebase a little bit better. I I like the slow loops. Yeah. And it has two dimensions. 你可以添加更多“只修一件”的 conditions，于是每天从一个 PR 变成四个独立 PR。另一个 dimension 是，随着 confidence 增加… …扩大 scope，从修一件变成四件。

So you can add now we have a blueprint for it and actually Kyle just shipped a skill so that you can build these yourself. you can add more like feedback mechanisms. So, we have React Doctor for the front end. We have another anti-attern that has no deterministic tooling, but Kyle's just like, "Here's what good looks like. Here's what bad looks like. Go fix one thing and bring it back." It's like prop narrowing basically. loop 也可理解为由非人工 trigger 启动：Sentry alert、user feedback/support ticket、PM ticket、failing test 或 cron schedule。核心是无需按按钮，有 defined workflow，并让一切逐渐变好。

We have a bunch of optional props and most of them don't need to be optional. It's like here's how to make the prop not optional so that you know that the code just is like cleaner and easier to reason about. And so, you can add more conditions, more things of like fix one thing. I want to wake up to a PR. So, now we wake up to like four PRs because there's four separate things. 广告：Dex 描述了无需人按按钮就让 agents 修问题。但如果 bug 对 agent 和 human 都难以 reproduce，更别说修复呢？这就是 presenting sponsor Antithesis 的用途。

And then the other dimension you can do here is as you gain confidence, you can increase the scope. Instead of fixing one thing, fix four things. And so these are like other ways to think about loops where it's like something that's not a human triggers it to start. 我最近与 Antithesis team 一起 walkthrough，看他们怎样帮助修复 Kubernetes 使用的 open-source key-value store etcd 中一个棘手 bug。

Whether it's, you know, an alert from Sentry, whether it's a user feedback like support ticket, whether it's PM writes a ticket, whether it's a test is failing, any of or it's a cron, it runs on a schedule, but it's like the trigger should be something that you don't have to like press a button on and there's a defined workflow and it makes everything a little bit better. Dex just described letting agents fix things without a human pressing a button. 常规 Antithesis runs 中，linearizability validation assertion 失败。这很严重，因为 linearizability 保证 strong consistency，必须修复。

But what if a bug is too difficult not just for an agent but also for human to reproduce let alone fix? This is where presenting sponsor anticysis comes in. I was recently pairing with the antithesis team where we did a walkthrough of how they helped fix a nasty bug inc the open source key value store used by Kubernetes. This is a bug that actually happened incd. The team noticed that the linearization validation assertion failed during the regular anticis runs. etcd team 在 Antithesis 中运行 causality analysis，生成 bug probability graph：x-axis 是 virtual time，y-axis 是 probability。 virtual time 24 前发生了一件事，让 bug probability 突然大幅上升。深入全部 timelines，向下 vertical lines 代表从同一 state 分支出来的 events，purple dots 是 bug 发生位置。

This is not good because the linearization guarantees strong consistency. So this needs to be fixed. So what the ETCD team did was run a casualty analysis inside anticysis. This generates this graph which is a bug probability graph. Here the x-axis is virtual time and the y-axis is probability. Now we see that something happened just before virtual time 24 that caused a huge jump in the probability that the bug would occur. 仔细看会发现全部 failures 都来自一个 parent branch，是非常有用的 debugging tool。最终 team 用 Antithesis debugging tools 确定 process pauses 导致 bug。

Going deeper, we can look at the entire set of timelines. Vertical lines going down represent events branching off from the same state and the purple dots are where the buck happens. If we look closely enough, we see that all of the failures come from one parent branch. Gotcha. This is such a useful debugging tool. In the end, the team was able to figure out that process pauses were causing the bug using all these anticis debugging tools. 这个 non-deterministic bug 被 deterministic 地诊断。这个真实 bug 后来在 etcd 中得到修复，可以在 GitHub repo 查看 bug 与 fix。 Antithesis 的 debugging tools 很有未来感，也非常强。访问 antithesis.com/pragmatic 了解更多。

This non-deterministic bug was diagnosed in a deterministic way. How cool is that? Oh, and this is an actual bug that then got fixed incd. You can see the bug and the fix in ATCD's GitHub repo. Honestly, the tools that Antys is built for debugging feel pretty darn futuristic, but they are also really powerful. Head over to antithesis.com/pragmatic to learn more. I'd also like to talk about our season sponsor, Sentry. 还要介绍 season sponsor Sentry，我在所有 projects，包括 Pragmatic Engineer backend，都用它做 application monitoring，已有十年，从 Uber 时期开始。

Sentry is a tool I use for application monitoring on all of my projects, including their pragmatic engine back end. I've used it for 10 years now,starting with when I worked at Uber. A neat Sentry feature I'm liking is their SE AI agent, which helps investigate production errors. For example, here's an actual error I had in my application. I can just ask Seir what might be the root cause and it brings context. 我喜欢 Sentry 的 Seer AI agent，它协助调查 production errors。比如这里有一个 application 中的真实 error，我可以直接问 Seer root cause，它会带入 context，也能从 web interface 制定修复 plan。 Seer 在 Slack 中也很好用。

And it can also make a plan to fix it right from the web interface. And a nice thing is how Seir also works great from Slack as well, not just from the web. One place I find even more handy to use Sentry is from codeex or clock code using Sentry MCP. Also, you can set up neat automations like when a resolve sentry issue resurfaces. You can kick off a cursor agent or GitHub copilot agent to investigate the regression, read the relevant code, and open a PR with a suggested fix. 更方便的是从 Codex 或 Claude Code 通过 Sentry MCP 使用。还可以设置 automation：当已 resolved 的 Sentry issue… …再次出现时，启动 Cursor agent 或 GitHub Copilot agent 调查 regression、读取 relevant code，再创建 suggested fix PR。

I'm not a fan for using AI tools just for the sake of it, but I really like the practical integrations where I can fix errors faster and with more context. Check out Sentry at centry.io/pragmatic and start monitoring and fixing regressions today. And with this, let's get back to Dex and to Agentic loops that trigger themselves. 我不喜欢为使用 AI 而使用 AI，但喜欢这种能以更多 context 更快修复 errors 的 practical integration。访问 sentry.io/pragmatic。

Now, you said we can get more ambitious and we can add more things to it, but I'm I'm going to quote you with uh with one of your tweets which says, "This may surprise you that this is coming from me, but I think we're in for a 1 to three year period where stuff might break at 3:00 a.m. and you're relying on loops to fix it and nobody understands what's under the hood, and you're looking at an ex existential threat to your company." Yes. Uh yeah, that one was great. 下面回到 Dex 与 self-triggered agentic loops。主持人：你说可以更有雄心、加入更多事情，但我要引用你的 tweet：“这话从我口中说出也许令人意外，但未来一到三年，可能凌晨三点系统坏了，你依赖 loops 修复，底层却无人理解，最终公司面临 existential threat。

嘉宾 That one did a lot of numbers. Uh ”Dex：对，那条 tweet 传播很广。

主持人 it resonated. Here's the other side of it is like I think that the today with today's models, today's programming languages, today's infrastructure, you might get away with not reading the code. Problem with loops is like at a certain point you're going to generate so much code that you can't read it anymore. This is the strong VM dark factory. This is like Ryan Leopo's like harness engineering. Just spend as many tokens as possible. We tried this. 另一面是，以今天的 models、programming languages 和 infrastructure，也许可以暂时不读 code。 loop 的问题是，最终会生成多到无法阅读的 code。这就是 strong-form dark factory，也是“尽可能花 tokens”的 harness engineering。

We built a lights off software factory in July of 2025 and by November we had shut it down. I think it takes about three to six months of you shipping all the time with nobody reading the code before you realize like, wow, this is getting way worse and it's easier to start over than it is to fix it. Like the models have made the codebase so bad that it is actually going to be easier to just like rethink this from scratch. 我们试过。 2025 年 7 月建立 lights-out software factory… …到 11 月就关闭了。我认为持续 shipping 且无人读 code 三到六个月后，就会发现情况变得糟糕到重做比修复容易。

And maybe that's okay because we have AI and it's easier to rebuild things from nothing. And like usually when engineers say like, "Oh, we can't fix this. We have to rebuild it." The feedback is like, "No, just refactor in place. Just constantly keep the codebase getting better." You mentioned what I said. You'll notice what I said was not use loops to ship the features that users want. models 把 codebase 变得太差，只能从头重新思考。也许 AI 让重建更容易，所以还可以接受。过去 engineers 说“无法修，只能重建”时，通常会被要求原地 refactor，持续改善 codebase。注意我说的不是用 loops 发布 users 想要的 features。

We use loops to actually improve the codebase quality and we read all the code because we care about how it's architected and we care not just about the system architecture but what I would call the program design which I think is something people are going to where are the interfaces where are the seams how are we doing dependency injection all of these things that like make your codebase more maintainable over time and keep you from falling into this trap of like okay well now if I change something over here I broke something over here. 我们用 loops 改善 codebase quality，而且会阅读全部 code，因为在意 architecture。除了 system architecture，还要关心 program design：interfaces 在哪里、seams 在哪里、怎样做 dependency injection；这些决定 codebase 长期 maintainability。

This is the classic problem of software engineering that like software engineering was invented in the 1970s because we realized we needed techniques for avoiding that problem of like this giant ball of spaghetti. 它们防止修改一处却破坏另一处。这是 software engineering 的经典问题；70 年代之所以出现这门学科，就是为了避免 giant spaghetti ball。

And I don't think the models are smart enough and I don't think we actually have the training and the benchmarking and the eval techniques to get models to write code that is more maintainable over time versus they're all trained on SWE and SWEBench looking things, right? All of the benchmarks are basically like here's a commit in Django. Here's an issue that was filed around that time. see if you can create the fix that the human created. 我不认为 models 足够聪明，也不认为目前 training、benchmarking 与 eval 技术足以让 models 编写长期更 maintainable 的 code。它们都在 SWE-bench 类任务上训练。

And it's Django and it's Apache and it's there's a hundred repos in Go and C++ and Typescript and Java and all these different languages, but they're all it's like the problem with training models on maintainability is like the cost function of bad architecture and bad program design can't be evaluated by running the unit test because it hits you 3 to 6 months later when you're like, "Holy crap, like no one can make it's this software has become so hard to change." Is this not similar to how senior software engineers why it took years for someone to become a senior? benchmarks 基本是：这里有 Django 某个 commit，以及当时 filed 的 issue，看看能否重建 human 的 fix。 repos 也包括 Apache、Go、C++、TypeScript、Java 等，但训练 maintainability 的问题在于… … bad architecture 与 bad program design 的 cost function 无法通过 unit tests 评估。

Because typically and in some environments you became a you can become a senior faster typically fast moving where there's a bunch of issues and you have to keep fixing it. Sometimes you know some people are working in the same place for 10 years and they're still not that level. 三到六个月后才会发现 software 已难以改变。主持人：这是否类似 senior software engineer 需要多年成长？

The point was it it just takes time for you to understand the m the small mistake that you make right now that snowballs into like something disastrous later and you get hit by it and you realize like okay things like you know like testing matters architecture matters tech depth can actually be a killer you know we don't talk about it anymore but we used to talk about how techdub kills or slows down companies so badly preai that their competitors can overtake them or they're just like stuck with a 2-year refactor not shipping any new features and the competition you shifts a bunch bunch of other stuff and now they're ahead. 人必须经历小错误如何滚雪球成灾难，才理解 testing、architecture 和 tech debt 为什么重要。有人在同一地方十年也未达到这个 level。关键是需要时间理解现在的小错误怎样演变成后来的灾难。 pre-AI 时代常说 tech debt 会杀死或严重拖慢 companies：竞争对手超越你，或者你被困在两年 refactor 中无法发布 new features，而竞争对手持续前进。

And I I will say like it is possible that GPT7 will fix this, but if you are turning the lights off in your software factory and you're saying like, "Hey, you know what? Like we're not going to read the code. It's fine. The models are smart enough. Dex：GPT-7 也许会解决，但如果现在就关掉 software factory 的灯，说无需读 code，models 已够聪明，只要给 feedback、投入足够 tokens 就会变好，那么…

If we give it the right feedback and just throw enough tokens at the problem, it will keep getting better." This is what led to this tweet. that might work, but if nobody read the code in three months and you replace all of your all of your like code review with loops of like, hey, if a user complains, we give it to an agent. If something crashes, we give it to an agent. If a if a PM writes a ticket, we give it to an agent. …这正是那条 tweet 的背景。也许暂时有效，但如果三个月无人读 code，并把 code review 全替换成 loops：user complaint 给 agent、crash 给 agent、PM ticket 给 agent、CEO 在 Slack 写一篇烦人的长文也给 agent；随后。

If a CEO writes an obnoxious essay about what we should be building in Slack, we give it to an agent. Yeah. And then you stop reading the code because that's going to produce way too much. Like, no one can read it. And like the the the PR reviews become the bottleneck. So, you replace that with aentic testing and agentic uh agentic code review. Uh, but none of these things have intuition for software architecture because we haven't trained it in yet. 因为产出太多，彻底停止读 code… … PR review 成为 bottleneck，于是再用 agentic testing 和 agentic code review 替代。

And so you're going to wake up one day and you're going to have an issue with this happened to us and like we got through it and at the time like it was still worth it. It was like spent 3 weeks onboarding back into the codebase that we had stopped reading 3 months ago because no matter how much sophisticating expert prompting we could not get Opus, I think it was Opus 4.1 at the time. We could not get Opus 4.1 to actually find the root cause. 可这些系统缺少 software architecture intuition，因为我们还没有训练出来。有一天会遇到严重 issue。我们真的遇到过：花三周重新 onboard 到三个月没读的 codebase，因为无论使用多复杂的 expert prompting，都无法让当时的 Opus 4.1… …找到 root cause。

We had to go spend several days digging through the code and figuring out like, oh, there's just actually a primary key that's being routed through this whole thing that needs to be changed to a different type of object and it needs it. This actually happened to you. This happened to us. Yeah. And when it happened, I was like, you know what? That sucked. That was terrible. But we did it. We solved it. 我们只能花几天深入 code，最终发现有一个 primary key 在整个系统中传递，却需要改成不同 object type。主持人：这真的发生在你身上？ Dex：对。当时我觉得虽然糟糕，但解决了；多数时候不读 code，偶尔花两周人工修一个问题，仍值得。

And uh it's still worth it's still worth not reading the code for most of the time at the cost of every once in a while I'm going to have to spend two weeks fixing an issue by hand. And I don't believe that anymore because I think the amount of code we're able to write now is actually like 10xed or 100xed and I think the problem's just getting worse. So let's talk about software factories. 现在我不再相信，因为 code volume 已提高 10 倍甚至 100 倍，问题只会更严重。主持人：那就谈 software factories。这个词 overloaded。 AI 前后分别是什么意思？ Dex：你知道 software factory 最早何时被定义吗？

Yeah. In your mind, cuz I feel it's an overloaded word, but what do you think of a software factory before AI and now post AI? Do you know the first definition of software factory the first time it was used? No. It was a NATO conference in 1968. Oh, Grady Buch would know about this. Yeah, exactly. Yeah, great. You should ask Grady about it. They talked about the idea of like, okay, you actually need to build a system of steps and like just like a factory floor. 主持人：不知道。 Dex：1968 年 NATO conference。主持人：Grady Booch 会知道。 Dex：对，可以问他。当时提出，需要建立一套像 factory floor 的 steps：coding、testing、validation、integration。

You have like the coding part and the testing part and the validation part and the integration part. 那时没有 CI/CD，甚至几乎没有 version control，但人们已经认为需要 factory。

We had no CI/CD. we barely had version control like but you needed a factory and then it was adopted by like um Toshiba and a bunch of companies and then the the next moment was like DevOps and you have like this idea of like okay we're going to do CI/CD we're going to automate we're chef and anible puppet whatever all these technologies is like instead of having dudes running around data centers like resizing discs and stuff or clicking around the AWS console yeah exactly it was like cool we build loops the server hits 90% disc space that sends an alert to Nagios Nagios triggers a chef front chefs makes the disc the disc bigger feedback loops, right? 后来 Toshiba 等 companies 采用。下一个时刻是 DevOps：引入 CI/CD 和 automation，以及 Chef、Ansible、Puppet 等 technologies。不再让人跑进 data center 调整 disks，或在 AWS console 到处点击，而是构建 feedback loops：server disk space 达到 90%，Nagios 触发 alert，再触发 Chef run 把 disk 扩大。

This has been around for a while. And in 2018, I want to say this guy Nick Chalane who was uh he was like the CTO or chie ch ch ch ch ch ch ch ch ch ch ch ch ch chief software officer of the air force, he wrote this 100page essay of hey the DoD needs a software factory,the department of defense. Yeah, the the department of defense and the air force. And he called it the dev sec ops factory. And he said we need all the things that all of the good enterprises are using. 这些 loops 已存在很久。大约 2018 年，Air Force 的 Chief Software Officer Nicolas Chaillan 写过一篇 100 页 essay… …说 DoD 和 Air Force 需要 software factory，称为 DevSecOps factory。

We need Jenkins. We need like code quality scanning. We need security scanning. We need CI/CD. We need to be able to ship. We're shipping once every three months or once a year. We need to be able to ship every day like all these other companies. 他指出需要优秀 enterprises 使用的一切：Jenkins、code-quality scanning、security scanning、CI/CD。

And the way we do that is we actually embrace all these automations and technologies so that engineers are are 90% of the issues are caught by automations instead of people actually like manually checking it or manually reading the code or manually integrating modules together. Wow. Talk about forward thinking in in the government. DoD 当时三个月甚至一年才发布一次，需要像其他 companies 一样每天发布。

I know. Oh no, as I was surprised like oh nice like this is I mean and that was part of it is like hey look we're falling behind in like you know I don't know exactly all the reason but I I imagine also about like attracting really good talent is like hey look if we have like the modern software stack and we're building things fast and we care about efficiency and we care about people's using people's time well we care about them spending time on the hard parts of the job not manually looking for SQL injections like you could automate that. 方法是拥抱 automations 和 technologies，让 90% issues 由 automation 发现，而不是人工检查、读 code 或集成 modules。主持人：government 竟如此 forward-thinking。 Dex：这部分原因大概也包括吸引优秀 talent：拥有 modern software stack、快速构建、重视 efficiency，把人的时间用在 hard parts，而不是手动寻找可以自动检测的 SQL injection。

So this was software factories pre AAI. Pre AI. Now I've heard the term a lot more because of AI. Yeah. Is it the same? Is it different? So this is really hard to say without a drawing, but I'll try to draw it out. At the core of a software factory, you have like a source of work. Most you you can imagine a linear a Jira a the st source of truth your object whether it's a spreadsheet or whatever is you have like what stages is the work in. 主持人：这是 pre-AI software factory。如今 AI 让这个词更常见，它还是同一件事吗？ Dex：没有 drawing 很难讲，我试着描述。 software factory 核心有一个 source of work。

Yep. And prei you would take, you know, you would maybe do some architecture review planning. You would maybe do some sprint planning and then people would take tickets off the queue and they would go build them. 可以想象 Linear、Jira、spreadsheet 等 source of truth，其中记录 work 处于哪个 stage。

And then you would make a pull request and people would review it and you would run CI checks and then you would send it to prod and then it would make contact with your users and your users would complain about stuff and that would go to your support team and back into your work tracker and it would crash and you would have issues and that would go into your monitoring stack and that would go into your tracker and that was your loop. pre-AI 时可能先做 architecture review、planning、sprint planning；people 从 queue 取 ticket，构建、创建 PR，由 people review、运行 CI checks、发布 prod；接触 users 后，再收到 complaint。

And then people would take stuff off the tracker based on priorities. product managers, engineering managers, engineers prioritizing work and then we go and do that and the first change is like this long wind lot lots of phases and this is also why when like a developer shifts a bug but by the time it comes back to you it might be two or 3 months or even longer and by the time it get fixed it might be a year or two and you know this is why when you're using a piece of software it's like that annoying bug and you talk with customer support but it's just a very like long latencies at each each part of the the factory if you will. complaints 进入 support team，再回到 work tracker；crashes 和 issues 进入 monitoring stack，再进入 tracker，形成 loop。然后 people 根据 priorities 从 tracker 取工作，由 product managers、engineering managers、engineers 排序并执行。

Yeah. And the the step where someone pulls a work item off a queue and starts working on it is, you know, couple hours to a couple days before it actually gets integrated into everything else and touches user. And that's in a in a in a great world, right? Sometimes you go build it and then you merge it and then it actually gets released 3 months later. 这条链路很长、phase 很多，所以 developer 发布 bug 后，两三个月甚至更久才重新看到。真正修复可能要一两年。使用 software 时会遇到某个 annoying bug，与 customer support 沟通，但 factory 每个环节都有很长 latency。

But we're going to assume we're in a fairly modern like we're somewhere like the a Netflix or a meta where engineers are capable of shipping 100 times a day or a thousand times a day, but it still takes 2 three hours to do the work. And now with an identic factory, what you do is you take out that person building the thing and you replace it with an agent building the thing. And so you have orchestration to trigger things. 有人从 queue 取 item 并开始工作，通常要几小时到几天才能 integrate 并触达 user，这还是理想情况；有时 build 和 merge 后三个月才 release。

You have a sandbox, you have an LLM, you have an inner harness, you have an outer harness, which is like the dev environment you build for the agent. And maybe you give it a browser, you give it a video recorder if you use like things like cursor background agents. They've kind of built this outer harness around the inner harness that is the coding agent. 假设是在 Netflix 或 Meta 这类现代环境，engineers 每天能发布 100 或 1,000 次，但完成工作仍要两三小时。 agentic factory 会移除 building 的 person，换成 agent。

And then you make PRs with that. problem there is that like okay now now it takes 10 minutes to do a build instead of two hours or two days and so now the bottleneck is code review so okay let's throw a bunch of AI agents at code review and let's do agentic testing so that like we can basically catch a lot of the easy stuff and humans are only focused on the most like important critical core parts of the codebase and then the next level up of your agentic factory is you do the top it's like okay then it gets deployed it goes to prod and a user complains you just hook your support queue right up to the agent someone complains about something agent tries to fix it and instead of looking at a ticket and then saying okay go send you just close that loop and instead every time something goes wrong you just get a PR and then every time something crashes in Sentry or Data Dog or whatever it goes into the tracker it gets picked up by an agent and you get a PR this is the ramp inspect thing this is the the only difference is like then you have so much code to review and people say well let's try turning the lights off let's just take all the human testing and review steps out and we'll say okay cool if users complain then it's broken and if users don't complain and it's working and we're not going to read the code. 系统需要 orchestration trigger、sandbox、LLM、inner harness 和 outer harness，也就是为 agent 构建的 dev environment。也许还提供 browser、video recorder。 Cursor background agents 就是在 coding agent 这个 inner harness 外构建 outer harness，再产出 PR。问题是 build 从两小时或两天缩短到十分钟后，bottleneck 转到 code review；于是用更多 AI agents 做 review，并做 agentic testing，捕获简单问题，让 humans 只关注最重要的 core code。 agentic factory 下一层是部署 prod 后直接连接 support queue。 user complaint 触发 agent 尝试修复，不再只是查看 ticket，而是关闭 loop：每当出错就得到 PR；每当 Sentry、Datadog 等 crash 进入 tracker，就被 agent 接手并产出 PR。

We're going to use we're going to treat the whole system as a black box. So, you said you tried this out uh when it was like Opus Formula and you you built the software factory was running beautifully until it just blew up on your faces. How do you think of this model? cuz I I can see an ideal world where it works, but clearly we're not in an ideal world. 这类似 Ramp Inspect。唯一差别是，code 多到无法 review。

Like where do you think we are like and could some of this actually work at some point or you know like like what what progress are you seeing right now and and what is the the today the situation like how much of this do you believe we can automate or should we automate? Yep. So if you know me, you follow my stuff, you know I stand for three things. 于是有人说关灯吧，去掉全部 human testing 与 review，以 user 是否 complain 作为是否 broken 的判断，不再读 code，把整个 system 当作 black box。主持人：你在 Opus 4.1 时试过，factory 一开始运行很好，后来爆炸。

Number one is like cutting through the hype and the jargon and going trying things and talking to people who are using things and figuring out which parts of this actually work and are valuable. Number two, we talked about words. I try to find and protect useful bits of language because I think it helps us all move forward. 理想世界里似乎可行，但现在显然不是。我们现在处在哪？未来可能实现吗？今天应该自动化多少？ Dex：了解我的人知道，我坚持三件事。第一，穿透 hype 和 jargon，真正试用，并与实际使用者交流，找出哪些部分有效且有价值。

And when you take a useful word like agents or you take a useful word like software factory and then you semantically diffuse it, this is another Martin Fowler word. You make it mean everybody likes the word and it all becomes hype and everyone starts agents means nothing anymore. agents could be a chatbot, it could be a Slackbot, it could be a coding agent, it could be tools in a loop, whatever it is. 第二，我们谈过 words。我试图发现并保护有用 language，因为它帮助大家前进。当 agents、software factory 这类词发生 Martin Fowler 所说的 semantic diffusion，所有人都喜欢这个词、把它变成 hype，最后 agent 就什么都能指。

So, I like to protect important useful words and like help help us all like elevate the conversation out of that hype and jargon. And then I care a lot about going one level down beneath where I'm generally working. 它可以是 chatbot、Slackbot、coding agent，也可以是 tools in a loop。

I think there's always this is the same thing with context engineering is like I was rarely actually going and like building LLMs or understanding or training LLMs but knowing how they're trained how transformers works informs how you build at one layer up and for the software factory my version of that is I spent the last couple weeks going really deep on uh reinforcement learning with uh verifiable rewards RLVR which is like this very productionized like it's not like RH RHF is still like fairly academic and pure RLVR are is this like it's a machine in these labs of how we train these models and I'm studying like the benchmarks for coding agents and the techniques for training them and how we like give it a small problem have it solve it delete the test changes it made revert them apply a test patch see if it passed and then even the frontier this year we have like we can get into this later but like frontier code and marathon these new benchmarks that are supposed to be like better at evaluating models's ability to maintain a codebase over time and write maintainable code um and they are better But I don't think they're sufficient. 我希望保护重要、有用的 words，让讨论摆脱 hype 和 jargon。第三，我很重视深入自己工作层级下面一层。这与 context engineering 相同：我很少真正构建或训练 LLM，但理解训练方法和 transformer 原理，会指导上一层如何构建。对于 software factory，我最近几周深入研究 reinforcement learning with verifiable rewards，也就是 RLVR。它已经是 labs 中训练 models 的 production machine，不像 RLHF 那样仍带较强 academic 色彩。我研究 coding-agent benchmarks、training techniques，以及怎样给 model 一个小 problem。

But it's basically this idea that like the only thing that made claude code good was reinforcement learning. And the dimension along which it got good was like we made a model. We trained the model and the harness together. And so the model got really good at calling the specific tools in that harness. Really good at reading files, writing files, searching for files, all this stuff through doing these problems. 让它解决，删除其 test changes、revert，再应用 test patch 检查是否通过。今年还有 FrontierCode、Marathon 等新 benchmarks，试图更好评估 model 长期 maintain codebase 和写 maintainable code 的能力。

And that was what made it feel so much better than all the other CLI coding agents that came before it. And so people like, "Okay, that was so much better." And they're just going to keep getting better. But it's like it got really good in one dimension. And the dimension that they're not getting better in because it's hard, expensive. 它们更好，但仍不充分。 Claude Code 之所以变好，主要来自 reinforcement learning。

Maybe we need to like get a lot more creative with how we design these these verifiers and benchmarks is in how do I make code that in three months is going to like improve the productivity of humans and agents, mostly agents, but humans and agents in the codebase instead of making it worse over time. And so you think that part is just missing? We haven't seen too much improvement. 它变强的 dimension 是 model 与 harness 一起训练，因此非常擅长调用该 harness 的特定 tools，包括 reading、writing、searching files。这使它比之前 CLI coding agents 好得多。人们因而相信它会持续全面提升，但实际上它只在一个 dimension 变强。

I haven't seen obviously no one knows what the labs are doing internally cuz it's all very secret. But I think if we looking at where the bench the benchmarks tend to reflect where the labs are, right? If there is no benchmark that can convey to me did this model write code that is going to make my codebase better or worse. The best we have is I I think frontier code from the cognition team is really interesting. 尚未变强、也很难且昂贵的 dimension 是：怎样写出三个月后仍能提高 humans 和 agents productivity、而不是持续恶化 codebase 的 code。主持人：你认为这部分仍缺失，没有太多改善？ Dex：没人知道 labs 内部的秘密工作。

They have like did the test pass and then they have like two layers of model review. So they have a judge model that checks okay is the patch the model made similar to the patch that is like the golden answer set. So even if the model didn't write the exact code that the benchmark was expecting did was it functionally equivalent and the next one is like a like code quality review from another judge model and like that's better but it's not it's not sufficient. 但 benchmarks 通常反映 labs 的方向。如果没有 benchmark 能告诉我 model 写的 code 会让 codebase 变好还是变坏，就说明能力缺口仍在。 Cognition team 的 FrontierCode 很有意思：先检查 tests，再用两层 model review。

And this is why I also think agentic code review is like yes it will catch things and it will raise your floor but I don't believe like the model writing the code is the same model reading the code and if you ask a model hey is this code good it's going to be like oh yeah it's great comprehensive it's got unit tests you've tried this I'm sure and you say okay review this PR that my coworker wrote and tell me everything that's wrong with it I was like oh it has this problem and this problem and this is sickopantic and they want to tell you what you want to hear and so like it's really hard for me to trust a model to evaluate the quality of of of code that's written And so I I I have some ideas on like, okay, can you build a benchmark where the model builds 20 features in a row and maintains the codebase the whole time and it doesn't know what features are coming. 第一层 judge model 检查 patch 是否与 golden answer patch 相似。即使 code 不完全相同，也判断是否 functionally equivalent；下一层让另一个 judge model 做 code-quality review。这样更好，但不够。这也是我对 agentic code review 保留意见的原因：它能发现问题、提高 floor，但让同一种 model 写 code 又读 code… …你问“这 code 好吗”，它会说“非常全面，还有 unit tests”；如果说“review coworker 的 PR，找出全部问题”，它又会列出问题。

You treat it like a real product team where you don't know what you're going to build next week until you get there and you find out what's most important and then can we try to evaluate like can we build a problem like that that's hard enough that most frontier models fail by issue six or seven. models 带有 sycophancy，想告诉你想听的答案，所以很难信任它评估 code quality。

Is it fair to say that you know like we've had the software factory like before AI it was just like lots of loop it was like the the PM giving the ticket to the dev the dev building it deploying to production user customers using it customer support getting tickets and then you creating PM triaging and it kind of goes around like in this loop is it fair to say that the software factory of how a company a team builds and maintains software that is changing because now everyone's replacing some parts of it, you know, maybe the the least advanced teams will just be devs are starting to use cloud code or codecs to write faster. 我设想一种 benchmark：让 model 连续构建 20 个 features，并一直 maintain codebase，而且不知道下周 feature 是什么。像真实 product team 一样，直到下周才知道最重要的事。能否设计一个足够难的问题，让多数 frontier models 在第六或第七个 issue 就失败？主持人：pre-AI software factory 由 PM、dev、prod、users、support 与 triage 组成 loop。

They're not spending as much time on there. Some others are also having the deployment the feedback. Some some actually have the agents already oneshotting bucks. So like is it fair to say that that the software factory is just is just changing everywhere maybe at different speeds but everyone I think every team who is building production software they're like they're frantically experimenting trying and everyone's at a different pace. 如今每个 team 都在替换其中某些部分。最初级是 developers 用 Claude Code 或 Codex 写得更快；有些已连接 deployment 和 feedback；另一些让 agents one-shot bugs。

You'll have the AI native starters where most of this will have agents in them and you'll have the laggers who are or more more cautious ones. They have agents in a few places but not in the others. Well, and I think that's the key is like if you want to do loops engineering, you should build one loop at a time and you should keep them small and contained. Basically, I think everything except stop reading the code is really good advice. 可以说每个 production software team 的 factory 都在变化，只是速度不同吗？ AI-native startups 会在多数环节使用 agents，谨慎或落后的 teams 只在少数环节使用。 Dex：关键是，做 loop engineering 应一次只构建一个 loop，并保持 small、contained。

Take support tickets and turn them into tickets in your system and then maybe turn those into PRs. Great. The advice that I have and like what we kind of like are chasing at human layer is like how can I add another checkpoint in that factory? So instead of having one human re view point where you're reviewing PRs and sometimes they're 100 lines and sometimes they're a thousand lines but it's quite a lot of effort for especially if it's bad especially if it needs rework. 除了“停止读 code”以外，其他 advice 大多很好。把 support tickets 变成 system tickets，再变成 PRs，都很好。 HumanLayer 追求的是怎样在 factory 中添加另一个 checkpoint。目前只有 PR review 这一个 human review point。

主持人 It's quite a lot of effort for a human to be like okay this is wrong go change it in this way and then you loop back to the agent and then you come with another one and like doing a lot of loops on there once once the direction has been committed to it's really hard to steer off like you're better off just kind of restarting from scratch. How do you build like controls and mechanisms around that? PR 有时 100 行、有时 1,000 行，尤其 code 很差、需要 rework 时，人要指出错误和修改方向，再循环给 agent，成本很高。一旦方向已经被 committed，就很难 steering，甚至重启比修正更好。

And then my take is like if you do a little bit of human agent planning and like discussion before you hand it to the impletor whether it's I mean planning and specs whatever you want to call it again this is spec driven development is another word that has become kind of very like muddled as far as what it means but basically how can we spend an hour before we start building so that the PR when we read it only takes 20 minutes because the code is perfect instead of not touching it just literally saying every user reported issue becomes a PR through the loop and then we read that PR and it takes six hours because there's back and forth and we have to make changes and things. 怎样围绕它构建 controls 和 mechanisms？我的观点是，在交给 implementer 之前，human 与 agent 先做 planning 和 discussion。无论叫 planning、spec 还是 spec-driven development… …这个词的含义也变得混乱。核心是，能否在 building 前花一小时，让之后 review PR 只需 20 分钟，因为 code 已基本正确；而不是每个 user issue 都直接 loop 成 PR，最后 review 六小时、反复修改。

It's all I'm all about like let's find leverage. And so you basically you have three options in the software factory world. If you're going to go all in on aentic software factories, you can turn the lights off and just let everything flow and pray that you don't create too much slop and pray that the next generation of models comes fast enough before you create a giant pile of ash. you can slow way down and read every PR and read every line of code. 我的目标是寻找 leverage。 software factory 有三种选择：第一，完全投入 agentic factory、关灯让一切自动流动，祈祷不要生成太多 slop，也祈祷 next-generation models 在代码堆变成灰烬前及时到来。

Uh, and then you're only going to really get modest benefits from AI because that becomes I I think you should expect maybe 30 to 50% lift in productivity is kind of what I see when we go into teams or you can find the right leverage points where humans can actually an hour spent over here in planning can save you four hours in in implementation in terms of fixing and going back and and getting the design right. And that's what I call like seeking leverage. 第二，大幅减速，读每个 PR、每行 code。这样只能获得 modest AI benefits。根据我们进入 teams 后的观察，productivity lift 也许是 30%–50%。

If you can find the right leverage points for the agents to guide the work, then you can actually move like two to three times faster while maintaining a like 99% like accuracy to like if the humans were carefully writing this code by hand, how would it come out? Now jumping a little bit back to ideas. I will come back to this. This was earlier maybe it was last year but you had the research plan implement. 第三，寻找正确 leverage points，让 human 在 planning 上花一小时，节省 implementation 中四小时 fix 和 back-and-forth。如果找到 human 引导 agent work 的正确 leverage points，就能在维持接近 human 仔细手写 code 的 99% accuracy 时，速度提高两到三倍。

Can we talk about the original research plan implement framework and then also what you've learned about this approach? what you got wrong about it. Yeah, sure. Yeah. So, um I mean the first time we talked about RPI was in August of 2025. Um and it was basically like the research was this thing of like, hey, before you go build anything,go read lots and lots of code. Use a bunch of sub aent sub aents in parallel, understand all the code. 主持人：稍微跳回 ideas。去年你提出 research-plan-implement，可以谈原始 framework、后来的经验和错误吗？ Dex：第一次谈 RPI 是 2025 年 8 月。 research 指在构建前阅读大量 code，使用许多 parallel sub-agents 理解全部 code。

It was this technique that like worked really well for hard problems in complex code bases. You just ask Claude uh to do a thing that that's it would read three files and make a change. It would have no context. So, you start the research. You don't even tell it what you're working on. 这项 technique 对 complex codebase 中的 hard problems 很有效。

You just tell it, "Hey, can you tell me how this system works and this system and how they connect together and then you get a markdown dock out and this is the context engineering part is like that would take a 100,000 tokens of context, but you would get a 10k token dock out of it that summarized it. 过去直接让 Claude 做事，它只读三个 files 就修改，缺乏 context。 research 阶段甚至不告诉它要构建什么，只问“这个 system 如何工作，另一个如何工作，两者怎样连接”，最后得到 Markdown doc。

Then you would start a new context window and you would do planning and the planning would be and actually realize like the plans that we were building last summer were actually terrible. But it would basically be this long. You would say, "Okay, now here's what we're building. Here's the research doc. build a plan to implement it. 这就是 context engineering。 100,000-token context 被压缩成 10K-token doc。

And uh in retrospect, now that we see like everyone is obsessed with how do I get agents to work for longer, I think the reason why in like May, June, July, August of 2025 that a lot of people became really interested in planning was it was a very powerful lever to get agents to work for longer. If you said, "Build me a B2B SAS for uh burrito delivery," you'd get like a homepage and that's it. But if you said, "Build me a plan," it would build out this big plan. 再开新的 context window 做 planning：“这是要构建的东西，这是 research doc，请制定 implementation plan。 ”回头看，去年夏天的 plans 很糟糕；但当下大家都在研究如何让 agents 工作更久。 2025 年 5 到 8 月 planning 流行，是因为它能显著延长 agent 工作。

And then in the next context window, you'd say, "Hey, here's the plan. Here's all the changes we're going to make. Go imple it would actually keep going until the plan was done." So the plan was a really good way to anchor an agent and remind it that like, hey, you're not done until this is all finished. So that was the original RPI. And the plan doc, what was bad about it is it didn't give you leverage. 如果只说“构建 burrito delivery B2B SaaS”，可能只得到 homepage；先要求 plan，会生成完整计划，下一 context 再给 plan 与 changes，agent 会持续做到完成。 plan 是 anchor，提醒 agent 未完成全部内容就不算结束。

The plan was every single line of code that was going to change like in diff blocks and like all the new stuff to write. And so like people would review these plans. We recommended this. We told people to read the plans. We read all our plans. And then eventually I found myself like I just kind of skimmed the plans. And so you're not really using it as a way to resteer the agent. 原始 RPI 就是如此。问题在于 plan 没有创造 leverage：它把每行要改的 code 都写进 diff blocks，列出全部新内容。我们推荐人们读 plans，也确实读了，但后来自己只会 skim，因此无法用于重新 steering agent。

It's just kind of there. And then you go write the code and there's a crap. Some people would review the plans and the code and it's like okay well the plan was took you 20 minutes to read and then the pull request takes you 20 minutes to read and they're different. And so you actually doubled the amount of time you're spending reading code instead of like doing less of it. You've anti- leverage. plan 只是存在那里。接下来写 code，plan 阅读 20 分钟，PR 又读 20 分钟，而且两者不同。结果阅读时间翻倍，反而是 anti-leverage。

And hang on was spec different development not related to this the one that Amazon Kira for example and and GitHub workflows again a year ago did which was it also it first generated a plan and it had the human review it and then it started to and you could edit it as well and then it went off and implement this part and it it looked beautifully on the surface. It it should have worked great but it's tossed into the garbage outside of some m some maintenance projects. 主持人：Amazon Kiro、GitHub workflows 等 spec-driven development 不是也先生成可编辑 plan、human review 后再 implementation 吗？表面上很漂亮，理论上应该奏效，但除了某些 maintenance projects 外似乎被丢弃。我收到的 feedback 是大家停止使用，因为效果不太好。它与原始 RPI 很相似。

I I think it just didn't work. like all all the feedback I got, people just stopped using it because it just didn't really work that well. It just rhymes to the RPI framework a little bit, the original one, right? Well, so our thing too, like the biggest difference between RPI and specri development and some people refer to RPI as specriven dev because for some people SD all it means is I use a bunch of markdown files while I'm coding and forget what's in them. Dex：RPI 与 spec-driven development 最大差异在于… …有些人把 RPI 称为 spec-driven development，因为在他们眼中，SDD 只意味着 coding 时使用一堆 Markdown files，之后忘掉内容。

I just specri those are my specs and I'm using them to drive development. There was this OpenAI researcher who talked about specri dev and like hey stop reading the code just write the specs and treat like the coding part as compiling specs into code. that part never really materialized. Maybe with GPT7, you know. 曾有 OpenAI researcher 提到 SDD：停止读 code，只写 specs，把 coding 当作把 specs compile 成 code。这一直没有真正实现，也许 GPT-7 可以。

Um, but the challen I'm on a GitHub issue in specit uh that has been open for a year and every couple weeks I get there's a new email on the thread of people complaining about this problem of like, okay, I edit my specs and then I edit the code and then the code drifts and the specs how do I keep the specs up to date as the code is changing and it's basically like you now have two sources of truth and it's it stops being useful. 但挑战是，Spec Kit 中有一个 GitHub issue 已开一年，每隔几周就有人抱怨同一问题：编辑 specs，再编辑 code，code 发生漂移后怎样保持 specs 更新？最终产生两个 sources of truth，system 失去作用。所以 RPI docs 虽然一度保留，但两三个月后发现它们只是 tactical execution docs。

And so like that's why when RPI the idea of the docs is they were all for a while we kept them around but after two or three months we're like oh these are actually like tactical execution docs.

I do the research I do the plan I do the implementation I throw the docs out and the next time I need research I just do it from scratch because tokens are cheap and my time is expensive and the amount of time I might waste if I reuse a research that is no longer in sync with the real state of the codebase. So we just create it live every time. This is why it's like context engineering still matters. 做 research、plan、implementation 后就丢掉 docs；下次需要时重新 research，因为 tokens 便宜，而人的时间昂贵。复用已与 codebase 真实状态不同步的 research，反而浪费时间。每次都 live 重建。这也是 context engineering 仍重要的原因。

Creating artifacts that compress the state of the codebase and compress the intent of the builder into small things that can be reused in the future for the scope of a task is like a very powerful like tactical approach, but it's not a thing like I I have very few opinions on like what sorts of docs that you should leave lying around your codebase that are like evergreen. 把 codebase state 与 builder intent 压缩成小 artifacts，在单个 task scope 内复用，是强大的 tactical approach；但我对 codebase 中长期保留哪些 evergreen docs 几乎没有强观点。

I've seen people try to maintain parody between documentation or specs and the code itself and I don't think anyone actually like found it very useful. Like you can do it and it works but it's like the ratio of the effort it takes to keep them up to date and the and trivially you could do this with AI probably but I've never known anyone who was like yeah this is great and we're glad we have it. 有人试图保持 documentation/specs 与 code parity。但我没见过谁真正觉得很有用。技术上可以实现，也许 AI 能自动维护；可保持 specs 与 code 同步的 effort/value ratio 通常不够好，不如始终把 code 作为 source of truth。

Like you could do it and it might help but I I don't think anyone found it useful enough to like maintain a system to keep the specs and the code in sync versus just using the code as the source of truth always.

Now you mentioned something interesting which is with context engineering you need to sometimes compact and you've previously co talked about intentional compaction that when context is noisy deliberately compress the useful part into a clear like markdown artifact verify it and then start a fresh conversation. 主持人：你提到 context noisy 时进行 intentional compaction：把有用部分压缩成清晰 Markdown artifact，verify 后开启 fresh conversation。可以谈谈这种 compaction 及其重要性吗？

Can we talk about this kind of compaction and why it's important and and it sounds like it's going to be a building block where it already is for context engineering, right? Yeah. No, frequent intentional compaction is the building block. It is it is completely comes from context engineering is context engineering is like how do we get the most out of today's models? How do we change what we're putting into the model into the context window into the agentic chat? 它已经是 context engineering 的 building block。 Dex：frequent intentional compaction 就是 building block，完全源自 context engineering：怎样改变传入 model、context window 和 agentic chat 的内容，尽可能发挥 today's models。

How do we control that in such a way that we get the best results possible which means doing as much work as possible in the smart zone the you know first 100,000 tokens of the context window. And uh this intentional frequent intentional compaction is basically like okay the research step we're going to go read a bunch of code and turn it into a doc. That's our compaction. We take that forward in the next session. 目标是尽可能多地在 context window 的 smart zone，也就是最初约 100,000 tokens 中工作。 research step 会阅读大量 code 并转成 doc，这就是 compaction；再把 doc 带入下一 session。

We're going to read we're going to read the ticket and the intent and turn that into a design document that we call is like okay here's the highle spec of what we want to do. Here's a high level like current state desired end state and then a bunch of design questions the model has kind of like a very thorough maybe even overengineered like plan mode. And then you take the research and the design and you do a new session, new context one. You're like, cool. 之后阅读 ticket 与 intent，转成 design document。 design doc 包含 high-level spec、current state、desired end state，以及一组 design questions，model 在其中以很 thorough、甚至 overengineered 的 plan mode 工作。

You you've compressed the intent and you've compressed the state of the codebase so that you can then do your planning of like, okay, we know what the end state looks like. We know where we're going. Now, let's break down how we're going to get there. All of these different steps of the process exist because models have shortcomings in each of these phases. So, the research is pretty hands-off. I don't read the research docs. 然后把 research 与 design 带入新的 session/context。这样已压缩 intent 与 codebase state…

It's just like go read a bunch of code and then like make a doc out of it. Models are pretty damn good at that. If you ask it to find a bug and have opinions about the codebase, that's different. But if you just ask it what is the intent and how do this stuff fit together, uh that's usually pretty straightforward. But designing the end state of the of the software, the architecture and the program design, models are not great at. …接下来可以 planning：知道 end state 和目的地，再拆解路径。 process 的每个 step 都对应 models 在不同 phase 的 shortcomings。 research 通常 hands-off，我不会读 research docs，只让它读大量 code 并生成 doc；models 很擅长这件事。

They make a lot of like they make decisions and sometimes they're right and sometimes they're wrong. So we have want to have a human in the loop there. And then the steps to get there, I we talked about this before, but models love making what I call like horizontal plans. If you ask a model like build a plan of steps to go build this app, it's like cool. 若要求找 bug、评价 codebase 就不同，但单纯解释 intent 和 system 关系通常很直接。但设计 software end state、architecture 和 program design，models 不擅长。

We're going to do the database and then we're going to do the services layer, then we're going to do the API and then we're going to do the front end. It's like, well, that actually kind of sucks because we're going to be on the other side of 2,000 lines of code and let's imagine this is an existing codebase, right? We're going to make changes to all these different parts of the system. I can't test it till the end. 它们会做 decisions，有时对，有时错，所以这里需要 human-in-the-loop。到达目标的 steps 也有问题：models 喜欢所谓 horizontal plans。

And so what I would do is like, okay, how would I have built this if I were building by hand? Well, okay, I would probably create a mock API endpoint with fake data. And then I would go kind of get the front end kind of how I want it to look. And then I would actually go like build a services layer and actually wire the data through. And then I would make a database migration and make my new table. And then I would actually add a lot of business logic. 要求它制定 app build plan，它会先做 database、再 services layer、API、frontend。这很糟，因为完成 2,000 行 code、修改 existing codebase 各部分后，直到最后才能 test。

And then I would add a bunch of error handling. And it's completely orthogonal to how model like models would write the database layer and all the error handling without ever like anyone's ever touched or seen the code or whatever it is. And so this is another place where we like we like to have humans involved because humans have really good taste and judgment. 若由我手写，会先创建带 fake data 的 mock API endpoint，让 frontend 大致呈现预期；再做 services layer、接通 data，随后 database migration 和 new table… …再添加 business logic 和 error handling。

Like I would rather read five separate little mini diffs of like things that I can manually verify and explore than read 2,000 lines of code and be like well it's not working. I don't know where. You don't know where cuz you wrote the code. You were supposed to get it right. We talk about compaction context engineer. It's like how can you stay in the smart zone of the context window which is again the dumb zone. 这与 model 的方式完全正交；model 可能先把 database layer 与所有 error handling 一次写完，却没人实际接触或观察。这里同样需要 humans，因为 humans 有 taste 和 judgment。

I will say disclaimer it's really good training wheels if you don't have intuition about this. So let's just define these things. What is a smart zone and what is a dumb zone? So, it's it's it's a little bit blurriier than like I would like I would like it to be. I think in November we we talked about the first 40% of the context window, but then we had million smart zone. Yeah. Then we had million token context window. 我宁愿读五个可手动 verify、explore 的小 diffs，也不想面对 2,000 行 code，只知道系统不工作。而 model 明明写了 code，却也不知道问题在哪。 context engineering 的 compaction 目标是留在 smart zone。

So then I changed it to like the first 100,000 tokens if it's a really like 4.8 I usually will go up to like 200k. But basically the the thing Jeff Huntley had and Ralph Wickham was like the less context window you use the better outcomes you'll get. dumb zone 更像 training wheels，在缺少 intuition 时很有帮助。主持人：先定义一下 smart zone 和 dumb zone。

And basically the smart smart zone mean meaning if you have context in that first part it should work a lot better and then like the dumb zone is like once you have stuff there it's kind of forget about it like it'll be confused it's not going to do much like it'll degrade. Yeah. And there are times and this is an intuition thing like I will often go up to 3 400k tokens. Dex：边界比我希望的更模糊。 11 月时我们说 context window 前 40%。后来出现 million-token context window，我改成最初 100,000 tokens；对于非常强的 4.8 级 model，通常可到 200K。

Four is rare but I will go up to 250 300k tokens for certain types of work where my intuition tells me that I can keep working without degrading the performance. But if you don't have good LLM intuition, like 100K for smaller models, 200K for these like really beefy like Codeex and Opus 4.8 models is usually a good like training wheel guideline of like if you pass there, your quality of results may be degrading. Jeff Huntley 与 Ralph Wiggum 的核心是：context window 用得越少，outcome 越好。 smart zone 指最前部 context，应该表现明显更好；进入 dumb zone 后则会遗忘、困惑、能力下降。

The biggest tell I see for this is often the uh model's trying to get the test to pass and your 200k token. Well, let me try this. 有时我会走到 250K、300K，偶尔 400K tokens，因为 intuition 告诉我特定 work 可继续而不显著 degradation。

Okay, let me try that. and it's like trying a bunch of stuff and it's getting more and more extreme and it's like thing oh let me delete your end file and try again like this is where things get really weird and so it's like if you start to see certain types of if I'm like oh we're at 300k tokens and I need to like fix the unit test I'm like cool write everything we did to a file or even I'll just do like a a built-in compaction depending on the model and then I'm starting a new session at 30k or 50k tokens and I'm like cool we're going to do a hard thing which is you're going to get this freaking test to pass and you're not going to be stupid about it by the One thing that you said like about the the the model being dumb is you said that if the model ever tells you you are absolutely right you should start over and we've all had that when it tells me like oh you know you didn't you're absolutely right and I'm like we just get annoyed but why should we start over what's happening there in your um observations yeah that's great yeah and the new the new you're absolutely right I think is uh you're right to push back on that right yes that's opus right yeah opus is like you didn't run the test did you right could push back on that. 但若缺少 LLM intuition，smaller models 用 100K、强大的 Codex 与 Opus 4.8 models 用 200K，是不错 training-wheel guideline；超过后 output quality 可能下降。最明显信号是 model 在 200K tokens 附近尝试让 test 通过：“试这个，再试那个”，行为越来越极端，甚至说“删除 env file 再试”。看到这类行为… …如果已到 300K，又要 fix unit test，我会让它把所有工作写进 file，或使用 built-in compaction，再启动一个 30K 或 50K 的 fresh session：“现在做一件难事，把这个 test 通过，而且别用愚蠢方式。 ”主持人：你还说，model 一旦回复“You are absolutely right”就应重新开始。

I totally did it. But no, for me, you're absolutely right was always what the model would respond. If you were like, "That's totally wrong. You did it." Like you if you if you said something where you were angry or frustrated or just wanted to point out that it's done something wrong, it would respond with, "You're absolutely right." And most of us have had the experience of it says that and then it continues to do the wrong thing. 大家都会被这句话惹烦，为什么要 start over？ Dex：现在新版“You are absolutely right”可能是“You're right to push back on that”。 Opus 会说：“你没跑 test，对吧？

So, it's like once it starts doing dumb things because there's there's four things in your context window that matter. There's like the size of it, how many tokens? There's like the quality of the information is like is there any incorrect information? Like if the model had some thinking trace where it decided the wrong thing was true. Is there missing information? Does this like have context missing that it should have? And then there's the trajectory. 这个质疑很合理。 ”但过去，当你愤怒、frustrated，指出它做错时，model 总回复“You are absolutely right”。多数人都经历过：它说完后继续做错。

And the trajectory is very subtle, but you may have had sessions. The trajectory meaning you're prompting the actual history of everything. I call it trajectory is like the actual history of like what the agent has done in the past. And so if I say, "Hey, make this change." and the agent makes the change and then it runs the test and then they're broken and then it fixes the test. 一旦开始做 dumb things，要看 context window 中四类因素：size，也就是 token 数；information quality，是否有 incorrect information，比如 thinking trace 认定了错误事实；是否缺少必要 information；还有 trajectory。

I have very high confidence the next change I asked it to make, it's going to follow that path again because it's like, okay, here's a conversation and the last time the user asked me to do a thing, I made the change, I ran the test, test broken, fixed the test, and then I told the user. But if I say make a change and it makes a change, it doesn't run the tests, then I'm on a different trajectory. trajectory 很微妙，指完整历史，即 agent 过去做过什么。如果我要求 change，agent 修改、运行 tests、发现失败并修复，那么我高度相信下一次 change 也会遵循这条 path。如果要求 change 后它修改却不运行 tests，就处在不同 trajectory。

And if I say, okay, make another change, it's like basically the they're auto reggressive. So they're they're predicting the ne what's the next message in this conversation. And so the example we we talked about in uh No Vibes Allowed was of course the like hey the model makes a mistake and then you yelled at it and then it made another mistake and then you yelled at it and then it's like cool what's the next message in this conversation. 再要求另一个 change，由于 models 是 autoregressive，在预测 conversation 的 next message。 No Vibes Allowed 中的例子是：model 犯错，你骂它；再犯错，你再骂。下一条 message 应是什么？

Well look if I read the history I should probably make another mistake so the human can yell at me. So I was like okay that's a great that's a great example of like uh time to start over. Let's talk about some observations on how software engineuring is changing. One thing you talked about recently on the evolution of the coding meta is going from token harder to token smarter. Can we talk about what you mean by token harder and token smarter? 从 history 看，大概继续犯错，让 human 再骂。所以这是明确的 start-over signal。主持人：谈谈 software engineering 的变化。你最近说 coding meta 正从 token harder 转到 token smarter。这两个词是什么意思？

嘉宾 Yeah. So token harder is I mean I'm in a group chat called hyperengineering and it's all like people trying to max out their cloud subs. Oh wow. Okay. It's just like that sounds like a fun is it fun place? It's a fun place but it's like all token harder. It's like look at all the side projects I built. It's look at everything that uh I I I've gotten my Claude token. I've got six six cloud code accounts. I've gotten all of them maxed out every 5 hour period. Dex：token harder 就像我所在的 Hyperengineering group chat，里面全是试图把 Claude subscriptions 用到极限的人。主持人：听起来很有趣。 Dex：确实有趣，但全部都是 token harder：“看看我做了多少 side projects”“我有六个 Claude Code accounts，每五小时都精确把额度用满”。

I've timed it out so I always use all the tokens and it starts up immediately when the limit resets. And so it's like I mean getting into Eli Goldrat and the goal is like optimizing for utilization and efficiency of one node in your factory rather than the end to end goal of like how do we ship value and things that people like that are stable and like will last a long time. 每次 limit reset 就立刻启动继续用满。这相当于 Goldratt《The Goal》中的问题：优化 factory 中单一 node 的 utilization 和 efficiency，而不是端到端目标——怎样发布 people 喜欢、稳定、能长期存在的 value。这就是 token harder。

嘉宾 But that's my idea of token harder and it's the same thing with the dark factory thing is like hey if you if you if you remove humans from code review you can push more tokens through the system. So we talk about software factories but what is the dark factory? dark factory 也类似：移除 humans 的 code review，就能让更多 tokens 流过 system。主持人：我们谈 software factory，那么 dark factory 是什么？

主持人 Ah so the dark factory is this comes from this idea of like there are factories where uh everything is automated by robotics. So you can imagine like a car factory where it's all robots building the cars and they don't have lights because there's no humans. Oh, so that's where it comes from. The dark factory. Yeah. You walk in there's no lights. There's not even light switches. Dex：它来自 fully robotic factory 的 idea。想象 car factory 全由 robots 造车，不需要 lights，因为没有 humans。主持人：原来 dark factory 一词由此而来。 Dex：对，走进去没有灯，甚至没有 light switches。

So, it will be the fully automated software factory where it it it will be like no human input basically. No human input. Raw materials go in, cars come out. Yep. 主持人：它就是 fully automated software factory，基本没有 human input。 Dex：raw materials 进去，cars 出来。

嘉宾 And I think in in a micro like you can have many loops that are dark in your in your thing of like, hey, if uh if the code review agent comes back with a problem, you loop that back to the builder agent, it fixes it and comes back and that's dark. You don't need a human loop for that. But the full dark factory where you don't read any code, yeah, it's a good way to maximize your token utilization. 在 micro level，可以有许多 dark loops：code-review agent 发现问题，回传 builder agent 修复，再返回，无需 human loop。但 full dark factory 完全不读 code，确实能最大化 token utilization。

And it's like if if your belief is like my job is to extract as much intelligence out of the machine god as I can because that's how I get the most value and the most leverage on my time then token harder. Um and my take is basically what we talked about before token smarter is like okay how do I move faster? 如果相信自己的工作是尽量从“machine god”提取 intelligence，以获得最高 value 和个人 leverage，那就是 token harder。

How do I get as much value out of as AI as I can without having to turn the lights off while still maintaining control and taste and judgment and understanding the system architecture and having a lot of like applying my hard one opinions through 10 years of software engineering to the design of the program so that I can feel confident that the code's going to get better and more maintainable over time. token smarter 则是：怎样更快、从 AI 获取尽可能多 value，同时不关灯，仍保留 control、taste、judgment，理解 system architecture，并把十年 software engineering 形成的 hard-won opinions 应用于 program design，使 code 随时间更好、更 maintainable。

It's the same thing of like you look at like the S sur team inside Google.

They brought out this book SR site reliability engineering and the whole take was like hey we're going to go from one data center to five data centers and we need the same sixperson team to be able to manage five data centers and we need the same sixperson team to be able to manage 50 data centers next year and it's basically how do we apply software to this problem so that instead of scaling linearly of like okay every data center needs five devops people so we need to scale the people with the things how do we continually automate the parts that we don't need so a little bit orthogonal and maybe even like contra contradictory to what I just said, but this idea of like how do you find leverage and the way the way well I I think what you were saying there is like when Google did that never seek to remove those SRE from the process at all. 这类似 Google SRE team。 Site Reliability Engineering 一书的核心是：从一个 data center 扩到五个时，仍由同一支六人 team 管理；下一年扩到 50 个，也不应让人数线性增长。怎样用 software 解决这个问题？不是每个 data center 都配五名 DevOps，而是持续 automate 不需要人工的部分。这也许与刚才观点略显矛盾，但核心仍是寻找 leverage。主持人：Google 并未试图完全移除 SRE，而是提前思考如何 scale people；实际上也扩大了 team，只是没有从六人线性扩成 60 人，再叠加 management layers。

They just said like look can we think ahead and scale yourselves and they actually grew the team. It wasn't actually six people. It was more like I think Google specifically said, "Okay, we have five data centers. Next year we'll have 50. There's six of you. We do not want to have 60 people. We don't want and and then management layer and all that. 也许从 6 增到 10 或 12，等到 500 个 data centers，SRE team 确实会再增长。

It's like how can we do it with like 12 or like or like 10 and then when we'll have 500 and now actually their SRE has grown but but of course yeah but but they never you know I I think as engineers like we feel pretty threatened when someone says like all right we just want to have zero engineers like I mean that's not a fun place to work at but what it sounds like engineers 听到“最终零人”会感到威胁，那也不是有趣的 workplace。 Dex：如果零 engineers，我们两个都无法在那里工作。主持人：token smarter 就是保留 humans-in-the-loop，继续创造价值，并找出不重要、boring、无需人工的部分。

主持人 it's not a possible place to work at if they have zero engineers neither of us can work there right but do understand the token smarter is like let's keep humans in the loop let's keep adding value and figure out what are the parts which are not as relevant, boring, where we don't need it. And so like one developer can probably do more than before, but you are built to like be part of this whole thing and the lights are on in a factory. 一位 developer 也许能做更多，但仍参与整个 system，factory 的灯保持亮着。 Dex：对。 SRE 构建的 system 让 headcount 以 square-root 或 logarithmic function 增长，而 output 近似 linear scaling。

Yeah. And it's like basically I think I think what I'm trying to get to is like the connection here is like S builts a thing where like headcount scales at like a square root function or a logarithmic function whereas their output scales like linearly and you want the same that the way you do that is with good architecture and good program design. 实现方式是 good architecture 和 good program design。为了避免必须向问题投入更多 people 或 tokens，如果 software 设计得随时间更 maintainable、scalable，就能获得这种 leverage。

And so in order to like avoid this problem where you have to throw more people or more tokens at at the problem, if you design good software in such a way that it gets more maintainable and more scalable over time and like just today it doesn't feel like basically you need humans in the loop to be able to do that. Let's talk about uh AI slop. 以今天的能力看，基本仍需要 humans-in-the-loop。主持人：谈谈 AI slop。

At one point you wrote, "Yeah, AI can write your code, but it can also write your specs and PRDs." But the same the same rule is always slop in, slop out. If you outsource your thinking, you're gonna get garbage. 你写过：“AI 能写 code，也能写 specs 和 PRDs；但规则始终是 slop in, slop out。外包思考只会得到 garbage。

Yep. Um, so yeah, that's basically the idea is like the way we think about like getting high quality outputs is like yeah, you could write the code by hand or you could sit with a model and work back and forth and go maybe a little bit faster and you have control and every time it makes a change, you go read the change and if it's bad, you tell it, nope, we want it like this and you kind of incrementally slowly. ” Dex：对。获得 high-quality output 的一种方式是手写 code；另一种是与 model 来回合作，稍微加速但保持 control。每次 change 都读，bad 就告诉它按另一种方式做。这类似使用 agents 的 stage two 或 stage three：agent 写所有 code，但你深度在 loop 中。

This is like kind of the stage two or stage three version of working with agents where like the agents writing all your code, but you're kind of very much in the loop. And this will make you go faster, but it won't make you go that much faster. It won't make you go anywhere near there's like there's like that level and then there's like the maximum speed you can go while still caring about the code. 这样会更快，但不会快很多。速度存在几个 level：深度 human-in-loop；仍然关心 code 时可达到的最大速度；以及关灯后的最高速度。

主持人 And then there's like the maximum speed you can go if you turn the lights off. 我们从 leverage 角度思考：所有任务都始于一句话或 voice-note ramble，比如“我要构建这个东西”。

And so we always think about it as like in terms of leverage is like, okay, let me take everything starts with like a sentence or a voice note ramble like I want to build this thing. is going to work like this or whatever it is to let's say like on average like two sentences I got to fix this thing or there's a support ticket I got to fix this thing if you can turn that with AI into a one pager and then turn that one page and make sure that's correct and then turn that one pager into a three-pager and make sure that's correct and then turn that three-pager into a 10-page like detailed outline then you can write a 100 pages worth of code and it's maybe not perfect you shouldn't like sweat over these documents and make sure they're perfect but you're increasing the chance that like you're decreasing the uncertainty of the outputs. 平均也许两句话：“我要修这个问题”，或一张 support ticket。用 AI 把它变成 one-pager，确认正确；再扩成 three-pager，确认正确；再扩成十页 detailed outline。随后就能写出上百页 code。文档无需追求完美，但逐层 review 会降低 output uncertainty。想象目标方向是一条线，而可能结果在一定范围内分布；随着 building 方式越来越详细，沿途 review 会收缩 uncertainty 与可能 end states。

It's like you can think of like you have like a line of like where it's going and then you have like the probabilities of where like it might go in that range if you are kind of reviewing along the way as you get more and more detailed into how what you're building and how you want it to be built. You kind of collapse the uncertainty and the set of end states that you could land in. 这是 physics 式表达：superimpose probabilities。我认为喜欢 real-time strategy games 的人可能很擅长 AI，因为必须处理 fog of war：problem 前沿有未知信息，怎样发现？

That's me doing the physics thing of like you got to superimpose all these probabilities and like I don't know I have this thing that like I think people who really like playing real-time strategy games uh are probably going to be really good with AI because you kind of have to like I don't know. Matt Matt PCO was just talking about fog of war and like things that are at the frontier of like there's stuff we don't know about this problem yet. 如何根据已见内容做当前最佳 decision？看到几条 information 后，可能这个解释概率 30%，另一个 40%。怎样获取更多 information，在脑中重新计算 probabilities，选择最可能成功的 path？主持人：说到 success path，谈谈刚走出 stealth 的 HumanLayer。

How can we find that out and how can I make the best decision now knowing what I have seen? there's a I've seen a couple pieces of information and so there's a 30% chance it's this and there's a 40% chance it's this. How could I get more information? So in my head I can like recalculate those probabilities and decide what's the most likely path that's going to lead us to success. 它是什么，成功概率是多少？ Dex：100%，也许 110%。 HumanLayer 是 AI IDE、collaboration platform，也是 software factory building blocks。

Speaking of the most likely path that leads you to success, let's talk about your company that's you've just come out of stealth human layer. What is human layer and what is the probability that you're setting up for success? 基本 pitch 是：在 complex codebase 中解决 hard problems 的 engineers。

That's a good question. 100% 100% probability uh maybe 110 but uh no uh so human layer is it's an AI IDE it's a collaboration platform and it is building blocks for your software factory and the basic pitch is like engineers solving hard problems and complex code bases basically there's two categories of builders there's like vibe coders building side projects and then there's people building production software where the stakes are high and if something breaks we're going to get fined millions of dollars or you know we're going to lose millions of dollars of money for the company and there's a whole spectrum in between there. builders 可分为 vibe coders 做 side projects，以及构建 high-stakes production software 的人。后者一旦 broken，company 可能被罚或损失数百万美元，中间还有完整 spectrum。如果处在高 stakes 的一侧，software 必须长期存在，HumanLayer 要帮助这类人以两到三倍速度解决问题，同时不陷入 slop：怎样保持 near-human quality 又加速两到三倍。

But it's like if you're kind of in the left half of that spectrum, you're building software that matters and it has to last and be around for a while, then you're helping people like that solve problems two to three times faster without descending into slop is like how do you maintain that near human level of quality and move two to three times faster? And what were the ideas that you you built and that you came with? one idea that we're really excited about right now. 主持人：你们带来了哪些 ideas？ Dex：当前最兴奋的一个源自 RPI 和 specs：从 high level 开始，一层层 zoom in 并重新 steering，寻找 leverage，提高 agent 构建准确目标或 high-quality result 的概率。

I mean, it all comes from this RPI and this like using specs to like I mean I've kind of been hinting at it this whole time, right, of like okay cool like start really high level and zoom in layer by layer and resteer and like find that leverage that helps you move faster and increase the chance that your agent's going to build exactly what you want or something that's really high quality. 另一个想法是我昨天发的：“我们应该杀掉 pull request 吗？ ”现在不能多谈，但基本观点是，future IDE 需要从头针对 agents 重构。

The other thing I think that's really interesting that where I just posted yesterday I said, "Hey chat, should we uh kill the poll request?" And that's uh something I can't talk too much about, but basically the idea is like the IDE of the future needs to be rethought from the ground up for agents. And it might not even be a like I don't know a lot of editors kind of started with the text field and bolted on an agents tab. 很多 editors 从 text field 起步，再加 agents tab；Cursor 3 则 agent-first 到几乎找不到 text editor。我们从“怎样设计帮助 developer 与 agents 互动、管理 agent work 的 IDE”出发，再 zoom out：怎样让它 collaborative？

And then eventually you've seen like cursor 3. I can't even find the text editor. I know it exists. People have told me you can get to a text view of files, but it's also very agent first. And so we started from the ground up of like what is an IDE designed for helping a developer interact with and manage the work of agents. 加入 sync engine、durable streams 等 technology，使 people 能实时对 agent work 提供 input 和 feedback，而不是等到 pull request 阶段。

And then we zoomed out and said how do we make this collaborative and build in a sync engine and durable streams and all of these like pieces of tech that enable me to get human input and feedback on what I'm doing with agents in real time rather than waiting for the pull request time. 优秀 engineering teams 几十年来都先做 design review。

And great engineering teams have been doing this for decades of like, hey, we're gonna have a design review where we're going to talk about how we're going to build the thing as like a two-page Google doc or whatever 10page what, however,BRD er architecture requirements document and then you go to sprint planning and you break it down into little tickets and you decide who's going to do what. It's like AI can help with all of this. 他们用两页 Google Doc 或十页 architecture requirements document 讨论怎样构建，再在 sprint planning 中拆 tickets、决定谁做什么。 AI 可以帮助全部流程。如果只用 AI 写 code，就错过了它给 SDLC 带来的大量收益。

You should, if you're just using AI to write the code, you're missing out on a lot of the benefits that AI can bring to your SDLC. And a lot of people say like, "Well, we don't need any of those meetings anymore because we have the loop. We have the dark factory. 有人说 dark factory loop 会替代 meetings，事情自己流转。但若想真正更快并维持 quality，就应在写 code 前设置 checkpoints，并用 AI 协助。

Things just fly around the loop." But it's like, "Okay, but if you want to actually move faster and maintain quality, then like you should have these checkpoints before you go to actually write the code and you should use AI to help with that." So, we built this like cloud platform that's kind of has like a Google Doc style component where you can comment and the agent can surface like mockups and mermaid diagrams and HTML and all these things. 因此我们构建了 cloud platform，带 Google Docs 式 commenting，agent 可展示 mockups、Mermaid diagrams、HTML 等。目标是像“大 Figma”一样：全部在 cloud、全部 collaborative，能看到 coworkers 的 sessions，他们也能看到我的。

So, basically, how do we make agents like Big Figma style? Everything's in the cloud. Everything's collaborative. I see all my co-workers sessions. they see all of mine. It's almost like the benefit that Slack had over email was that you didn't have to be in every conversation to know what was happening. You could maintain you could see all these channels light up. You could check on them. Okay, I don't care about any of that. 这类似 Slack 相比 email 的优势：无需加入每段 conversation，也能看到 channels 活跃，按需查看；若看到关心的 conversation，可以加入。

But if you saw a conversation that you cared about, you could jump in on that. And it's like how do we do that for engineering work versus like we really had these like very strict even when we called it agile it's very waterfall like PRD ard tickets everyone goes and builds for a day and then you get the PR back and then one person reviews it. engineering work 过去即使称为 agile，也非常 waterfall：PRD、tickets、每个人构建一天，再由一人 review PR。怎样变成更流动的“soup”？那个世界的数据模型是什么？

How do you create this more just like soup and like what is the data model for that world where you have like agentic traces, you have documents, you have tasks and projects that group these things, you have actual git diffs being streamed everywhere where it's like why would I review all the code at once when I can just always every everybody's work lives in a shared environment that anyone can go interact with. 其中有 agentic traces、documents、将它们组织起来的 tasks/projects，以及到处 stream 的 git diffs。既然所有人的 work 都在 shared environment，任何人可随时 interact，为什么一定要一次 review 全部 code？

I mean what it reminds me is like what you know GitHub that did the software teams before GitHub and its competitors you might have a tracker somewhere but most teams were just kind of like in inside the company you didn't know what one one team was I I remember pre- GitHub like you know you had individual teams they some of them had like a board with stickers but no one else in the company knew what they were doing they were all working isolation and now when you have GitHub or even the internal version of of GitHub inside a company you can always see when when you go to a team you you see the pull request flying you you can join in you have history it's all it is all kind of connected and it it came together and now it's like you know for a very long time I was like you duh you're going to use GitHub or or people will copy it so do I sense that you're trying to build something like this this workflow for like when you have the the software factories which are like dark factories and loops at a bunch of places how can we have this this new way of working which which will feel natural but like coming up with it like is is hard work and it's it's counterintuitive. 主持人：这让我想到 GitHub 给 software teams 带来的变化。 GitHub 之前，company 内不知道其他 team 在做什么；每个 team 可能有自己的 board 和 stickers，彼此隔离。 GitHub 或 internal equivalent 让人看到 PR 流动、随时加入、有完整 history，工作被连接起来。过去长期默认大家会用 GitHub 或其 copy。你是否正尝试为 software factories——存在 dark factories 与许多 loops——构建类似的新 workflow？它最终会感觉自然，但发明出来很困难、反直觉。 Dex：怎样做出完成 GitHub 作用、但好十倍、更 continuous、real-time、collaborative，而非依赖 pull request 这种 discrete unit of work 的东西？

How can we do something that accomplishes what GitHub did but like 10x better like more specifically like more continuous and more real time and more collaborative than like these discrete units of work that is like the poll request. Well, I now I'm starting to understand why you're saying maybe we should kill the poll request because pull request was invented by GitHub, right? 主持人：现在我理解你为什么说也许该杀掉 pull request。 pull request 是 GitHub 发明的，并非 Git 本身的一部分，它提供 pre-merge code review，可 modify 或 reject。

like it's it is not part of Git, but they did it as a way for you to do a code review merge before it goes in and be able to modify it or or like just reject it, etc. And it's probably a lot better than whatever we had before, which I guess was like emailing your git patch to Lionus and ask him to merge it into the kernel or whatever. They still do that. It work it works for them. That's the point. But it only works for them. Yeah. I don't know anybody else who does that. 它大概比之前把 git patch email 给 Linus、请求 merge 到 kernel 好得多。 Dex：他们现在仍这样做，而且适合他们，但只适合他们。主持人：其他人几乎不用。 GitHub 前可能用 CVS。

I mean, I'm sure even before Get Up for you, you guys had what, like CVS or CVS? So, if you had a lot of money for Microsoft,they made us use subversion at in undergrad because the guy who invented subversion uh was a you Chicago guy. The year after I graduated, they switched everybody to Git. And I was like, damn, I learned a useless thing just for somebody's ego. specifically for AI startups or startups building on top of AI or building AI products. Dex：本科时学校让我们用 Subversion，因为 inventor 来自 University of Chicago。我毕业后一年就全切 Git，感觉自己因某人的 ego 学了无用工具。主持人：对 AI startups 或构建 AI products 的 startups，location 和 network 有多重要？

How important do you think location and network is especially you are based in the the valley we see research that AI startups are more frequently funded from here than normal startups as well. Do do you see this advantage and also do you see some disadvantages of being a specific may that be Silicon Valley or elsewhere? I don't have really strong opinions on this. Actually, like Paul Graham gave a talk in Sweden about why SF is cool. 你位于 Valley，研究也显示 AI startups 更常在这里融资。有什么优势与劣势？ Dex：我没有强观点。 Paul Graham 曾在 Sweden 演讲为什么 SF 很酷，我不重复，可以把链接放 show notes。

Rather than just regurgitate that, I will I will forward people onto that one. Um, we can put it in the show notes or whatever, but he talks about all of the dynamics of Silicon Valley and the pay it forward culture and the like people take you way more seriously just because you're based here. I lived in Chicago for a long time. I have a lot of really good friends from high school, from college, from growing up in LA. 他谈 Silicon Valley dynamics、pay-it-forward culture，以及只因 based here 就更容易被认真对待。我长期住 Chicago，也有高中、college 和 LA 成长时期的好朋友。但从未像现在这样与“my people”高度契合、被理解和连接。

And never before have I felt like so locked in with like my people more. Never have I felt more seen, more connected. Like there's just so many people here. Again, talking about the founder thing, people who care deeply, who are incredibly competent, who like we have all the same types of problems. We love all the same types of things. 这里有很多深度投入、极其 competent 的人。大家有相同 problems，也热爱相似事物。我不办打 video games 的 LAN party，但朋友会来，一起在 office 待到晚上十一点，co-work、hack 有趣 projects。

Like I don't do land parties where we play video games, but all my buddies will come over and we'll sit in the office till 11. We'll just do co-working and like hack on cool fun projects and stuff. And like you can't do that anywhere else. There's not enough like uh critical mass for that to just happen organically everywhere you go. and and I absolutely love it. I wouldn't trade it for anything. Yeah, I think a critical mass nails it on on the head. 其他地方缺少足够 critical mass，很难自然发生。我非常喜欢，不愿交换。主持人：critical mass 概括得很好。招聘时寻找什么人？ AI 时代的 standout engineer 是什么，怎样确认 traits？

When it comes to hiring, what types of folks are you hiring for specifically? Cuz I'm interested in how hiring changes and and what what a standout engineer means for you and how you are trying to, you know, confirm that those traits exist. In general, we we are looking um for people who are have really strong software fundamentals. So, understand distributed systems, understand like the core fundamentals of CS and operating systems and these kind of things. Dex：总体寻找拥有很强 software fundamentals 的人，理解 distributed systems、CS、operating systems 等核心 fundamentals。不必是 kernel design PhD，但这很重要。

I mean, you don't have to be a PhD in freaking kernel design or whatever, but it's a lot easier. We can we can teach somebody, I think, to be a really good AI developer in a few months. You can build enough intuition where you are, you know, accelerated off the ground and you can go like keep growing there. It's really hard to teach someone a CS undergrad program in in 3 months. 几个月可以教会一个人做优秀 AI developer，并建立足够 intuition 继续成长；但三个月无法补完整个 CS undergrad program。主持人：未来几年，你最兴奋、希望解决的 software/product engineering problem space 是什么？

And what's a problem space that you're excited about in in software engineering or even product engineering or building products that you think in the next few years is going to be one of the interesting things that you're going to be attacking? Dex：co-founder 更适合回答，但 cloud、sandboxes、sync 等 real-time 领域有很多有趣进展。

My co-founder could talk more about this, but like there's a lot of interesting things happening in in real time in cloud and sandboxes in sync and kind of like using these new building blocks that have gotten really solid in the last couple years. We're big fans of the electric SQL team. where users have durable streams. It's like how can you build systems that kind of are a lot more spread out and distributed and almost like decentralized. 过去几年新 building blocks 已很成熟。我们非常喜欢 ElectricSQL team。 users 拥有 durable streams，systems 可以更 distributed，甚至接近 decentralized。

This is really interesting for coding because you want to be able to run coding agents anywhere. You want to be able to run them for a short time, for a long time, on demand, on a schedule, all these things and have them all be part of this kind of like brain. So I don't know, parts of what we're doing are really boring like all our data is in Postgress and then parts of what we're doing is really interesting. Um, but there's a lot of distributed systems problems. 这对 coding 很有意思，因为希望 coding agents 能在任何地方运行：短时、长时、on-demand、scheduled，并全部成为同一个 brain 的一部分。我们做的一部分非常普通，比如所有 data 都在 Postgres；另一部分很有意思。这里有大量 distributed systems 和 infrastructure problems。

There's a lot of infrastructure problems. Like we are building tools for AI, but there's a lot of problems in building collaboration platforms that are really really hard and there's a lot of new tech that makes it easier and more interesting, but it's still uh by far from an easy problem. 我们构建 AI tools，但 collaboration platform 本身也有非常困难的问题。新 technology 让它更容易、更有意思，却远非简单。

It sounds like what you're saying is like the infr layers to some extent a new infrar being built and it'll take some time and but it'll be like just new new blocks and it will eventually become the primitives like for cloud we have primitives already but it took freaking decade to get those together or more. Yeah. You had AWS in what like 2008 2006. Yeah. Uh and then you got Kubernetes a decade later. Yep. And as closing, what's a book or or reading that you would recommend? 主持人：听起来新的 infrastructure layer 正在形成。它需要时间，最终会形成新 primitives。 cloud primitives 也花了十多年才齐备。 Dex：AWS 大概 2006 或 2008 年出现，Kubernetes 又晚十年。主持人：最后推荐一本 book 或 reading？

Something that you personally enjoyed.

嘉宾 Nowadays, we talk a lot about refactoring by Martin Fowler classic. I think it's because we spent a lot of time uh improving the design of existing code and trying to figure out how to get models to build code that is easy to maintain and like easy to read and easy to understand and easy to to build on. I feel like I probably have a better answer than that, but that's that's what's top of mind these days. Uh we're reading a lot of like classics of software engineering. Dex：最近常谈 Martin Fowler 的经典《Refactoring》，因为我们花大量时间改善 existing code design，尝试让 models 写出 maintainable、易读、易懂、易扩展的 code。也许我能给更好的答案，但这是目前最先想到的。我们正在重读许多 software engineering classics：《Refactoring》《Clean Code》《The Pragmatic Programmer》等，它们比过去任何时候都更 relevant。

Refactoring clean code, the pragmatic programmer, like all that stuff is I think is more relevant now than it has ever been. Love it. Well, Dex, thanks so much. This was fun. This was a blast, dude. Thanks for having me on. This was great. Uh I had a lot of fun. I don't know about you, but I really enjoyed this conversation. Dex is such a big believer in gender coding. 主持人：很喜欢。 Dex，感谢分享。 Dex：非常愉快，谢谢邀请。旁白：我不知道你是否同感，但我很享受这次 conversation。 Dex 非常相信 agentic coding，却也警告：如果停止读 code，大约三到六个月后，codebase 会变得重写比修复容易。

Yet, he's the one warning us that if you stop reading the code, you have about 3 to 6 months before your codebase becomes easier to rewrite than to fix. And this comes from first hasn't experience. His team built a light software factory, ran it, and then had to shut it down. I also like the idea of the slow loop. Loop engineering feels like a somewhat meaningless term to me. What Dex's team does is actually pretty boring. 这来自亲身经验，他的 team 构建并运行 lights-out software factory，最终不得不关闭。我也喜欢 slow loop。 loop engineering 对我来说有点空泛，但 Dex team 的做法很朴素：cron job 每晚运行，只修一个 issue 或 anti-pattern，打开一个 small pull request。

A cron job runs every night, fixes one issue or one anti-attern, and opens one small pull request. The team wakes up to a codebase that's a little bit better every morning, and dev still needs to review and prove it. This is a practice that honestly any engineering team could just adopt today. Finally, I really enjoy the history lesson. team 每天醒来时 codebase 稍好一点，developer 仍需 review 和证明正确。这是任何 engineering team 今天都能采用的 practice。最后，我很喜欢历史课。 software factory 一词来自 1968 年 NATO conference。

The term software factory comes from a NATO conference in 1968. The idea of software used to build software with analogies to a factory is more than 60 years old and every generation of our industry has tried to automate more of the loop of building software. AI agents are just yet one more attempt, although probably the most successful one. 用 factory 类比、通过 software 构建 software 的 idea 已超过半个世纪。 industry 每一代都试图 automate software-building loop 的更多部分，AI agents 只是又一次尝试，也许是最成功的一次。

Do check out show notes below for the related the pragmatic engineer deep dives that go even deeper into AI engineering and other related topics. If you enjoy this podcast, please do subscribe on your favorite podcast platform and on YouTube. A special thank you if you also leave a rating on the show. Thanks and see you in the next 请查看 show notes，其中有 The Pragmatic Engineer 关于 AI engineering 等主题的 deep dives。喜欢节目请订阅。也可以在 YouTube 订阅；如果留下 rating，我们会特别感谢。谢谢，下期再见。