Netflix 为什么在 AI 时代需要更多 Systems Thinker

主持人 Everyone can be everything now. PMs can ship code, designers can write PRDs, engineers can product, and there's this confusion and frustration of what is my job anymore. Anytime a new technology comes along, you go through a storming phase before you go through the forming phase of things. We are in the middle of that right now. I don't think that means we should put AI back into the box and say let's not use it. 现在每个人似乎什么都能做：PM 可以交付 code，designer 可以写 PRD，engineer 可以做 product，因此大家困惑又沮丧：我的工作到底是什么？每当新技术出现，事物在进入 forming phase 前都会经历 storming phase，我们现在正处其中。这并不意味着应把 AI 放回盒子里、停止使用。

If we all become builders, will we still need separate functions? I still see a craft excellence that's really important that I don't think is going away anytime soon. I still find great engineering to be scarce, great data science to be scarce, great creativity to be scarce. If you look at the early culture deck of Netflix, high agency, autonomy, paying top of market, this is what I hear constantly now from how the top AI labs operate. 主持人：如果人人都成为 builder，还需要独立 function 吗？嘉宾：我仍认为 craft excellence非常重要，短期不会消失。优秀 engineering、data science 和 creativity 仍然稀缺。主持人：Netflix 早期 culture deck 强调 high agency、autonomy、top-of-market pay，这也是我如今不断听到顶尖 AI lab 的运作方式。

Netflix's culture has always been excellence as an operating system. It's a resistance to do the thing that a lot of bigger companies would do and to feel comfortable in that discomfort very often. What are the ingredients to make this happen? Talent density is the non-negotiable, being very comfortable with risk-taking in cases where things are not going well, not assume that process is going to fix it. What have you added to the career ladders within this AI world? 嘉宾：Netflix culture 一直把 excellence 当作 operating system，愿意抵抗大公司通常会做的事情，并经常在不适中保持自在。主持人：实现它需要哪些 ingredient？嘉宾：talent density 不可妥协，要能坦然承担 risk；事情进展不佳时，不要假设 process 会自动修好。主持人：AI 时代的 career ladder 增加了什么？

more systems thinkers, people who can look across all the business domains and abstract that to here's the building blocks we're going to need. 嘉宾：更多 systems thinker，能跨越所有 business domain，抽象出我们将需要的 building block。

Elizabeth Stone How do people learn this? Small trick, each problem you're trying to solve, step out one click to the what am I assuming is true about the broader space. Today my guest is Elizabeth Stone, product and technology officer at Netflix. This is Elizabeth's second visit to the podcast. Her first visit, when she was just a CTO, was for the longest time one of the most popular episodes of this podcast. 主持人：人们怎样学习？嘉宾：有个小技巧：面对每个待解决问题，向外退一步，问“我对更大范围假定了什么是真的？ ”今天的嘉宾是 Netflix Product and Technology Officer Elizabeth Stone，这是她第二次来。第一次来时她还只是 CTO，那期很长时间都是本节目最受欢迎的 episode 之一。

You'll soon see why this is such a killer conversation because when we chatted two and a half years ago, AI was only starting to emerge. And as a long time head of engineering and product and data science, Elizabeth has such a unique perspective on where things are heading and what's worth paying attention to. Prior to Netflix, Elizabeth was VP of Science at Lyft, Chief Operating Officer at Nuna, an economist at The Analysis Group, and a trader at Merrill Lynch. 你很快会明白为什么这次对话这么精彩。两年半前我们交流时，AI 才刚开始兴起；作为长期领导 engineering、product 和 data science 的人，Elizabeth 对未来走向和值得关注的事情有独特视角。

Before we get into it, don't forget to check out Lenny's Product Pass dot com for an entire year free of the hottest and best crafted AI products in the world available exclusively to Lenny's newsletter subscribers. With that, I bring you Elizabeth Stone. Elizabeth, thank you so much for being here and welcome back to the podcast. Thank you. I'm honored to be here. Once and now twice. That's right. That's a rare a rare treat for me. 加入 Netflix 前，她曾任 Lyft VP of Science、Nuna COO、Analysis Group economist，以及 Merrill Lynch trader。开始前别忘了查看 LennysProductPass.com，Lenny's Newsletter subscriber 可免费使用一整年世界上最热门、设计最好的 AI product。现在欢迎 Elizabeth Stone。

I don't know if you know this,but your first visit to the podcast, your episode ended up being my second most popular episode. You're right behind Brian Chesky for the longest time. Well, I I I'm pleasantly surprised and also mildly competitive of how do I get to the first spot? But I'll set that aside for now. That's This is our This is our shot. Bri- Brian's amazing, so I'll let that one go. Elizabeth，感谢到场，欢迎再次回来。嘉宾：谢谢，很荣幸第一次、现在第二次来到这里。主持人：这对我很罕见，不知道你是否知道，你第一次来那期成了第二受欢迎的 episode，很长时间仅次于 Brian Chesky。嘉宾：我既惊喜又有一点竞争心：怎样才能升到第一？

Yeah, he is uh and then there's just like all these fancy AI people that are just coming, you know, coming in hot. 先放一边。 Brian 很棒，让他保住第一吧。主持人：是的，而且各种知名 AI 人物正在快速追上来。

主持人 Um so, it's been 2 and 1/2 years at this point. A lot's changed. Uh obviously AI, something AI is allowing uh people to do is everyone can kind of be everything now. This idea of PMs can ship code, designers can write PRDs, and engineers can product, and everyone's everything. There's a bunch of elements to this conversation. One is that I've heard from people that there's also this kind of confusion and frustration of like what is my job anymore? 已经过去两年半，变化很多。 AI 让每个人似乎都能做一切：PM ship code，designer 写 PRD，engineer 做 product。这个讨论有许多方面，其中一点是我听说人们感到困惑和沮丧：我的工作究竟是什么？作为 PM 或 designer，我负责什么？你体验过吗？嘉宾：Netflix 内部当然听到过。

Like what am I responsible for as a PM, as a designer? Is that something you've experienced? I hear it within Netflix, for sure. I think anytime a new technology comes along, especially one that's as transformative as GenAI, you go through a storming phase before you go through the forming phase of things. And I think we are in the middle of that right now. 每当新 technology 出现，尤其是 GenAI 这种 transformative technology，事物在 forming phase 前都会经历 storming phase。

I don't think that means we should put AI back into the box and say let's not use it cuz this is kind of this is complicating all of our preconceived notions about our roles, but I do think it means we have to be much more thoughtful about how do we get the benefits while reducing the costs. I think it's a great thing that people are experimenting with how can I develop an idea faster, prototype an idea, put together an initial set of code that would allow us to test it. 我们现在正处于这个阶段。这不意味着应把 AI 放回盒子、因为它扰乱了对角色的 preconceived notion 就停止使用；而是要更审慎地思考怎样获得 benefit、降低 cost。人们探索怎样更快 develop、prototype idea，或先写出一套initial code 来测试，这是好事。我是否认为每个人都应该把 code ship 到 production、每个人都真的做所有事？

Do I believe that means anyone should be shipping code to production? That everyone should actually be doing everything? Probably not. But I think that it's good for people to be exploring what's possible. 大概不是。但探索可能性是好的。 product 与 tech team 放在一起的优势是：若 business problem 清楚，角色有一定 fluidity 是健康的。

And then, like I mentioned earlier, the benefit of having product and tech teams together is that if the business problem is clear, I think it's okay and it's healthy for there to be some fluidity in the roles that people play because instead of having to wait for the engineering team to be ready to be able to prototype something, product and design can move faster on it. But they should still work with their engineering partner to think through how should we productize this? product 和 design 不必等 engineering team 准备好才能 prototype，可以更快行动；但仍要与 engineering partner 一起思考怎样 productize、scale，以及 guardrail 是什么。

How do we scale it? What are the guardrails for it? So, I don't think it makes the functional expertise obsolete. I think it means that teams have to be more comfortable with maybe this helps us move faster in a certain direction. From an organizational perspective, things I think about to make this more coherent or less frustrating are some of the things that have to be in place for us to get the benefits rather than the costs. 所以 functional expertise 不会 obsolete，只是 team 要接受这种 fluidity 能让某些方向更快。从 organization 角度，要让它更 coherent、少一些 frustration，需要具备一些条件，才能获得 benefit 而不是 cost：明确 source-of-truth data；对 production code shipping 设 guardrail；大改动前测试；思考哪些 AI output 可信，哪些。

So, that includes clarity on source of truth data, guardrails on shipping code to production or testing before we make large changes, thinking about opportunities where we can trust the output of AI versus we should have a process or review that helps us check that we're getting high quality outcomes. And the importance of reiterating that humans are still responsible for what happens. 需要 review process 来确认 high-quality outcome；并反复强调 human 仍对结果负责。

So, it can be that an agent wrote the code or I helped to do an analysis when that's not really my background, but it doesn't make it doesn't make people not have the responsibility that comes with what they've created. So, I think the investing in some of those core infrastructure and practices and reiterating the accountability and responsibility for the outcomes helps to balance some of like what's possible with what we should actually be doing. 即使 code 由 agent 写，或由非专业背景的人借 AI 做 analysis，创建者仍承担责任。投资这些core infrastructure 与 practice，重申 outcome accountability，才能平衡“可以做什么”和“应该做什么”。

This episode is brought to you by our season's presenting sponsor WorkOS. What do OpenAI and Vercel, Replit, Sierra, Clay, and hundreds of other winning companies all have in common? They are all powered by WorkOS. If you're building a product for the enterprise, you've felt the pain of integrating single sign-on, SCIM, RBAC, audit logs, and other features required by large companies. 以下为 WorkOS sponsor 广告：OpenAI、Vercel、Replit、Sierra、Clay 等公司都使用 WorkOS。如果你在为enterprise 构建 product，就会体验 single sign-on、SCIM、RBAC、audit log 等 integration 的痛苦。

WorkOS turns those deal blockers into drop-in APIs with a modern developer platform built specifically for B2B SaaS. Literally every startup that I'm an investor in that starts to expand upmarket ends up working with WorkOS. And that's because they are the best. Whether you are seed-stage startup trying to land your first enterprise customer or unicorn expanding globally, WorkOS is the fastest path to becoming enterprise ready and unlocking growth. WorkOS 把这些 deal blocker 变成 drop-in API，并提供专为 B2B SaaS 打造的现代 developer platform。我投资的 startup 只要开始向 upmarket 扩展，几乎都会使用 WorkOS，因为它们最好。

It's essentially Stripe for enterprise features. Visit workos.com to get started or just hit up their Slack where they have actual engineers waiting to answer your questions. WorkOS allows you to build faster with delightful APIs, comprehensive docs, and a smooth developer experience. Go to workos.com to make your app enterprise ready today. 无论 seed-stage startup 想拿下第一个 enterprise customer，还是 unicorn 全球扩张，它都是最快的 enterprise-ready 路径，像 enterprise feature 领域的 Stripe。可访问 workos.com 或加入其 Slack 与 engineer 交流。

Elizabeth Stone What's really awesome about having you back on the podcast is we chatted like before AI was a massive transformation in the world. So, it's a really cool arc that we can explore here. This the shift that we've all gone through. Mhm. Coming back to the roles of the product and inch team, I'm curious how much these roles have changed in the last two and a half years. If you think about product engineering, uh design, data science, user research,which roles have changed most? 你再次来很有意义，因为上次聊天时 AI 还没成为全球巨大 transformation，我们可以探索这段转变。回到 product 与 engineering team，过去两年半中 product、engineering、design、data science、user research 哪些 role 变化最大或最小？嘉宾：你提到的几点我会重申并扩展。

Which roles have changed least? Like, what's most different in the last two since two and a half years ago? So, you've mentioned some of the things, so I'll I'll reiterate them and then maybe build. So, I have found that PMs, designers, data scientists are able to get farther in the product development life cycle before engineering really needs to be front of the line in unlocking things than was true a couple years ago. 相比几年前，现在 PM、designer、data scientist 可以在 product-development lifecycle 中走得更远，之后才真正需要 engineering 站到最前面来解锁。

I say that with some caution because, like we were talking about, I don't think it's great to all of a sudden have thousands of prototypes if they're not aimed at this is an important problem to solve for the business and the engineering partners are aware that we're solving that problem and that designers and product managers are going to take the lead in starting to shape the idea, but it's not working in a vacuum and it's not throwing a bunch of spaghetti at the wall to see what sticks. 但这要谨慎，因为我不认为突然出现几千个 prototype 是好事，除非它们针对重要 business problem，engineering partner 知道正在解决什么，也知道 designer 与 product manager 会先 shape idea，而不是在 vacuum 中工作或把一堆 spaghetti 扔到墙上看什么能粘住。

But when it's the right problem, approached in a thoughtful way with some alignment on that, I've seen product design data science move faster in the direction of let's get to something that's testable on this hypothesis. So, that's prototyping, that's writing code. The other thing I've seen as being very valuable is we have a lot of information running around in the virtual walls of Netflix. We have experiments we've run over decades. We have insights from consumers. 若选择正确 problem，以 thoughtful way 处理并达成 alignment，我看到 product、design、data science 更快走向“怎样把 hypothesis 做成可测试东西”，包括 prototyping 与 writing code。另一个很有价值的变化是，Netflix virtual wall 内流动着大量 information：数十年 experiment、。

We have input from stakeholders across the business. And that was a problem that really presented a challenge of like, how do we get the most out of that long history of knowledge and learnings to say, let's apply that to the problem we've got now to move faster in this is a promising path or this is something that we've learned something about and we could leverage here. consumer insight，以及全公司 stakeholder input。过去挑战是怎样充分利用长期积累的 knowledge 与 learning，把它用于当前 problem，更快判断某条 path 有前景，或过去已有经验可 leverage。

And AI is very powerful at distilling information, looking across a broad set of things, doing an analysis around it, getting to the core of here's some insights to start with. I I would hesitate to rely on that exclusively, but I think it's a head start. And I find even in my own work day-to-day, instead of sending an email that disrupts someone of like, remind me what research did we do in what year and what was the question and what was the test we ran? AI 很擅长 distill information、跨越广泛资料做 analysis，再提炼出初步 insight。我会犹豫是否完全依赖它，但它能带来 head start。就连我日常工作中，也不再发 email 打断别人，问某年做过什么 research、question 与 test，而是几乎立刻找到。然后我可以形成自己的判断：其中什么有意思，是否有 actionable insight。

I can find that almost instantly. Then I can form my own, here's what I find interesting about this and I've now skipped a couple steps towards is there something actionable here? So that's data analysis, it's modeling, it's distillation of information and I'm seeing more people do that to your original question. 我已经跳过了几步。这包括 data analysis、modeling、information distillation。原本只有在公司二十年、看过每次 experiment 或知道资料位置的 expert 能做，现在 product 与 tech各 function 都能更快完成。

So instead of that needing to be only the experts who were here for 20 years and saw every experiment or know where to find it, we're now able to do that faster within product and tech across all functions and a big unlock for us is our business stakeholders sitting in finance and content and advertising can do that as well and then bring back an initial hypothesis where they want to work more deeply with the data scientists and engineer and so on. 更大的 unlock 是 finance、content、advertising 的 business stakeholder 也能这样做，先提出 initial hypothesis，再与 data scientist、engineer 等深入合作。

So there's something there about the the hypothesis generation, prototyping, thinking deeply about problems that feels like it's accelerating and that functions are able to do that in a more fluid way. 所以 hypothesis generation、prototyping 与深入思考 problem 正在加速，各 function 也更 fluid。

But I still see comparative strengths. So data scientists are still going to be experts at can we trust this data? Are we interpreting it the right way? What's the data versus judgment that we should be applying here? A product manager is still going to be exceptional at saying, have we really framed the what of this? Like the problem we're solving in the right way? An engineer still has a craft around the how. How does this scale? What does high quality look like? 但 comparative strength 仍存在。 data scientist 仍最擅长判断 data 是否可信、interpretation 是否正确，以及哪里该用 data、哪里该用 judgment；PM 擅长判断是否正确 frame 了“what”，也就是待解决 problem；engineer 则掌握“how”的 craft：怎样 scale、high quality 是什么、。

What problems is this going to create for us based on how we build and deploy something? So I still see the nuggets of that comparative advantage. It's just that we're able to move more fluidly in a lot of steps that normally we would have blockers on. There's so much interesting stuff here. One is this last point you made is something I've been thinking about. If we all become builders, will we still need separate functions? build 与 deploy 方式会制造什么问题。 comparative advantage 的核心仍在，只是过去会成为 blocker 的许多 step 现在更 fluid。主持人：这里信息很多。如果人人成为 builder，是否仍需要独立 function？

There's this like member of technical staff trend that is happening in the past where it's like, all right, we don't have a title, you could be anything. 现在有一种 Member of Technical Staff trend，不再设 title，任何事都可做。

You don't have to be in a bucket. What you're saying here is you believe we will continue to have specialties, product person, engineer, data science, designer. While they do more of other functions, there's still a lot of value in Tell me if I'm hearing you correct in having the specific discipline and skill and background. I still see a craft excellence that's really important in the disciplines that I don't think is going away anytime soon. 你的意思是，我们仍会有 product、engineering、data science、design 等 specialty。尽管大家会做更多其他 function 的事，特定 discipline、skill 与 background 仍有价值，对吗？

Even if there's fluidity or blurring of the work across the functional lines. It goes back to what I mentioned earlier of you still have humans who have to make sure that what we're doing makes sense. We're solving the right problems in a way that is best for Netflix members or business stakeholders. And that if I talk to an engineer, a data scientist, a designer, yes, they speak more languages now than they used to because they have the benefit of these AI tools. 嘉宾：我仍看到 discipline 中非常重要的 craft excellence，短期不会消失，即使 functional line 之间更加 fluid 或模糊。因为 human 仍必须确认事情有意义，正在用最适合 Netflix member 或 business stakeholder 的方式解决正确 problem。现在 engineer、data scientist、designer 借助 AI tool 会更多“语言”，但在 craft 与“good 是什么”的思考上，仍有不可替代的部分，而且各 level 都如此。

But there's still something that is not replaceable when I think about the craft and how they think about what good looks like. And that feels true across all levels and you know, I still find great engineering to be scarce. Great data science to be scarce. Great creativity to be scarce. So I yes, some things are easier, but that hasn't dissolved in my mind. Are there functions that you are finding you are hiring more of? 优秀 engineering、data science 与 creativity 仍然稀缺。所以一些事情确实更容易，却没有让专业优势消失。主持人：AI tool 与 LLM 兴起后，哪些 function 招得更多，哪些需要更少？

Like the pie chart pie expanding say for engineering or PM or design or something and then functions you're need less of with AI tool and LLMs rising. Not sure that it matches exactly to functions, but I can tell you what we're having we're seeing more of, we need more of. We need more systems thinkers in a world with AI. That looks a little bit different across functions, but I could play out a couple examples. 嘉宾：不一定能直接映射到 function，但我可以说需要更多什么：AI 世界需要更多 systems thinker。不同 function 的形式略有不同。

Elizabeth Stone So, in our core infrastructure team at Netflix in central engineering, a lot of what made Netflix successful over time was that local teams with specific business problems could move fast to deliver. They very often were not feeling like they needed to be on a central paved path. They built the stack that they needed to solve the problem and have the impact. 以 Netflix central engineering 的 core infrastructure team 为例，Netflix 过去的成功很大程度上来自 local team 可围绕具体 business problem 快速交付；他们常常不觉得必须走 central paved path，而是建立自己所需的 stack。但 AI 世界中，agent 会跨多个 system 操作，又需要 source-of-truth data。

In a world of AI with agents operating across multiple systems, wanting source of truth data, the importance of having preferred paved paths that get the most of the benefits and produce some guardrails so we can make sure we're doing good work, common infrastructure, common paved paths, solving problems once with a core set of capabilities becomes more important. 此时 preferred paved path 更重要：它既提供 benefit，也给 guardrail，确保高质量工作。 common infrastructure、common paved path 与一次解决 core capability 变得更重要。

So, we are hiring more people who can look across all the business domains and abstract that to here's the building blocks we're going to need in a world with AI. So, that's one of the lenses, but also just with a lens of what got Netflix here doesn't get Netflix there. And we're going to have to have a stronger set of infrastructure to move quickly in this future. 所以我们招聘更多能跨 business domain 观察，并抽象出 AI 世界所需 building block 的人。另一个视角是：让 Netflix 走到今天的方式，无法自动让 Netflix 走向未来；需要更强 infrastructure 才能快速前进。

So, that means that engineering profiles are more distributed systems, more infrastructure, more of that system thinking mindset than a a local business expertise. Though, of course, we still have people who are deep in personalization and advertising and content delivery. So, it's more something additive for us to have that core infrastructure and systems thinking. 因此 engineering profile 更偏 distributed system、infrastructure 和 system thinking，而不是 local business expertise。

If I take another example, like design, it's extremely important that our experience design team is developing templates and again systems thinking for what does great user design look like at Netflix so that they can enable lots of people, including those who are not designers by training, to develop products that are coherent, that fit into the end-end member experience. 当然仍有人深耕 personalization、advertising、content delivery；core infrastructure 与 systems thinking 是 additive。 design 也一样，experience-design team 必须开发 template，并用 systems thinking 定义 Netflix 的 great user design，让包括非 designer 在内的许多人都能开发 coherent product，融入 end-to-end member experience。

I get really nervous about having different design languages or different types of user interactions and shipping Frankensteins, basically. So, designers need to then be the people we're hiring again for design systems thinking. How do we think about templates and expression of the brand and what a good user experience looks like and what is Netflix and like the Netflix differentiated special sauce. 我很担心出现不同 design language、不同 user interaction，最后 ship 出 Frankenstein。

So, there's more people on our design team that have to think that way now than could I help to design a specific feature for a specific product. So, there's this stepping back to look at the big picture that I think is happening in every single function and that requires some, yeah, reorientation of skills among the existing team and also hiring people who've got that that type of expertise. And across all of it, it's a mindset shift. 因此 designer 需要成为我们招聘的design-systems thinker：思考 template、brand expression、good UX 的含义，以及什么是 Netflix 特有的 special sauce。 design team 中需要这样思考的人会变多，而不只是为某个 product 设计 specific feature。

So, we are not hiring people who are not excited to explore, try new things, understand lots is changing and feel comfortable with that ambiguity, be comfortable that there's a blurring of how we work and how we partner. It that's true for people who are already at Netflix and people who we are adding to the team that that curiosity innovation mindset has not it's not been more important, at least in the time that I've been working in this field. 每个 function 都需要后退一步看 big picture，这要求 existing team 重新调整 skill，也要招聘拥有相应 expertise 的人。整体更是 mindset shift：我们不会聘请不愿探索、不想尝试新事物、不能接受大量变化和 ambiguity，也不适应工作与合作方式模糊化的人。

On the systems thinking piece, is the reason this is becoming more important that it is people are moving so fast that you need to invest in platforms and frameworks and and design language and basically teach people to fish so they can not be blocked or is there is there other reasons? I think it's probably velocity. So platforms do have a benefit of leverage. So in general, that that's an opportunity with or without AI for a platform to get most teams 80% of the way there. 无论 Netflix 现有员工还是新加入的人，curiosity 与 innovation mindset 在我从业期间从未如此重要。主持人：systems thinking 更重要，是否因为大家行动太快，所以需要投资 platform、framework 与 design language，相当于教人钓鱼，避免 blocker？嘉宾：大概是 velocity。

And then they don't have to reinvent those building blocks. We have more bets that we're making across the business, more things we're trying to build. So platform mindsets are good and it's something that is relatively more recent for Netflix to think about that being a real critical enabler. There is also the sense of a scaffolding in a world of AI. platform 本来就有 leverage，无论有没有 AI，都能让多数 team 先走完 80%，不必重建 building block。现在 business 中有更多 bet、更多待构建事物，所以 platform mindset 更有价值。 Netflix 直到相对近期才把它视作 critical enabler。

So not just the higher velocity, but you have more people doing more types of work that are different or new like we were talking about. And there's risk that comes with how do you think about access and identity in that situation? How do you think about security in that situation? How do you think about how shipping high quality code and design and user experiences? And so I I don't think it scales well to have each person who's building something have to go figure out. AI 世界还需要 scaffolding：不仅 velocity 更高，而且如前所述，更多人在做不同或全新的工作，于是 access、identity、security 都带来 risk；还要考虑怎样ship high-quality code、design 与 UX。让每个 builder 都重新去问“这里 good 是什么、有哪些 bumper 或 guardrail”无法 scale。

Could you remind me what good looks like here and what are the bumpers or guardrails I should keep in mind? I think we need to encode that in our paved paths and our ways of working. And for a a data science or analytical field to encode here's the source of truth data, here's how to interpret it, here's how to access it, here's what to do with it or not to do with it and to be careful with certain types of data. 应该把它 encode 进 paved path 与 working method。 data science 和 analytical field 也应 encode：source-of-truth data 是什么、怎样 interpret、怎样access、能做什么与不能做什么、哪些 data 要谨慎。

I don't an organization that has thousands of people can no longer rely on tribal knowledge or I'm going to find the one person who knows this. So this was a challenge that was there before AI. It's probably a more urgent challenge with AI and I like the idea of using AI or any new tech to motivate like we knew this is work we needed to do. No time like the present to invest in that more heavily across the team. 拥有数千人的 organization 不能再依靠 tribal knowledge，或去找唯一知道答案的人。这在 AI 前已经是 challenge，AI 让它更 urgent。我喜欢用 AI 或任何 new tech 激励大家：既然早就知道该做这项工作，现在就应加大投资。

I wonder if another reason for this becoming more valuable is because agents are now doing a lot of work and giving them the context, giving them the scaffolding, giving them the design language just speeds all that up. Yeah, and one of the visions we have at Netflix is we will have so many agents that are contributing to doing work that you need to be able to reason and rationalize throughout that. 主持人：另一个原因是否是 agent 正在做大量工作，给它们 context、scaffolding 与 design language 就能加速？嘉宾：对。 Netflix 的 vision 是，将有大量 agent 参与工作，需要能在其中 reason 与 rationalize。

You know, the humans are the ones guiding what's the problem we need to solve. Do I feel like what we're producing is impactful and high-quality output? But the work will be done by both humans and agents. And that creates velocity and benefits and it creates risks. And I think that's important from especially from an engineering perspective that we figure out how to manage that in a way that lets people move quickly but doesn't create undue downside or risks for the company. human 负责引导“要解决什么 problem”，判断 output 是否 impactful、high quality；工作本身由 human 与 agent 共同完成。这会带来 velocity 和 benefit，也带来 risk。尤其从 engineering 角度，必须找到让人快速行动、又不制造过度 downside 或 company risk 的管理方式。主持人：这直接呼应 Jenny Wen 的访谈。

This connects so directly with Jenny Wen was on the podcast. She was head of design for Cloud Code and Co-work and had this whole design process is dead kind of thesis and the pitch there is just there's no time for design, the design process. And instead as a designer, you're just kind of steering people and pointing them in the direction and adjusting and also thinking big picture is when you have the time. 她曾领导 Claude Code 与 Cowork design，提出“design process 已死”：没有时间走完整 design process；designer 主要 steering、指引方向、不断调整，有时间时才看 big picture。

And it feels like that's kind of what you're describing here is like create the platform for people to move fast and then there's no time for like design process of a specific new feature. I have mixed feelings about that because I we do want to enable with infrastructure and systems thinking more people to do great work with strong design as part of it. 你描述的似乎也是建立 platform 让人快速行动，不再为 specific feature 走完整 design process。嘉宾：我对此感受复杂。我们确实希望通过 infrastructure 与 systems thinking，让更多人做出包含 strong design 的优秀工作，为什么不利用新技术的机会？

Why not take that opportunity that the new tech provides. But for our most important priorities, design is critical to solve things in the right way. So, we do still make time for important design work. We It can move faster. The designers themselves have more tools in their toolkit, so they can do incredible work at a faster velocity, show more options, learn, iterate, test more quickly. 但对于最重要的 priority，design 对正确解题至关重要，我们仍会为重要 design work 留出时间。它可以更快，designer 也拥有更多 tool，能以更高 velocity 做出优秀工作、展示更多 option、更快 learn、iterate 和 test。

But I think it would be a mistake to say design and deep design expertise and thinking gets squeezed out just because we can write code faster. We can do data analysis faster. That feels like, at least for a large-scale consumer product like Netflix, I feel like we would lose one of the things that makes Netflix great, which is the product, technology, and design makes a lot of complexity invisible, and makes for a seamless customer experience. 但因为 code 与 data analysis 更快，就把 design、deep expertise 和 thinking 挤掉，是个错误。至少对 Netflix 这样的大型 consumer product，这会失去让 Netflix 出色的一点：product、technology 与 design 把大量 complexity 隐藏起来，提供 seamless customer experience。

Elizabeth Stone That That's a design mindset that has to be core to it. So, if the work itself might look different, but I don't think we lose the mindset. That's an awesome counterpoint. So, what I'm hearing is kind of trending up skills, attributes you look for, systems thinking, and this kind of mindset of being comfortable and excited about change and what's coming and not being stuck in your own ways. What are you finding is trending down? 这种 design mindset 必须处于核心。 work 形式也许改变，但 mindset 不会消失。主持人：很好的反驳。我听到 trend 上升的 skill 与 attribute 是 systems thinking，以及对 change 和未来保持舒适、兴奋，不固守自己的方式。那什么在下降？

What are you less looking for that used to value more highly? The days of very narrow, deep specialization feel more limited to me. I can come up with examples where we still need it because there's an industry or technology expertise where there's only a few people in the world who really know how things work. We have examples of that on the team for encoding or how our playback systems work and things that have been incredibly innovative and novel for Netflix. 嘉宾：非常狭窄、深入 specialization 的空间更有限。当然仍有例外：某些 industry 或 technology expertise 全世界只有几个人真正懂，Netflix team 在 encoding、playback system 等高度 innovative、novel 领域就有这种 specialist，仍然需要。

I I still believe we need specialized practitioners in those spaces. But as a general rule, uh compared to 5 or 10 years ago, I I would believe we have fewer specialists and more people who are generalists or adaptable in multiple directions. And that could be adaptable across functional expertise. It could be adaptable across flavors of engineering. So, can I navigate both back end and front end systems? Can I hook into infrastructure with a lot of expertise? 但一般而言，相比五到十年前，我认为 specialist 更少，generalist 或能向多个方向 adapt 的人更多。 adaptability 可以跨 functional expertise，也可以跨 engineering flavor：能否同时处理 backend 与 frontend system？能否深入连接 infrastructure？

I think the the mindset now needs to be I can learn that quickly, and that goes back to the systems thinking. So, I think specialists can learn to have a broader array of tools more easily than was true in the past. So, it we need fewer of them perhaps because talent's able to grow in that direction. 现在需要的 mindset 是“我可以快速学会”，又回到 systems thinking。相比过去，specialist 更容易扩展 toolset。也许因此需要更少 specialist，因为 talent 可以向更广方向成长。

And there's something about sticking to a narrow specialty that maybe triggers for me a concern about what about the mindset of growing in different directions and exploring boring, and I don't want to be too narrow even in my own assessment of that, but I it's important that people who are specialists still have that sense of I want to try a new way of solving these problems versus the way we have in the past. 固守狭窄 specialty 会让我担心：是否缺乏向不同方向成长和探索的 mindset？但我也不想在评价上过于狭窄；重要的是 specialist 仍要愿意尝试新解法，而非沿用过去。主持人：所谓 specialist 是 frontend engineer 对 backend，还是其他？

And when you say specialist, are you thinking like front end, I'm a front end engineer versus a back end, or are there other Yeah, or it could be a domain set of knowledge of Yeah, I'm a deep expert. I'm a payments expert. I'm an ads marketplace design expert. I'm an an expert in this very specific tooling that studio productions use. Mhm. 嘉宾：也可能是 domain knowledge，例如 payments expert、ads marketplace design expert，或 studio production 某种 specific tooling 的 expert。

主持人 So, there specialist in subject matter expertise is an advantage provided that person is willing to grow and extend into is this really still the right tool or the right way to think about the problem? So, I think it's the layers of the stack from an engineering perspective that there's less specialty. And then tools that are unlikely to be static or like to have a lot of inertia around them. subject-matter expertise 是优势，前提是愿意继续成长，追问这是否仍是正确 tool 或 problem framing。

I would think like we would want people who are able to innovate and imagine like what's the future version of this? And so we want more talent like that. Awesome. So coming back to the systems thinking piece, people hearing this are like, okay, I got to work on my systems thinking skill set. How do people develop the skill? Other Is it just do it for a long time? Work at a lot of complex projects? 从 engineering stack 各 layer 看，specialty 会减少；面对不太可能 static、也没有强 inertia 的 tool，我们更需要能 innovate、想象 future version 的人。主持人：回到 systems thinking，听众会想提升这种 skill。

主持人 Like I think of this book that everyone always references with the slinky on the front, Thinking in Systems. 怎样学习？是长期做复杂 project，还是读那本封面有弹簧、大家常提的《Thinking in Systems》？

Elizabeth Stone Yeah, how do people learn this? Small trick. Each problem you're trying to solve, step out one click. Do the like, what am I assuming is true about the broader space in solving this problem? So I was given a task to build some new feature for the Netflix member experience. 嘉宾：一个小技巧：对每个待解决 problem 向外退一步，问自己在解决时对 broader space 假定了什么。例如收到任务，为 Netflix member experience 构建新 feature，

Let me take one beat and think about what is the bigger consumer problem we're trying to solve here? What's the type of content that this feature is going to be able to support? Do I think that the way I was planning to build this is going to make sense in a way that scales across multiple content types? Or it could be something that's a capability that then is contributed to a platform set of offerings for multiple areas. 先停一下想更大的 consumer problem 是什么；这个 feature 支持哪类 content；原计划的 build 方式是否能 scale 到多种 content type；它是否可以成为 platform capability，为多个 area 服务。

Is the consumer problem that I'm solving with this feature going to be one of the most important consumer problems that Netflix is going to need to solve as we have an expanding world of entertainment and we want to make it more personalized and immersive. Those are all questions that like you don't have to boil the whole ocean. You don't have to solve for Netflix's overall strategy and who are we relative to competition. 这个 feature 解决的 consumer problem，是否是 Netflix 在 entertainment 越来越丰富、希望体验更 personalized 与 immersive 时最重要的问题之一？

But you take the thing you're responsible for and you just do one zoom out of the problem you're solving and question that. I wouldn't spend too long in the questioning state because then you're stuck. Then you're not making forward progress, but I think that helps people to think in terms of systems and question that are we solving the right problem in the right way that matters for the end consumer. 不必 boil the ocean，也不用解决 Netflix 总体 strategy 或与 competition 的关系；只需把自己负责的事情 zoom。 out 一层并质疑它。别在 questioning state 停太久，否则无法前进；但这能帮助人们用 system 思考，确认是否以正确方式解决对 end consumer 重要的正确 problem。

Another way as you describe it, another way I'm thinking about it is like think if you were your manager how would they what's their broader perspective across not just your one team and problem and KPI, but the larger picture? I've got advice over years that is similar to that which is are there ways that I can do my job that helps my manager do their job. 主持人：也可以想象自己是 manager，理解其不只看你的 team、problem 与 KPI，而是更广视角。嘉宾：我多年收到类似建议：是否能用帮助 manager 完成其工作的方式完成自己的工作。

And so if I thought about all the things I'm directly responsible for, but I thought about it from the perspective of my manager. So not just product and tech, but finance and content and other parts of the business, I would naturally zoom out and think about how all these component pieces need to come together and how the whole could be greater than the sum of the parts. I think that's useful thinking. 若从 manager 角度思考直接责任，不只看 product 与 tech，也看 finance、content 与其他。 business 部分，就会自然 zoom out，思考 component 怎样结合、whole 怎样大于 parts 之和。 engineer 也应思考怎样留下更好的 system，怎样构建 high-quality、可供他人 scale 的东西。

And for engineers to think about how do I leave a better version of these systems? How do I think about the thing that's going to be high quality and scale for others? There's both a how do I help my manager and there's how do I help my colleagues, which is a core part of some of our engineering principles of do the thing that is right for the broader organization instead of just what's right for you locally. That's systems thinking as well. 既要帮助 manager，也要帮助 colleague；我们的 engineering principle 也要求做对broader organization 正确的事情，而非只对 local 自己正确。这也是 systems thinking。

So it's not just seniority, but it's breadth of the way I solve this problem and I build this, is it going to be useful to my colleagues and am I going to leave a stronger version of things for the future set of innovations that we want to make? That is an awesome tactical advice. Uh making your manager's life easier is always a good a good tactic. Career-wise, several reasons. Yeah. 它不只关乎 seniority，而关乎解题 breadth：build 的东西是否对 colleague 有用，是否为 future innovation 留下更强基础。主持人：很实用，帮助 manager 总是好 tactic，对 career 也有多个好处。

主持人 Following the thread a little bit I know you all added career ladders and levels recently. It was like a new thing you guys used to not have these things. So kind of all on that thread what have you added to the career ladders within this AI world. If anything that you find you want people to lean into more, you're looking to more or or not. Like, did you not change your career ladders and performance you know, criteria? 主持人：沿着这个话题，Netflix 最近新增 career ladder 与 level，过去并没有。 AI 世界中，你们在 ladder 增加了什么，希望大家更重视哪些方面？是否改变了 performance criteria？

So, the way we've approached this so far is instead of trying to articulate at each level exactly how AI changes those expectations, to instead put an overlay across all of the talent at Netflix, people on the team, and those who are hiring to talk about an aspiration for AI fluency. And what that looks like is going to vary by function. It's going to vary based on where you are in your career. 嘉宾：我们的做法不是在每个 level 精确写明 AI 怎样改变 expectation，而是在 Netflix 全体 talent——现有 team 与招聘对象——之上增加一层 AI fluency aspiration。具体形式会因 function、career stage、level、role 或工作 persona 而异。

That could be what level you're in or what type of role or persona work you're doing. But the aspiration for AI fluency, which is a tough thing to define. So, does it mean that I have an experimentation mindset? Does it mean that I know where AI is useful and not useful? Does it mean that I've actually built things using AI? AI fluency 很难定义：是 experimentation mindset？知道 AI 何时有用或无用？真的用 AI build 过东西？

I feel like the way that has shown up in career ladders and how we talk about it evolves almost by the quarter, if not month or day, because the tech itself is advancing so much. So, the most useful thing is not to make it level specific or role specific, but to encourage everyone towards the expectation on AI fluency, which doesn't mean use it as a tech for the sake of tech. 它在 career ladder 中的体现几乎每季度、每月甚至每天都演变，因为 technology 进步太快。最有用的不是为 level 或 role 写死，而是鼓励所有人达到 AI fluency expectation。

It's tech where it's useful, to have good judgment about that, and to have the mindset to be open-minded to explore and try new things. That's the non-negotiable for all roles, and that's true at the senior most levels of of Netflix, where we talk about we too need to have deep fluency in AI, even if we're not writing code as part of our day jobs. So, that's that's changed, and then that's showing up in our hiring practices as well. 这不意味着为用 technology 而用，而是在有用处使用，并有 good judgment，同时 open-minded、愿意探索和尝试。所有 role 都不可妥协，包括 Netflix 最 senior level；即便日常不写 code，也要深度理解 AI。这也改变了 hiring practice。

Getting comfortable within interviews exploring how are people thinking about AI or technology? What are they using in their day-to-day or their current job? How comfortable are they with change and exploration? And even for things like coding interviews, allowing candidates, of course, to use AI tools because that's going to be part of what the work requires now. interview 中会探索候选人怎样看 AI 或 technology、日常或当前工作使用什么、对 change 和 exploration 多舒适。 coding interview 当然允许 candidate 使用 AI tool，因为如今工作就需要它。

So, those have been shifts that we've made, but I I doubt it's a shift that's done versus we're right in the middle of it. And she's going to keep following this thread. Obviously, AI is transformative for coding. It's a big unlock for prototyping. Are there other use cases of AI at Netflix that have been really impactful that people may not think about or not realize? 这些 shift 仍未结束，我们正处其中。主持人：继续这个话题。 AI 对 coding 很 transformative，也极大解锁 prototyping。 Netflix 还有哪些很 impactful、外界可能没想到的 AI use case？

Elizabeth Stone So, there's two that come to mind. So, the first is data analysis, distillation of information, modeling, which is, you know, get using the tools to get our arms around all the insights we have, similar to what I mentioned before. What experiments have we run? What are the metrics that I should be looking at for a certain problem? What's the consumer research that we've done? 嘉宾：有两个。第一是 data analysis、information distillation 与 modeling：用 tool 整理所有 insight，例如过去跑过什么 experiment、某 problem 应看哪些 metric、做过哪些 consumer research。

And that is much higher velocity and much higher quality, contingent on you check that the results are valid, you work with your local data scientist and am I using the source of truth data on this? But, that's been a great one and that's one personally that I would say I most use some of these tools for. So, that goes beyond prototyping and coding to general analytical thinking and translating data to action and insight. 它带来更高 velocity 与 quality，前提是检查 result 有效，与 local data scientist 合作，确认使用 source-of-truth data。这是很好的 use case，也是我个人最常用的，超越 prototyping 与 coding，进入 general analytical thinking，以及把 data 转为 action 与 insight。

The other one is on the content production, creation part of the business, which has lots of applications. This was true before GenAI. So, ML and AI were deeply used in a lot of the production tools. We've used them to think about how to create promotional assets at scale, how to localize in subtitles and dubs. So, GenAI is a big step function in where the impact can be in creative ideation. 第二是 content production 与 creation。其中有很多 application，而且 GenAI 前 ML 和 AI 已深入用于 production tool，例如规模化创建 promotional asset、本地化 subtitle 与 dub。

We call those things like pre-visualization or basically bringing a creator's vision to life before you even get into the you bring people to a set and start to actually go through the production itself. There's lots of use cases in post-production. GenAI 的 step-function impact 则在 creative ideation，例如 pre-visualization：在真正召集团队进 set 开始 production 前，先把 creator vision呈现出来。

So we recently acquired a company Inner Positive that was started by Ben Affleck that built a set of models and capabilities that allow you after you've shot something to relight, reframe, reshoot, change dialogue in ways that are very impactful to get higher quality content are still led by the filmmaker creator saying, you know what? I would like to try something else to bring this vision to life. post-production 也有很多 use case。

But that impact is extremely promising and we're seeing lots of productions leverage different tools, some of them built in-house, some of them that we enable through other vendors for those content creation use cases. 我们最近收购 Ben Affleck 创办的 InterPositive（英文自动字幕拼写），它建立一组 model 与 capability，让拍摄后还能 relight、reframe、reshoot、change dialogue，从而获得更高 quality content；仍由 filmmaker或 creator 决定想尝试什么来实现 vision。

And then as we think about how content comes to the product, I mentioned localization, subtitles and dubs, but also how we create high-quality trailers, images, artwork at scale that then we can use to help make sure that titles find their audiences around the world. Those all are huge levers when we think about the AI impact. 这很有前景，许多 production 正使用不同 tool，有些 in-house build，有些来自 vendor。

So that that again goes well beyond prototyping or coding to some of the creative use cases and you can imagine that just like they work for studio productions for film and TV, they work for advertising, they work for marketing, off-service campaigns and so those are all areas that we're exploring. This episode is brought to you by Mercury, radically different banking loved by over 300,000 entrepreneurs. And now with Command. content 进入 product 时，还包括 localization、subtitle、dub，以及规模化创建 high-quality trailer、image 与 artwork，帮助 title 在全球找到 audience。这些都是 AI impact 的巨大 lever，远超 prototyping 和 coding。它们既适用于 film/TV studio production，也适用于 advertising、marketing 与 off-service campaign，都是探索领域。

I've been a customer of Mercury's for over 6 years. I have never once thought about leaving. Mercury is basically what happens when banking is built by product people, not by bankers. They make it so easy, dare I say fun, to send invoices, move money around, set up virtual cards for folks on my team. Does your bank have an API, a terminal native CLI, or an AI-ready MCB server? I don't think so. 以下为 Mercury sponsor 广告：超过 30 万 entrepreneur 使用的不同 banking 服务，现在推出 Command。我使用 Mercury 六年，从未想过离开。

And just recently they launched command, a conversational interface built directly into Mercury, which acts as your financial operator. I've been using command to transfer money around, to figure out what categories I've been spending the most money in, analyze my cash flows. And just today I used it to find out how much I've made from a specific sponsor over the past year. I just asked, "How much have I made from X over the past year?" 10 seconds later I have an answer. 它像是由 product people 而不是 banker 构建的 banking，让 invoice、transfer 与 virtual card 都很容易，甚至有API、terminal-native CLI 和 AI-ready MCP server。新推出的 Command 是直接内置在 Mercury 的 conversational interface，作为 financial operator。

It is so freaking cool. Visit mercury.com to learn more and apply online in minutes. Mercury is a fintech company, not an FDIC insured bank. Banking services provided through Choice Financial Group and Column NA, members FDIC. You mentioned how Netflix has been very early to AI and ML for a long time. Uh younger people may not remember this, but y'all had this contest to optimize things. Yeah. Yeah, the Netflix prize. 我用它 transfer money、分析 spending category 与 cash flow。今天还用它查询过去一年从某 sponsor 赚了多少钱，十秒就得到答案。可访问 mercury.com。

Elizabeth Stone Like like just showed an example of how early you were to AI and ML. People There was I think it was a million-dollar prize to optimize the Netflix ranking algorithm a little bit. Like whoever could optimize it the most. And I think the winner optimized it by a few percentage points, something like that. And it was like the a huge deal. All these super smart people got around around the world. Uh and it happened a few times, right? I mean, you said it on my behalf. Mercury 是 fintech company，不是 FDIC-insured bank；banking service 由 Choice Financial Group和 Column N.A. 提供，二者为 FDIC member。主持人：Netflix 长期很早采用 AI/ML。年轻人可能不知道 Netflix Prize：好像是一百万美元奖金，奖励把 ranking algorithm 优化一点的人。最多的人获胜。我想 winner 只优化了几个百分点，却成为大事件，全球聪明人都参与，而且办过几次。

Um often when there's questions about how is Netflix thinking about AI, it's great to remind people of exactly that point, that this is not new to us, that especially for personalization, it's been central to delivering a great experience to members. It's impossible to take the breadth of content that we have. There's ever more content. That's one of the challenges we face. And make discovery easier and easier and easier, which is one of the challenges that Netflix has. 嘉宾：你替我说了。有人问 Netflix 怎样看 AI 时，我常提醒这点：对我们并不新，尤其 personalization 长期是为 member 提供优秀 experience 的核心。 Netflix content breadth 越来越大，而 discovery 必须越来越容易，这本来就是 challenge。

And using AI and ML has been a way to do that. You want to personalize right title for the right person at the right moment, that problem gets harder. The more exciting our catalog gets, the greater breadth of content we have, not just film and TV, but games and live and podcasts, personalization becomes even more important in what that experience is. AI/ML 一直用于此。要在正确 moment 为正确 person personalize 正确 title；catalog 越精彩、content 越丰富——不只 film/TV，还有 game、live、podcast——personalization 对 experience 越重要。

So, we can take a lot of that history and say, "Okay, well, now how do we solve this problem?" Because the tech is even more powerful, but it gives us a running head start in being clear about the problem to solve, how important it is that Netflix solve that for our members. And then the same is true, as I was mentioning, on the creative side of the house. AI and ML have been in things like visual effects or in localizing language for a long time. 我们可以借历史经验面对 tech 更强的新阶段，已有明确 problem definition 和 member value，形成 running head start。 creative side 也一样。 AI/ML 长期用于 visual effect 与 language localization，现在要问 tech 更强后下一阶段是什么。

Now we say, "What's the next era of that when the tech is more powerful?" And in In both cases, it ends up taking a strength that Netflix has, which is marrying entertainment and technology, and making sure we stay ahead of the game to deliver things that are even better. So, I I love that it it's part of our history. 两边都发挥 Netflix 的优势：结合 entertainment 与 technology，保持领先并交付更好体验。这是我们的历史与持续优势，而且面对 entertainment breadth 与 great experience 的 challenge，必须继续是优势。

It still continues to be a strength, and it's going to have to be a strength, given the size of the challenges we're facing around the breadth of entertainment while keeping a great experience. Yeah. And I I love that back then it was called machine learning, and AI was like, "No, no, this It's not AI. AI is Never never never Never going to happen. It's just machine learning." Well, then all of a sudden we call everything AI, and some of it's machine learning. That's right. 主持人：有趣的是当年叫 machine learning，人们说这不是 AI，AI 永远不会实现。嘉宾：现在又突然把一切都叫 AI，其中一些其实仍是 machine learning。我们会用当下流行的描述，把所有东西都归为 AI。其中很多 AI use case 并非 generative。若展开会掉进深洞，但 Netflix 显然使用广泛技术。

So, I I I tried to You know, it depends like the thing that is of the moment to describe. So, I think we bucket all of it as AI now. Yeah, AI has become of AI use cases that are not generative use cases. So, we could go down a deep dark hole of all the specific things. But in general, like I don't think it would surprise anyone that Netflix is using a broad array. 对热爱 tech 在 creative outlet、consumer product 或 infrastructure 应用的人，Netflix 有所有这些 problem，AI 处于中心。即使 Netflix 不以 AI company 自居，也不要忘了这一点。

And it with so much excitement about what's possible, the fun thing at Netflix for the people who work here is that if you're really passionate about the applications of tech for creative outlets, for consumer products, for infrastructure, we have all of those problems and AI is at the center of them and it's good not to forget that that that's true even if Netflix isn't branded as an AI company. AI 是我们很熟悉的 tool，用来实现优秀 entertainment 与 technology outcome。主持人：继续夸 Netflix。看早期 culture deck 和上次对话，会看到 high agency、autonomy、high talent density、bottom-up thinking、快速 experiment 与 launch、top-of-market pay。

AI is a tool that we're very comfortable using to get these great entertainment and technology outcomes. The other really interesting thing just to kind of keep complimenting Netflix here. If you look at the early culture deck of Netflix and also our conversation last time, things that emerge from that are things like high agency. This is like something core to Netflix in the beginning. 这些正是如今顶尖 AI lab 常说的运作方式。大家最终走到了 Netflix 长期所在的位置。嘉宾：它有一点先见之明，理解什么让 talent 出色。

High agency, autonomy, high talent density, very bottom-up thinking, super quick experiments and launching, paying top of market. Uh this is all stuff that every AI like this is what I hear constantly now from how the top AI labs operate. So we're all ending here and this is where Netflix has been forever. 我把 Netflix culture 的这些方面称作——听起来有点 nerdy——“excellence as an operating system”。这些 cultural element 本身不是 end goal，不是只追求让每个人拥有尽可能多 responsibility，或因为不喜欢 process 就

主持人 Yeah, it's a little prescient in understanding what makes talent incredible. I've thought about all those aspects of the culture at Netflix as this is going to sound a little bit nerdy, but excellence as an operating system. So the goal of all those cultural elements wasn't the end goal in themselves. It wasn't let's just make sure people have as much responsibility as possible or let's you know, we don't like process. So let's make sure that we don't have any of that. 全部取消。背后是强烈观点：给人足够 agency 与 accountability，把 decision 尽可能推到 organization 深处，聘请拥有 good judgment、值得信任、能做 good decision 的优秀人才，才能达到 excellence。这样既产生出色 outcome，也带来更强 motivation 与 responsibility。

It was instead a very strongly held opinion that that you get to excellence by giving people a lot of agency and accountability. By pushing decisions as deep in the organization as possible, hiring great people who can be trusted to have good judgment and make good decisions. And that ends up driving incredible outcomes plus a lot more motivation and sense of responsibility. 每位 team member 都觉得自己拿到很多 key，也对结果承担很多 accountability；当你感到如此被信任、承担责任，就会想做出 best work。

It means every person on the team can feel like I'm being given a lot of keys and a lot of accountability for what happens here and I myself feel like when you know you're carrying that level of trust and accountability, you want to do your best work. And so there's something that feels very intuitive about Netflix's culture has always been aiming at excellence. Netflix culture 直觉上一直在追求 excellence：有 great talent，又允许他们不被 micromanage 或 process 淹没地做出 best work，outcome 会更好。新一代 company 正发现一些 Netflix 熟悉的东西。

And when you have great talent and you give them the ability to do their best work without micromanaging it or drowning it in process, you actually get much better outcomes. And so I do think that the newer era companies are picking up on something that is feeling very familiar to us. And it it's not something that comes easily. So having culture is not a static thing. Culture needs to grow and evolve as a company gets bigger, the types of problems you're solving change. 但这并不容易，culture 不是 static；company 变大、problem 改变时，culture 也必须成长演变。不变的是追求 excellence，并相信 exceptional talent 必须能完成 best work。这仍是 Netflix special sauce。主持人：我喜欢“excellence as an operating system”，很有 systems thinking。

But the notion that like we're going for excellence and trusting that exceptional talent needs to be able to do their best work. That's unchanged and something that I think continues to be a special sauce for us. I love this concept, excellence as an operating system. It's very uh systems thinking, he he might say, for how to set up a company. Exactly, Lenny. So for people that like everyone listening to this will want excellence as an operating system. 所有听众都会想要这种 operating system。实现它的 ingredient 是什么？首先是 high talent density，只聘最优秀的人；其次是 accountability。输入 top talent，给予 autonomy，再要求结果。

Like who would not want this? Uh it'd be helpful for people to hear what are kind of the ingredients to make this happen. One is obviously high talent density, just hiring only the best. Two is accountability. Kind of there's like the input and the output essentially. Uh input amazing people, top the top people, give them make them accountable, give them autonomy. founder 应怎样建立这种体系？嘉宾：talent density 是 non-negotiable，必须从这里开始。没有它，就无法信任 organization 各 level 的 decision，也无法让人承担 risk、快速 innovate。

What would you say kind of like the pillars of creating this uh excellence as an operating system if people if founders are listening to this like I want them to do that. Well, the talent density is the non-negotiable. Like you have to start with that. If you don't have that, you can't get to a place where you have confidence in decision-making at all levels of the organization, allowing people to take risks and innovate quickly. Netflix excellence culture 很重要的一点是非常适应 risk-taking：不试图避免 failure，而是发生后快速 recovery。进军 live 就是很好的例子。我们愿意承担大量 risk，知道过程不完美，也知道会快速 learn 并因此变好。看到 team 如何处理，我从未如此自豪。

That's a big part of excellence in the Netflix culture, which is being very comfortable with risk-taking. We don't try to avoid failures, we try to recover quickly when we have them. I think there's been great examples of that. Our foray into live was a wonderful example of being comfortable taking a ton of risk, knowing it would be imperfect, knowing we would learn fast, and we would be better for it. I've never been prouder of the team seeing how we worked through that. 因此需要 talent density，相信人们能使用 context、strong judgment 与 risk-taking，为 business best outcome 而战。还要非常明确，所做事情必须为 consumer 与 Netflix 带来 outcome：Netflix matters，Netflix member matters，而不是个人 success 或 preference。

So, you have to be talent density, comfortable that people are going to take the context that you give them, strong judgment and risk taking, and fight for the things that are the best outcomes for the business. 这里有 selflessness。另一个要求是适应一些 human 很不自然的事情。

You have to be very clear that what you're doing is driving outcomes for consumers and Netflix. So, it's Netflix matters, Netflix members matter. It's not about my own personal success or what I prefer. So, there's a selflessness that is part of this excellence operating system. And then the other thing I would say is some of the things that are they're really unnatural for humans to do. 例如有些日子我看到 decision，觉得自己会做不同选择，怀疑那是否最好。但在 Netflix culture 中，我的职责不是每次都介入、overrule、veto 或质疑，尤其事情不 material、不会把公司烧毁时。

So, I could give a couple examples of things to get comfortable with,which is there are certainly days where I see decisions happening, and I think, "Hmm, I would make a different decision." Like, is that really going to be the best thing? But, my job, especially in the Netflix culture, is not to step in in every one of those cases and overrule or veto or question someone, especially if it's not material, it's not going to burn the place down. 应让人做 decision，再事后 reflection：进展如何？也许我错了，decision 很棒。这与 risk-taking 相关，要帮助人适应自己做 decision，即使其中一些会错误、带来困难 learning。

Let people make that decision and learn from it. And ask for those reflections afterwards of like, "How did it go? Maybe I was wrong. Maybe the decision was a great one." But, that it's related to the risk taking and the like help people learn how to feel comfortable making their own decisions, especially when they're not all going to be the right decisions, and they're going to learn something tough from it. 我的 boss 与 peer 也对我说过：“这是你的decision，我们可以给 input、帮 brainstorm，但最终属于你。 ”stakes 很高、我又对 organization 负责时，允许别人承担 risk 并不自然，会令人不适。另一个例子是，事情不顺时不要假设 process 会修复。

I felt that myself from my boss and my peers saying, "This is your decision. You know, I can provide input. I can help you brainstorm. It's yours in the end." And I that it it just doesn't come naturally when the stakes are high, when I feel responsible for what the org's doing to let people lean into risk can be uncomfortable. And I think that also means in cases where things are not going well as another example to not assume that process is going to fix it. 过去几年我学到，planning 很困难，没人找到 perfect planning；feedback、leveling、compensation 也没有 perfect process。每次遇到困难就加更多 process，只会花更多时间，outcome 却没有变好。

So, if or something I've learned over the past few years, that when planning is difficult, I've never heard someone say like, "Oh, we figured out the perfect way to plan." Or the perfect way to go through feedback and leveling and compensation. But every time we saw that and we added more process, we spent more time without getting better outcomes. 人的本能是遇到复杂 problem 就加 constraint，以为这样能简化，但它会妨碍寻找更 creative 的 planning、people decision 或 prioritization 方法。

And so, it's another unnatural thing that I think everyone's inclination when things are hard and complicated is you think you're simplifying the problem by putting a lot of constraints around it, but it actually goes against the like, is there a more creative way to plan or to make people decisions or to make prioritization decisions that actually get us to better outcomes. 要抵抗大公司通常会做的事，并经常在这种不适中保持自在。 Netflix 很多人都会感到，因为我们努力不做 standard thing。主持人：说起来容易，实际有人犯错时，人们会问哪里出错，再加 process 防止重演。

And so, it's a resistance to do the thing that a lot of bigger companies would do and to feel comfortable in that discomfort very often. So, that's something I feel in my role and I I would believe a lot of people at Netflix feel it because you try not to do the thing that is standard. It's easy to say that and hear that, but I so know what you mean, where somebody screws up and you're like, "Okay, what was the thing that went wrong? 你的意思是要抵抗，因为它减慢速度，best people 也不想在充满 checklist、process 与 gate 的地方工作。嘉宾：best people 想知道会有 blameless retro，并会强烈感到个人 responsibility，主动问怎样避免重演；不是加 process，而是分享 learning、改变 working method，下次获得更好 outcome。

Let's put a process in place to avoid this from happening." And what you're saying is like, you need to resist that. Uh because that slows things down and the best people don't want to be working in a place with all these checklist and process and gates and things like that. 信任人们反思、学习和成长，长期会得到更好outcome 与更强 team。

No, I think the best people want to know there's going to be a blameless retro and they're going to feel so individually responsible that they're going to say, "How do I make sure this doesn't happen again?" Not with process, but with like how could I share these learnings? How could I do work differently to make sure that I get to a better outcome next time? leader 的工作是培养 resilient、durable、知道怎样产生 impact 的 team，而非控制一切。主持人：这也是建立 high talent density team 的关键。

When you are trusting people to take those reflections and learn and grow I think you get much better outcomes over time. You get a much stronger team, which I think is part of our role as leaders of like you're trying to grow a team that is resilient and durable and knows how to have great impact. You're not trying to control everything. Which is a key to building a team with high talent density. There's two sides to this that I want to chat about briefly. 我想简短聊 hiring 与 retention 两方面。 Netflix 以 keeper test 著名，上次谈过，这也是 human 不自然的事情。想了解的人可听第一次对话。过去几年 keeper test 怎样演变，是否仍是 culture 核心？

One is the hiring and the other is keeping the people. So you're famous for the keepers test. We talked about this last time. Another unnatural thing for people. People that want to understand what this is, they can listen to the first conversation, but has that How has that evolved over the last couple years? That's still core part of the culture, this idea of the keepers test? 嘉宾：人们常把 keeper test 理解成决定让某人离开、认为不适合 role 的那个 moment。但同样常见的用途是谈一个人多么 extraordinary、在 role 中表现多好。入口是我问 direct report，或他们问我：“我在你的 keeper test 中表现怎样？

It's often cited in a way where you think of keepers test as that moment where you decide to let someone go, that they're not the right fit for the role and the conversation about that. But it's equally commonly used to have a conversation about how extraordinary someone is. How well they're doing in a role. ”大多数时候我的回答是：“我会非常努力留住你。 ”然后说明他们做得好的事、strength、impact，以及怎样更好。

Because it the entry point is for me to say to one of my direct reports or for them to say to me "How am I doing on your keeper test?" And the lion's share of the time my response is, "I would fight so hard to keep you." Let me go through a set of things that I think you're doing such a great job at, what your strengths are, where you're having a lot of impact. Here's how you could be even better. 所以它可成为 positive、uplifting conversation，虽然 framing 是“我是否通过 keeper test？ ”当然也有困难情况：我评估某人未通过，或对方问我时，我必须诚实说“现在没有通过”。

So it's an entry into a conversation that is very positive and uplifting for people, but the framing is, "Do I pass the keeper test?" And then, of course, there's the harder situations where I'm evaluating does someone pass the keeper test or they're asking me, and it's This is the toughest thing to say, to be honest, you're not passing that right now. I think you could get there in some cases, and that comes with feedback and what are those milestones? 有时认为还能达到，就给 feedback 与 milestone；有时则是说，我们已经努力尝试，但看不到成功 path。它只是 anchor 与 conversation entry point，可以通向多种方向。我喜欢它，因为能保持 feedback 与 check-in 的 good hygiene，迫使人们进行有时很艰难的 conversation，而不是回避。

Or in some cases you're saying, we've really tried and I don't see the path to success. So, it it's just it's an anchor and an entry point for a conversation that can go lots of different directions. And the thing I like about it is it's good hygiene on feedback and checking in on how things are going and forcing a tough conversation sometimes instead of shying away from it. Or to keep great talent, you do need to say you're doing great. 要留住 great talent，也必须告诉对方做得很棒，让人感到 recognized 与 valued。它并非只有 negative view，也有 positive side。

Like that that's an important part of making peo- people feel recognized and valued. So, I don't want it to come across that we just have this very negative view of it. I think there's this positive side of the coin as well. Awesome. I guess just to explain to people what this is so they don't have to go listen to other podcasts, I'll try to briefly explain it. 主持人：简要解释一下：Netflix culture 要求 manager 对 report 持续思考，“以今天已知信息，我还会聘用此人吗？ ”若答案是否定，可能应让他离开，以保持 high bar，而不是因人已在 team 就将就，对吗？

The idea here a part of the Netflix culture is that when you have people reporting to you, you should always be thinking, if I were to would I hire this person today? 嘉宾：对；corollary 是，若此人今天说要离开，我会不会努力挽留？如果我的感受是 relieved，觉得换个人更好，那我早该

Knowing what I know about them, and if not, then I should probably let them go. And the idea there is to keep the high bar, to not ever just like settle, okay, this person they're here, I guess we'll keep them around. Is that is that roughly the way to understand it? Yeah, and the way it it can it's sort of a corollary to that if that person came to me today to say they were leaving, would I fight to keep them or not? 采取行动、开始 conversation。主持人：为了维持这种 culture，要做很多不舒服的事。嘉宾：keeper test、talent density、leader 的 context not control，以及 highly aligned but loosely coupled。

Or would I say, if if my sense is relief of, oh yeah, it probably would be better to have someone else in this role, I should have taken action in having that conversation sooner. I love as you said, it's such an so many uncomfortable things you have to do to maintain Yeah, it's the Well, the keeper test is one, maintaining talent entity, context not control among leaders. 我们只用最轻 process，确保 priority 清楚并可 execution。这些都不是 human 或 scale organization 自然会做的事，需要持续 diligence，才能保住 Netflix 的 special quality。

We talk about being highly aligned but loosely coupled, which is where light process, you know, the minimum to make sure we're clear on the priorities and we can execute them as what we're solving for. 最终吸引并留住人的，是 work 与 culture；business success 也依赖它。主持人：这正是我要谈的。

All of these things are not things that human beings or organizations at scale tend to do. So, it's constant diligence to try to maintain the thing that's made Netflix a special place. Cuz in the end, it's the work and the culture that attracts people and retains people, and we need that to be a successful business. So, that's exactly where I was going to go. Uh so, to make this work, you need to attract the best people. It's always been very hard to attract the best people. 要运作这种体系，必须吸引 best people。过去就很难，如今有大量 capital、知名 AI lab 和激烈 competition，似乎更难。怎样说服 top talent 仍加入 Netflix，而非其他 fancy place？嘉宾：我们一直面对 talent competition，现在也许更明显。

Feels insanely hard these days with the amount of dollars flying around, the fancy AI labs, so much competition. There's like everyone's just, you know, it's it's crazy. What have you found to be uh effective in convincing the top people to still come to Netflix and and join versus all the other fancy places they can go? Yeah, we've always had a lot of competition for talent. It might feel more pronounced right now, but we we have great talent on the team. 但 Netflix 已有 great talent，包括 recent hire 与 long-tenured employee，team 的工作总让我印象深刻。所以我不觉得我们受损，也不觉得其他 company 吸走所有优秀人才，很多仍在 Netflix。我觉得应该明确说出来，因为我确实相信。

Maybe that goes without saying, but I feel like I should say it out loud cuz I believe it. We have incredible talent at Netflix, recent hires, long-tenured people. I'm always impressed by the work that the team is doing. So, I don't feel like we've suffered or like other companies are vacuuming up all the good people because so many of them I do think sit at Netflix. Netflix 拥有不可思议的 talent，包括 recent hire 与 long-tenured employee，team 的工作总让我印象深刻。所以我不觉得我们受损，也不觉得其他 company 吸走了所有优秀人才，因为其中很多人就在 Netflix。

It does feel like we have to be more more explicit about the types of people and talent that tend to thrive at Netflix versus other companies like some of the frontier labs. So, people at Netflix have to be passionate about the application of technology. And the application or building products to solve a certain set of problems. You have to love entertainment. You have to love consumer products at scale. You have to love the global nature of that. 但我们似乎需要更明确说明，怎样的人更适合 Netflix，而非 frontier lab。 Netflix 的人要热爱 technology application，以及 build product 解决特定 problem；要喜欢 entertainment、scale consumer product 与 global nature。

There are a lot of incredibly talented people who love that sweet spot. I am one of them between tech and product and entertainment and how do you make those things come together in a way that's remarkable? And you use AI to do it. You use other technologies and products to do it. But that has to be something that drives you to be really excited about a lot of the roles at Netflix. 有很多极有才华的人热爱这个 sweet spot。我自己也是其中之一，着迷于 tech、product 与 entertainment 怎样结合出 remarkable outcome；会用 AI，也会用其他 technology 和 product 来实现。

If instead you're by some of the foundational work that the frontier model companies are doing, which is exciting in its own way, it's a different persona. It's a different like here's the problem space that I want to work in. But I don't think there's a shortage of people who get really excited about the applications of the technology and see the connection to that to things that they love and use every day like Netflix. 许多 Netflix role 都需要这种动力。若你反而更被 frontier model company 的 foundational work 吸引，那种工作也以自己的方式令人兴奋，但属于另一种 persona 与 problem space。

And so that, you know, that gets me up in the morning and I think it gets a lot of the team members up and we have this conversation about like that's something special that only talent at Netflix can do or fill in the blank for another industry that's deep in the application of it. I think that's inspiring. I want to kind of touch on a couple things that I've been thinking about in this world of AI that we're uh approaching. One is uh junior people. 我不认为热爱 technology application、又能看到它与 Netflix 这类日常喜爱和使用事物之联系的人会短缺。这让我和许多 team member 每天充满动力。我们也会谈到，有些 special work 只有Netflix talent 能做；其他深入应用 AI 的 industry 也可替换进来。这很 inspiring。

主持人 It feels like everyone's like there's a good example. You're hiring a lot of awesome senior people that have proven they're awesome and you know, high talent density, high bars. Uh also just AI makes it so easy to do stuff that people may not be learning how to do anything. They're like junior engineers I'm thinking or junior PMs, junior designers. Like there's just like how do new people become these awesome senior people? Is there anything you've you think about? 主持人：我想谈 AI 世界的几个问题。第一是 junior people。 Netflix 招许多 proven senior talent，强调 high talent density 与 high bar；同时 AI 让做事变得很容易，人们可能根本没学会底层能力。我想到 junior engineer、junior PM、junior designer：新人怎样成长为优秀 senior？

Are you hiring junior people? How do you think about this if this what happens with junior people not necessarily learning or having a path to learn to become the senior person? We are still hiring junior people and they're really important to our talent strategy. So, we still have an intern program, we still have a new grad program, which is a was new for us as of a few years ago. 你们还招聘 junior 吗？若他们缺少学习或成长 path，该怎么办？嘉宾：我们仍招聘 junior，他们对 talent strategy。

So, prior to a few years ago, we were only hiring more experienced talent across all the functions. Now, we do hire people straight from undergrad and graduate programs and we'll continue to do that. So, even in a world of AI where some things are easier, we were talking earlier about mindset, AI fluency. From my experience, younger folks are more open-minded. They tend to be more native in some of these new ways of working. 我们仍有 intern program，也有几年前新设的 new-grad program；此前所有 function 只招 experienced talent，现在会从 undergraduate 与 graduate program 直接招聘，并将继续。即使 AI 让一些事更容易，前面谈到的mindset 与 AI fluency 仍重要。

For a company that like Netflix, they're also very fluent in how entertainment is changing, how consumer behaviors are changing, how product and tech is influencing that in the products that they're using. That's really important to have on our team. 以我的经验，年轻人更 open-minded，也更 native 地适应新工作方式。对 Netflix 来说，他们也很了解 entertainment、consumer behavior 怎样变化，以及 product 与 tech 如何影响自己使用的产品。

So, there there's the part of the persona, which is who are you as a new grad who's an engineer, but there's also who are you as someone who's in their early 20s and has a perspective on the world that is highly valuable and a comfort with the way the world is changing. So, that's why I say it's a critical part of our talent strategy. 这对 team 很重要。 persona 不只有“你是怎样的 new-grad engineer”，还有“作为二十岁出头的人，你怎样看世界”，这种 perspective 与对世界变化的适应非常有价值。

To the Okay, so you step into the role and you have AI tools that didn't exist 5 or 10 years ago, I would say mastery of the craft is still very important. So, going back to as the team member, I'm responsible for the quality of code that I am submitting for production, I'm responsible for the quality of products that I'm building, how they are designed, what that user consumer experience is. None of that is going away. 因此 junior 是 talent strategy 的 critical part。进入 role 后拥有五到十年前不存在的 AI tool，但mastery of craft 仍重要。

So, if I think about more junior or earlier career talent on the teams, we need to be investing just as much in the mentorship of this is what good looks like, this is how you use these tools, but you still take accountability for what the outcomes are, what the quality of the output it And I think I mentioned this earlier, I find that mastery and that craft excellence scarce still. So, we want to make sure we're teaching that. team member 仍对提交到 production 的 code quality、所建 product、design 与 consumer experience 负责，这些不会消失。对 early-career talent，仍要同样投资 mentorship：什么是 good、怎样使用 tool，同时对 outcome 与output quality 承担 accountability。

I I think it's a valid concern of like, how do I get that if I'm not as hands-on as I would have had to be, but you still carry responsibility for reviewing code, testing code, being able to diagnose problems, knowing what a good product looks like. Like, I think that's a very scarce skill to say, "This is excellence in in a product that solves a problem that matters and in how it's designed." So, I don't think that craft mastery, the importance of it, is going away. 如前所述，mastery 与 craft excellence 仍稀缺，必须教授。我理解担忧：若不再像过去那样 hands-on，怎样获得这些能力？但仍需 review code、test code、diagnose problem，并知道 good product 什么样。

Probably the way we train and grow talent has to change cuz they're going to use different tools, and I can guarantee you that earlier career talent is going to be teaching older folks like me many new things, too. So, I think it goes in both directions. Where do you think engineering goes in the I don't know, 5, 10 years? Do you think people need to still understand code? 识别“这是 excellence：product 解决重要 problem，design 正确”仍是稀缺 skill。所以 craft mastery 不会消失；只是训练和培养 talent 的方式必须变化，因为会用不同 tool。

Or do you think there's this abstraction layer that sits on top where you don't even have to learn C++, Java, Python, whatever? I think there's a difference between being able to write lines of code in a particular language like Python or C++ and understanding how code, computer systems, products work. And I don't think the latter is going away. 当然 early-career talent 也会教我这样的 older folk 很多新东西，这是双向的。主持人：五到十年后 engineering 会怎样？人们仍需理解 code 吗，还是有一层 abstraction，根本不用学 C++、Java、Python？

Because if we trusted agents to know all the languages and write all the code, we're not going to know why is something Is it a good product? Is it a bad product? Is it working as we expected when it doesn't? Like I mentioned earlier, we take a lot of risk. We fail fast, we recover fast. That requires an understanding of how are these systems working. 嘉宾：能用特定 language 写 code line，与理解 code、computer system、product 怎样工作，是两回事。后者不会消失。如果完全信任 agent 掌握所有 language、写所有 code，我们会不知道为何某个 product 好或坏、是否按预期工作，出问题时也不知道原因。

I might use an agent to help me understand those things, help me detect an anomaly or something that's broken faster and triage it, but I still need to have a fluency of like, what is this thing that we're building and how does it work? So I know if it's good and I know how to fix it. I don't know I I hope that doesn't go away cuz it you know, that that's like a how do we make the world a better place through the stuff that we're building? 前面说 Netflix 承担很多 risk，fail fast、recover fast，这要求理解 system 怎样工作。可以用 agent 帮助理解、检测 anomaly、快速 triage broken system，但仍要理解正在 build 什么、怎样工作，才能判断 quality、知道怎样 fix。

I think requires some understanding of what we've built. What I'm hearing which it makes sense is you may not have to write the code but you have to understand it and what's happening. But it's so much harder to just as a person not writing it to actually you know, have that instilled in you. I think that's one of the the things that the learning curve is very steep on right now. 我希望这不会消失，因为想通过所建事物改善 world，就需要理解自己造了什么。主持人：也就是不一定亲自写 code，但必须理解发生了什么；不亲手写却要真正内化理解，会困难得多。

So looking at some of the code that some of these models or agents are writing they're very hard to follow. It's like I know I'm getting better performance from this but I have no idea why and if this thing breaks I'm going to have no idea how to fix it. That that's makes me uncomfortable. You know, maybe that's because I'm still on that learning curve of like how do we operate in that world? 嘉宾：这是当前 learning curve 最陡的部分之一。看 model 或 agent 写的一些 code，很难 follow：我知道 performance 变好，却不知道为什么；一旦 break，也不知道怎样 fix。

Like what's the set of tests or rationalization and understanding that we need to have to get comfortable with it? But at first glance it looks very unfamiliar and very unsettling. So I think engineering over time will evolve to be comfortable with that and have fluency in it and know how to guide new tech and agents and new capabilities to make sure that we feel really good about what the output is. 这让我不安。也许因为我仍在学习怎样在这种世界 operate：需要哪些 test、rationalization 与 understanding 才能放心？第一眼看它非常 unfamiliar、unsettling。 engineering 会逐渐演变，掌握这种 fluency，知道怎样引导 new tech、agent 与 capability，确保 output 令人信任。

I wonder what the metaphor is for this where this like it's I continue to be astounded by how much engineering has transformed in like 2 years. It's like a completely different drop down. You're just used to sit there and then I would write code and now you're just talking to agents and reviewing code and shipping a bunch PRs a day. It feels like it's a it's an acceleration of how much engineering has changed. 主持人：很难找到 metaphor。两年内 engineering transformation 令人震惊，像完全不同的职业：过去坐着写 code，现在与 agent 对话、review code、每天 ship 多个 PR。

But if you looked over the last 10 years or 20 years you would say the same thing. Mhm. So it there's just something that's moving faster and it's hard to wrap our heads around how quickly it's moved in the past couple of years but it's not it's not totally unfamiliar that engineering or data science or product would have these big shifts, just like how filmmaking works. 嘉宾：这似乎是 engineering 变化速度的 acceleration，但若看过去十年或二十年，也会说同样的话。只是这次更快，难以理解过去几年移动速度。但 engineering、data science、product 发生大 shift 并不陌生，filmmaking 也一样：一百年来因为 technology 与 new tool 已完全不同，只是 cycle 加速。

主持人 If you will go over the last 100 years, it's unbelievably different because of technology and new tools that we brought to it. Just feels like the cycle is speeding up. Okay, I want to talk about entertainment for for a brief moment. Just I'm curious just like how entertainment will change over time and say like 5, I don't know, 5, 10 just you know, today we open up Netflix, check out some shows, watch some videos. 主持人：简短谈 entertainment。如今打开 Netflix、选 show、看 video 的方式似乎很久没变；也有 TikTok、Instagram feed。

It hasn't changed in a while, just that idea of like cool, I'm going to watch the pit and watch it all. I'm going to watch a movie. Uh I got TikTok, I got Instagram feeds of stuff. Like how much different do you think this will be in I don't know, 5 years? The way we entertain ourselves. it's already changing at Netflix because entertainment is not going to be one thing in the future and it's already not one thing now. 五年后娱乐自己会有多不同？嘉宾：Netflix 已在变化，因为 future entertainment 不会只有一种，现在也不是。我们超越 film/TV，是因为 consumer 期待跨 format、device、day moment 的更大 variety；Netflix 必须满足并最终超越 expectation。

So, part of the reason that we are going beyond film and TV in our offering is because there's there's an expectation that consumers have of much greater variety across formats, devices, moments of the day that Netflix needs to be able to serve well in order to meet consumer expectations and hopefully exceed them over time. 因此我们加入 mobile/TV game、cloud game、live content、podcast，与更广 creator 合作，扩展 entertainment 的含义。

So, when we think about the addition of mobile and TV or cloud games live content podcasts, working with a broader set of creators who are now on the Netflix service all of those things create a greater breadth of what entertainment is and Netflix is able to define and expand that. And it puts a higher bar expectation on how do we make sense of that for a Netflix member? 这也提高标准：怎样让 Netflix member 理解和顺畅使用这些内容？怎样提供 seamless journey：从听Bill Simmons Podcast，到因为喜欢 Netflix 的传统 film/TV offering 而看《Quarterback》，再到玩最新 FIFA cloud game；既在 TV，也在移动中的 mobile phone，能在一天不同 moment 发现并参与 content。

So, how do we show you this very seamless journey from I listen to the Bill Simmons podcast to I watch Quarterback because I love that as one of the Netflix offerings in the more, you could say, traditional film or TV space to I play the most recent FIFA cloud game. And I want to be able to do that in both TV and on my mobile phone because now I'm on the move and I want to be able to discover and engage with the content at different moments of the day. 这条 journey 已在构建。它会越来越强。 future entertainment 不会只有一种，会更 personalized、immersive、interactive，像一个可按当下需求向不同方向探索的 world。

That's already a journey that we're building into Netflix, which I think will become stronger and stronger over time. So, the future of entertainment isn't going to be one thing and it's going to have to be more personalized, more immersive, more interactive with this sense of this is a world that I can explore in lots of different directions depending on what I'm looking for in the moment. Netflix 的 challenge 是让 discovery 与 engagement 比今天容易得多。 content 很多，也可能很 fragmented，尤其把所有 service与 offering 算在一起。

And that the challenge Netflix has is we've got to make discovery and engagement much easier than it feels today. We have tons of content and it can feel very fragmented, especially when you consider all the services or offerings out there. And I I think Netflix is very well positioned to understand how to solve that problem across entertainment, product, and tech. The other element of this is AI, obviously. Netflix 很适合跨 entertainment、product 与 tech 解决这个 problem。主持人：另一个因素当然是 AI。 tech 圈普遍说 AI 是未来、非常喜欢；Hollywood 则常说关掉它。嘉宾：其实有很广的 spectrum。

As an outside observer, it's like so interesting to see how in tech, it's like AI, I love it. It's the future. It's the best. In Hollywood, it's like, "No. Shut it down." There's There's a mix. There's a very wide array. So, we Netflix's role in this is to enable creators with whatever tools they want to use to bring their vision to life. There are going to be some creators or filmmakers who are on the end of the spectrum that says, "Absolutely not. Netflix 的角色是为 creator 提供他们愿意使用的 tool，帮助实现 vision。有些 creator 或 filmmaker 会明确说绝不用 AI，因为不符合 production 方法与 vision；没问题，我们与他们合作。

No AI. That is not how I do production. It's not It's not consistent with my vision." That's fine. We work with those creators. There's other creators a growing number of them, I would say,who are very interested in exploring, "Wait, can these gen AI tools make something possible that wasn't possible before? Can I tell a story in a new way? Can I make that story higher quality and more resonant for audiences? 另一端有越来越多 creator想探索 GenAI 能否实现以前不可能的事情、以新方式讲 story、让故事 quality 更高、与 audience 更 resonance，或更 creative 地呈现。我们也支持他们，以及中间所有人。这是 Netflix 很重要的位置，因为 entertainment 不会只有一种 format。

Can I do things that are extra creative and how I think about bringing a story to life? And we support them as well, and we support all the folks who are in the in-between. And then that's a really important position for us to be in again, because entertainment is not going to be one thing. There's not going to be one format. 会有传统 film/TV，也会有优秀 creator 带来的全新 format，Netflix 希望参与。因此需要 flexible tool 与 partnership，以 creator enablement 为视角，而非 prescriptive 地规定只能用一种方式。

I think there's going to be types of film and TV that feel traditional, and then there's going to be entirely new formats that unbelievable creators help to bring to life, and Netflix wants to participate in that. Which means we need to have a flexibility in the tools that we provide and the types of partnerships we have, and to really have a creator enablement view rather than a prescriptive that we only do this one way. 主持人：人们会惊讶 AI content 有多好。 Spencer Pratt 的 video 就很 entertaining，虽明显由 AI 生成。会不会出现整部 TV show 都是 AI，观众仍喜欢？嘉宾：我很难想象 entertainment 不以 human 为核心。

Elizabeth Stone I think people are going to be surprised by just how good AI content is. Like Spencer Pratt's videos are just like everyone's like, "Wow, this is entertaining." Obviously AI, but it's so interesting. Do you think Do you think we'll get to a place where it's just like whole TV shows are AI and people love it? I have a hard time picturing entertainment that doesn't have humans at the heart of it. human 会参与 storytelling creation，而 storytelling 是稀缺且 valuable 的 skill。讲故事与 humanity 同源，要理解什么能连接 people。 screen 上缺乏 humanity 的 character 对我吸引力较低。

So that that's humans in the creation of the storytelling, which I think is a scarce and valuable skill. Yeah,storytelling is one and the same with humanity. And like knowing what connects with people. So I think humans will be part of the always be a core part or a critical part of the story. And I think watching characters on screen who don't have that humanity feels less compelling to me. storytelling 的力量在于看到另一个 human，观察他怎样 perform role、呈现 emotion，这是深刻的 human element。 AI 会帮助实现，也会在一些 production 或 visual style 中发挥 material role，但我看不到完全没有 human backbone 的版本。

And what the power of storytelling really is, to like see another human and to watch how they perform a role or like bring an emotion to life. That's such a human element. Will AI help to bring that to life? Will play a material part in some of those productions or how we get them to look and feel a certain way? Yeah, definitely. But I don't see the version of it that doesn't have the human as the backbone. 主持人：有句可能误归于 Salman Rushdie 的话：child 出生先要 food、water 与 protection，然后会要求“给我讲一个 story”。嘉宾：从人类起源开始，storytelling 就是 community、social network、human feeling 与 connection 的关键。

There's a quote that I think is misattributed to Salman Rushdie, which is when a child is born, they first ask for food and water and projection, and then they ask for tell me a story. It's a thing going back since the beginning of time that storytelling has been a key part of community and social networks and human feeling and connection. 我喜欢 technology 能 amplify 它，以全新、novel、exciting 的方式呈现；但若 storytelling 不以 humanity 为中心，就会缺少某种东西。主持人：未来几年会出现很 wild 的东西。嘉宾：一定会，而且许多会很 entertaining。

So, I love the idea that technology can amplify that and can bring that to life in very new, novel, exciting ways. But, if to say storytelling wouldn't have that humanity at the center feels like something would be missing. Mhm. We're going to see some wild over the years coming out of this. Oh, I'm sure. There's no question about that. And a lot of it could be very entertaining. You know, I I don't debate that, either. 但会有 broad range，Netflix 要处在塑造并呈现它的中心，这就是计划。主持人：我们聊了很多。进入 lightning round 前，还有什么想留给听众，或再次强调？嘉宾：这应该已经贯穿全程：现在是构建 entertainment product 的极好时期。

But, I think there's going to be a broad range, and I think Netflix needs to be at the center of shaping that and bringing that to life, which is our plan. Amazing. Well, we covered a lot of ground, Elizabeth. Uh before we get to our very exciting lightning round, is there anything else that you wanted to share, leave listeners with, maybe double down on from things we've talked about? 它已经贯穿全程，但我仍想强调：现在是构建 entertainment product 的极好时期。 technology、consumer 以及 entertainment 的定义都在变化，我们处于不可思议的 high-velocity innovation period。

It probably came across throughout, but I I would underscore that this is a really exciting time to be building products in entertainment. Everything we talked about of like what's changing in the tech and consumers and like what is entertainment we're at this unbelievable high-velocity innovation period. So, it's what keeps me at Netflix. I think it's a fun place to be. I would be missing something if I didn't reinforce that I think that's true. 这让我留在 Netflix，也让这里很有趣。我若不再次强调这一点，就像漏掉了什么。 industry 有时花太多时间谈 pure tech 或 capability，反而见树不见林。

Um I also think that as an industry we spend a lot of time sometimes talking about the the pure tech or the the capability and we sort of lose the forest for the trees. We're trying to build great consumer products that people love. We're trying to make great entertainment that people love. 我们要 build 的是 consumer 喜爱的 great product 和 entertainment，而且要成为他们最喜欢的东西。底层当然有很棒的 tech 与 product work，但最终最 inspiring 的是我们给世界各地的人带来什么。

And it's their favorite thing that I don't want that to be lost in Of course, there's amazing tech and product stuff that sits underneath, but in the end, the thing that's most inspirational is what do we bring to people around the world? And on those lines, there's been such a uh the opposite of glut, a drought of consumer new consumer products, consumer experiences. Like there's very few success, like almost no consumer startup works. 主持人：沿着这个思路，新的 consumer product 与 consumer experience 长期不是 glut，而是 drought；几乎没有 consumer startup 成功。

Uh and AI feels like an opportunity for something else to work and I feel like Netflix is one of the rare companies and brands that continues to deliver an awesome consumer product and business. There's just not that many of them. Yeah. We're going to keep that up. Well, with that, we reached our very exciting lightning round. We've got five questions for you. Are you ready? Okay, I'm ready. AI 似乎提供了让新事物成功的机会，而 Netflix 是少数持续交付优秀 consumer product 与 experience 的 company，也是少数出色 consumer business。嘉宾：我们会继续。主持人：进入五个问题的 lightning round。准备好了吗？嘉宾：准备好了。

All right. What are two or three books that you find yourself recommending most to other people? I mean, I have to come up with different books than I said last time. I don't know. But I I think that sounds great. I still like a good throwback. So, two that are coming to my mind Into Thin Air, Jon Krakauer, and Liar's Poker, Michael Lewis. So, I I worked on Wall Street and I like reminding people what it was like in the way back time. 主持人：你最常向别人推荐的两三本 book 是什么？嘉宾：我得想和上次不同的书。两本想到的老书是 Jon Krakauer 的《Into Thin Air》和 Michael Lewis 的《Liar's Poker》。我在 Wall Street 工作过，喜欢提醒大家遥远过去是什么样。

Favorite recent movie or TV show you really enjoyed, which is maybe too hard for someone working at Netflix, but I'm going to see what comes out. The The list is very long. Um the most recent I watched, Remarkably Bright Creatures, after a recommendation from my mom. It's a tearjerker. Talk about the human part of storytelling. Favorite product you've recently discovered that you really love? Critical for my health and well-being, Eight Sleep. 主持人：最近真正喜欢的 movie 或 TV show 是什么？嘉宾：list 很长。最近在妈妈推荐下看了《Remarkably Bright Creatures》，很催泪，体现了 storytelling 的 human 部分。主持人：最近发现并很喜欢的 product？

Do you have a favorite life motto that you often come back to in work or in life? I often go back to the things that my parents instilled in me in very early times. So, the the risk of repeating, maybe. First, something good happens every day. Watch for it. Even in the most stressful times. And second, that the last 5% of effort usually makes all the difference. These are awesome. I They hit They hit me. Final question. 嘉宾：对健康与 well-being 很关键的 Eight Sleep。主持人：工作或生活中常回想的 life motto？嘉宾：我常回到父母很早教我的事情。第一，每天都会有好事发生，要留意，即使最 stressful 的时候；第二，最后 5% 的 effort 往往决定一切。

I don't know what anything about this, but you mentioned you're doing some kind of cycling event. Oh, yeah. Tell us Tell us what's going on. What are you doing here? So, my husband and I are doing a trip where we ride alongside the Tour de France for the last week of the race. So, the tour is 3 weeks. The last week has a lot of mountain stages. So, we get to ride part of the route each morning and then watch the race in the afternoon. Not for the faint of heart. 主持人：这些很棒，很触动我。最后一个问题：你提到某种 cycling event，是什么？嘉宾：我和丈夫会在 Tour de France 最后一周沿赛道骑行。赛事共三周，最后一周有很多 mountain stage；我们每天早上骑一部分 route，下午看比赛。

So, I'm trying to train up so I can enjoy those rides. It's supposed to be vacation after all. My god. I love this vacation. We're just going to a race. I love cycling. I love professional sports. It's fun to be able to participate in it. So, is this like racing or you just kind of try to go as nonchalantly through the course? You go nonchalantly. But, still there are I think I mentioned Yeah, it is It's physically and mentally challenging. 这并不轻松，所以我在训练，希望能享受骑行——毕竟是 vacation。主持人：天啊，我喜欢这种 vacation。嘉宾：我喜欢 cycling 与 professional sport，能参与很有趣。主持人：这是 racing，还是悠闲通过 course？

And I you know, it's not a race, but I don't want to be at the back of the pack. So, I got to be comfortable enough to hold my own. Wow. I love how different this is from your job. And it feels like something else to do. It's a good balance and it gets me outdoors and gives me some nice perspective. So, I'm looking forward to it. Elizabeth, you are awesome. Two final questions. 嘉宾：悠闲，但仍然有身体和心理 challenge。虽然不是 race，我也不想落在队尾，必须足够 comfortable、跟上大家。主持人：和你的 job 很不同，是一种平衡。嘉宾：对，能到户外，也带来 perspective，很期待。

Where can folks find you online if they want to follow you, reach out for maybe anything that came up? And how can listeners be useful to you? The best place to find me and some of the work we're doing or reach out is the Netflix tech blog, actually, where we're putting a lot of things that I've been talking about up there. We're trying to do a better job communicating about the fun stuff we're working on. So, that's a good first stop, usually. 主持人：最后两个问题，大家在哪里能 online 找到你进行 follow 或联系？听众怎样帮助你？嘉宾：最好访问 Netflix Tech Blog，可以看到我和 team 的工作，也可联系。我们正在更好地沟通有趣项目，会把很多刚谈到的内容发布在那里。

Um and then how listeners can be useful, try all the new stuff that we're putting out there. Um watch the live events, play the games, have fun with the new vertical video feed that we have on mobile called Clips, send us feedback. So, we want to make it better, and a lot of these things are new zero-to-one efforts for us. So, we're trying to get to great and excellent as quickly as possible. 听众可以试用我们推出的新东西：看 live event、玩 game、体验 mobile 上叫 Clips 的新 vertical-video feed，并发送 feedback。这些都是新的 zero-to-one effort，我们想尽快做到 great 与 excellent。

I love that the homework is go watch Netflix and You can also watch other things, tell us how we can be better, but I'm I'm definitely interested in how can we be better at Netflix. I love it. I'm going to go I'm going to go do that. Elizabeth, thank you so much for being here and being here again. Thank you for having me. Always fun. Thank you so much for listening. 主持人：homework 就是看 Netflix。嘉宾：也可以看别的，再告诉我们怎样能让 Netflix 变得更好。主持人：我会去做。 Elizabeth，感谢再次来。嘉宾：谢谢邀请，一如既往很愉快。

If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode. 主持人：感谢收听。若有价值，可在 Apple Podcasts、Spotify 或其他 podcast app 订阅，也请评分或 review，帮助其他听众发现节目。过去 episode 与更多信息可在LennysPodcast.com 查看。下期节目见。