Nathan Lambert

🚀 DeepSeek-V4.1-Flash-1017 is here! Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a Pareto improvement in performance and cost. Now live on our website, app, and API. 🛠️ Optimized for @DeepSeekHarness, Tencent CodeBuddy, WorkBuddy, and more. Upgrade to DeepSeek Harness v0.2.1-rc.1 or later for the best experience with the new model. 1/4

MofCom said they will have a near auto licensing regime for open-source AI that allows more rapid release it seems? "To support the development of an open-source AI innovation ecosystem, automatic licensing and other facilitation measures will apply to technology exports involving open-source releases of AI technologies, Chinese companies building and operating their own computing centers overseas, overseas chip tape-outs and fabrication, joint R&D using overseas resources, and participation in international standard-setting—essentially allowing exports to proceed immediately upon filing."

Its got all the restrictions across all license eras 1. Built with muse branding nonsense that no one followed 2. Revenue gate like kimi/glm (reasonable) 3. Restricted domains / authorized use policy 4. All of the above applies to models trained on ai outputs (maybe)

1/ muse spark open is the most capable US open-weight model to date. 816B total params, 42B active. 1M-token context. natively multimodal. built for coding and agentic work like long-horizon tasks, hard software engineering, multi-step reasoning. america! 🇺🇲

Today we’re releasing Muse Spark Open, the most capable US open-weight model to date. Built for coding and agentic work, it excels at long-horizon tasks, complex software engineering, and multi-step reasoning. Key Model Specs: → 816B total / 42B active parameters → 1M token context window → Natively multimodal across text, image, video, audio

Nathan Lambert, former post-training co-lead at the Allen Institute for AI, now launching Trillium, a nonprofit open AI lab, joins a16z's Lisha Li to discuss whether open models can really keep up with the frontier. When he tells researchers in China that an average OpenAI researcher reportedly has five thousand H100s to work with, "their jaw just falls to the floor." In his view, Chinese labs have the talent but are mostly short on compute. He also thinks the distillation story is overblown. Based on the research, he puts its edge at a few months, and says "there's a bit of cope" in thinking only American labs can build good models. A lot of frontier progress outside math and code comes down to labs buying 1) data and 2) RL environments. It's a multibillion-dollar market, and some of it is sloppy enough to teach models to reward hack. That's a big part of why Trillium exists. Nathan wants to give away the environments other people would sell, and show exactly how his team uses AI to build AI, including where the models still get stuck. His bet is that the more people can see how this works, the better prepared everyone is for what's coming. 0:55 Can open models keep up with the frontier? 2:05 What it takes to reach the frontier 3:55 Open weights and the fast-follower playbook 8:25 Chinese labs: equal talent, less compute 9:55 Why distillation is harder than it looks 14:55 Is distillation theft or competition? 16:20 The lab flywheel vs owning your weights 21:45 Can general models beat every specialist? 26:30 AI at 1% of today's cost in 4 years 27:15 The RL env boom and reward hacking 31:30 What Trillium will build in the open 35:30 Breaking into AI in the age of AI slop 38:05 Open-sourcing how AI builds AI 44:25 Why Nathan isn't pre-training 49:45 Agent swarms and why lab secrets leak 54:25 Fast takeoff vs risks already here YouTube: youtu.be/BKuK0M6oDTI @natolambert @trillium_labs @lishali88

We've been monitoring a decline in HuggingFace downloads across the top open LLMs we track for interconnects. It looks like there was a slight change in counting of downloads, as we saw a ~30% download rate drop for many top models, but this is continuing over time. What else do people think it may be?

Over the last few months, we’ve made significant progress in reducing the effectiveness of distillation attacks against Claude. In this post, we explain how: anthropic.com/research/distillation-defenses

Introducing Muse Wonder, our new family of frontier models with state of the art performance for personal and professional agentic work.