Skip to main content
Back

Karpathy unveils end-to-end ChatGPT clone repo

Google's NotebookLM now with Nano Banana visuals, OpenAI making chips, Alibaba's Qwen Code gets Plan Mode, Microsoft's i

October 14, 2025 4 min read

Hey

Welcome to AlphaSignal, the most read source of news by AI engineers and researchers.


Every day, we identify and summarize the top 1% of news, papers, models, and repos, so you’re always up to date.


Here’s today’s roundup:

Summary

Read time: 4 min 39 sec

Top News

▸ Andrej Karpathy launches nanochat, lightweight framework to build small ChatGPT models

Signals

▸ Google updates NotebookLM Video Overviews with Nano Banana visual styles

▸ Alibaba rolls out Qwen Code upgrade with Plan Mode and vision intelligence

▸ Google adds new AI Studio dashboard to monitor Gemini API usage

▸ Microsoft debuts MAI-Image-1, its first in-house text-to-image model

▸ OpenAI announces custom AI chips to meet global model demand

Fiddler AI

▸ Monitor multi-agent systems with Fiddler AI’s free observability guide

Trending Reads

▸ OpenAI shares prompt packs for everyday workplace roles

▸ Build agents that handle days-long tasks without collapsing

▸ Learn how OpenAI measures and reduces political bias in LLMs

Top Paper

▸ MIT paper shows LLMs can learn and fine-tune itself without human input

Top News

Andrej Karpathy introduces nanochat, open-source stack for scalable chatbot models with pretraining, fine-tuning, and inference

alpha_signal_image_2

Andrej Karpathy has released nanochat, a minimal, full-stack repo for training and running a ChatGPT-like model from scratch. Unlike his earlier nanoGPT (pretraining only), nanochat includes everything pretraining, midtraining, SFT, RL, inference, and a WebUI in ~8,000 lines of code.

You can boot a cloud GPU box and, in ~4 hours, get a working mini-chatbot.


Key Features

  • End-to-end: Covers tokenizer, pretraining, midtraining, SFT, RL, and inference.

  • Minimal codebase: ~8,000 lines, dependency-light, clean and hackable.

  • Cost-efficient: Train a ChatGPT-like model for ~$100 on 8×H100 (~4 hrs).

  • Scales up: At ~$1,000 (~41 hrs), coherence improves on math/code/MCQ tasks.

  • Interfaces: Talk to the model via CLI or ChatGPT-style WebUI.

  • Report card: Generates a markdown summary of training metrics.

How It Works

  • Architecture: Llama-like dense transformer with rotary embeddings and QK norm.

  • Optimizations: Uses Muon + AdamW, multi-query attention, logit softcap.

  • Tokenizer: Trains a Rust-based tokenizer from scratch.

  • Pretraining: Runs on FineWeb and evaluates with CORE metrics.

  • Midtraining: Uses SmolTalk conversations, MCQs, and tool-use data.

  • SFT: Fine-tunes on ARC, MMLU, GSM8K, and HumanEval.

  • RL: Optional reinforcement learning with GRPO on GSM8K math.

  • Inference: Supports KV cache, prefill/decode, and Python sandbox tool use.

Performance

  • ~$100 run (~4 hrs): Surpasses GPT-2 on CORE, simple Q&A/story tasks.

  • ~$1,000 run (~41 hrs): Handles math/code problems and MCQs with coherence.

  • Depth-30 model (24 hrs): 40s on MMLU, 70s ARC-Easy, 20s GSM8K.

  • Coherence improves rapidly with scaling, approaching GPT-3-small behavior.

How to Get Started

  • Clone: git clone https://github.com/karpathy/nanochat

  • Spin up a GPU box (e.g. 8×H100 for speedrun).

  • Run the training script (pretrain → midtrain → SFT → RL).

  • Launch CLI or WebUI to chat with your model.

Access & Availability

  • GitHub repo available
  • Codebase: ~8,000 lines, open-source and designed to be forkable.

Try now

Signals

Google adds creative Nano Banana visuals and short-form Briefs to NotebookLM Video Overviews

2,132 Likes

Alibaba rolls out Qwen Code update with AI plan approvals and vision-aware model switching

1,49 Likes

Google ships a redesigned AI Studio dashboard for real-time Gemini API usage tracking

1,957 Likes

Microsoft announces MAI-Image-1, trained for photorealism and creative value, ranking ninth on LMArena

834 Likes

OpenAI confirms it is designing custom chips, adding to partnerships with NVIDIA and Broadcom to expand capacity

2,439 Likes

Monitor Multi-Agent AI Workflows With Fiddler AI’s Observability Guide

Fiddler AI’s free guide shows you how to monitor agent hierarchies, trace failures, and visualize metrics across complex systems.

Get step-by-step methods to improve visibility and control over AI agent workflows in production.

Download the free guide ↗️

Reads

OpenAI shares prompt packs for everyday workplace roles

2,432 Likes

OpenAI’s prompt pack shows how to apply ChatGPT to everyday workplace tasks. It includes role-agnostic examples for communication, meetings, decisions, and productivity, plus direct links to runnable prompts.

Build agents that handle days-long tasks without collapsing

1,153 Likes

Discover practical design patterns for building Deep Agents: orchestrators, sub-agents, and persistent state. These techniques let you run reliable, multi-step workflows far beyond the limits of shallow loops.

Learn how OpenAI measures and reduces political bias in LLMs

1,966 Likes

OpenAI presents a method to measure political bias across axes like escalation and coverage, showing GPT-5 reduces bias by ~30%. The evaluation helps you understand model robustness and objectivity

Top Paper

MIT paper shows LLMs can learn and fine-tune itself without human input

9,285 Likes

alpha_signal_image_3

MIT researchers present SEAL (Self-Adapting Language Models), a system that lets language models generate their own fine-tuning data and weight updates. This enables persistent adaptation without external supervision.


The model produces a “self-edit” in natural language that can restructure input, choose hyperparameters, or trigger data augmentation to guide its own learning.


Key takeaways from the research:

  • It applies the self-edits via SFT, making updates to its internal weights. 

  • It rewrites new information and updates its own weights.

  • Uses a reinforcement loop to reward effective self-edits.

  • Achieves +40% factual recall improvement over baseline.

  • Outperforms GPT-4.1 using its own generated data.

  • Suffers from partial catastrophic forgetting during updates.

This paper shows the first practical path toward self-improving LLMs that can continuously learn and adapt without human intervention.

Read now

At Alpha Signal, our mission is to build a sharp, engaged community focused on AI, machine learning, and cutting-edge language models, helping over 200,000 developers stay informed and ahead. We’re passionate about curating the best in AI, from top research and trending technical blogs to expert insights and tailored job opportunities. We keep you connected to the breakthroughs and discussions that matter, so you can stay in the loop without endless searching. We also work closely with partners who value the future of AI, including employers and advertisers who want to reach an audience as passionate about AI as we are.


Our partnerships are based on shared values of ethics, responsibility, and a commitment to building a better world through technology.Privacy is a priority at Alpha Signal. Our Privacy Policy clearly explains how we collect, store, and use your personal and non-personal information. By using our website, you accept these terms, which you can review on our website. This policy applies across all Alpha Signal pages, outlining your rights and how to contact us if you want to adjust the use of your information. We’re based in the United States. By using our site, you agree to be governed by U.S. laws.

Looking to promote your company, product, service, or event to 250,000+ AI developers?

Work with us