| Hey | Welcome to AlphaSignal, the most read source of news by AI engineers and researchers.
Every day, we identify and summarize the top 1% of news, papers, models, and repos, so you’re always up to date.
Here’s today’s roundup: |
|
|
|
Top News
| | Andrej Karpathy introduces nanochat, open-source stack for scalable chatbot models with pretraining, fine-tuning, and inference
|
| Andrej Karpathy has released nanochat, a minimal, full-stack repo for training and running a ChatGPT-like model from scratch. Unlike his earlier nanoGPT (pretraining only), nanochat includes everything pretraining, midtraining, SFT, RL, inference, and a WebUI in ~8,000 lines of code.
You can boot a cloud GPU box and, in ~4 hours, get a working mini-chatbot.
Key Features
End-to-end: Covers tokenizer, pretraining, midtraining, SFT, RL, and inference.
Minimal codebase: ~8,000 lines, dependency-light, clean and hackable.
Cost-efficient: Train a ChatGPT-like model for ~$100 on 8×H100 (~4 hrs).
Scales up: At ~$1,000 (~41 hrs), coherence improves on math/code/MCQ tasks.
Interfaces: Talk to the model via CLI or ChatGPT-style WebUI.
Report card: Generates a markdown summary of training metrics. How It Works
Architecture: Llama-like dense transformer with rotary embeddings and QK norm.
Optimizations: Uses Muon + AdamW, multi-query attention, logit softcap.
Tokenizer: Trains a Rust-based tokenizer from scratch.
Pretraining: Runs on FineWeb and evaluates with CORE metrics.
Midtraining: Uses SmolTalk conversations, MCQs, and tool-use data.
SFT: Fine-tunes on ARC, MMLU, GSM8K, and HumanEval.
RL: Optional reinforcement learning with GRPO on GSM8K math.
Inference: Supports KV cache, prefill/decode, and Python sandbox tool use. Performance
~$100 run (~4 hrs): Surpasses GPT-2 on CORE, simple Q&A/story tasks.
~$1,000 run (~41 hrs): Handles math/code problems and MCQs with coherence.
Depth-30 model (24 hrs): 40s on MMLU, 70s ARC-Easy, 20s GSM8K.
Coherence improves rapidly with scaling, approaching GPT-3-small behavior. How to Get Started
Clone: git clone https://github.com/karpathy/nanochat
Spin up a GPU box (e.g. 8×H100 for speedrun).
Run the training script (pretrain → midtrain → SFT → RL).
Launch CLI or WebUI to chat with your model. Access & Availability |
|
| Try now
|
|
|
|
Signals
| |
|
2,132 Likes | |
|
1,49 Likes | |
|
1,957 Likes | |
|
834 Likes | |
|
|
2,439 Likes | | |
|
|
|
|
Monitor Multi-Agent AI Workflows With Fiddler AI’s Observability Guide
| Fiddler AI’s free guide shows you how to monitor agent hierarchies, trace failures, and visualize metrics across complex systems.
Get step-by-step methods to improve visibility and control over AI agent workflows in production. |
Download the free guide ↗️ |
|
|
|
Reads
| |
|
2,432 Likes | OpenAI’s prompt pack shows how to apply ChatGPT to everyday workplace tasks. It includes role-agnostic examples for communication, meetings, decisions, and productivity, plus direct links to runnable prompts. |
|
|
1,153 Likes | Discover practical design patterns for building Deep Agents: orchestrators, sub-agents, and persistent state. These techniques let you run reliable, multi-step workflows far beyond the limits of shallow loops. |
|
|
| 1,966 Likes |
OpenAI presents a method to measure political bias across axes like escalation and coverage, showing GPT-5 reduces bias by ~30%. The evaluation helps you understand model robustness and objectivity |
| |
| |
|
|
Top Paper
| |
MIT paper shows LLMs can learn and fine-tune itself without human input |
9,285 Likes |
| MIT researchers present SEAL (Self-Adapting Language Models), a system that lets language models generate their own fine-tuning data and weight updates. This enables persistent adaptation without external supervision.
The model produces a “self-edit” in natural language that can restructure input, choose hyperparameters, or trigger data augmentation to guide its own learning.
Key takeaways from the research:
It applies the self-edits via SFT, making updates to its internal weights.
It rewrites new information and updates its own weights.
Uses a reinforcement loop to reward effective self-edits.
Achieves +40% factual recall improvement over baseline.
Outperforms GPT-4.1 using its own generated data.
Suffers from partial catastrophic forgetting during updates. This paper shows the first practical path toward self-improving LLMs that can continuously learn and adapt without human intervention. |
| Read now
|
|
|
|
At Alpha Signal, our mission is to build a sharp, engaged community focused on AI, machine learning, and cutting-edge language models, helping over 200,000 developers stay informed and ahead. We’re passionate about curating the best in AI, from top research and trending technical blogs to expert insights and tailored job opportunities. We keep you connected to the breakthroughs and discussions that matter, so you can stay in the loop without endless searching. We also work closely with partners who value the future of AI, including employers and advertisers who want to reach an audience as passionate about AI as we are.
Our partnerships are based on shared values of ethics, responsibility, and a commitment to building a better world through technology.Privacy is a priority at Alpha Signal. Our Privacy Policy clearly explains how we collect, store, and use your personal and non-personal information. By using our website, you accept these terms, which you can review on our website. This policy applies across all Alpha Signal pages, outlining your rights and how to contact us if you want to adjust the use of your information. We’re based in the United States. By using our site, you agree to be governed by U.S. laws. |
|
|
|
Looking to promote your company, product, service, or event to 250,000+ AI developers?
|
| Work with us
|
|
|
|
|