Skip to main content
Back

LMSYS Org's SGLang Pulls Nvidia, AMD, and Intel Into One Open-Source Roadmap

Nvidia, AMD, Intel, Google, Meta and four others are founding members of the SGLang Consortium. The group gives the industry's biggest rival chipmakers a seat at the table where an open-source inference engine plans its roadmap.

  • LMSYS launched the SGLang Consortium with Nvidia, AMD, Intel, Google, Meta and four other founding members.
  • Thinking Machines Lab, Cloudflare, Modal and RadixArk round out the nine founders and join twice-yearly roadmap meetings.
  • Maintainers keep control of technical decisions and code acceptance. No fees or budget have been disclosed.
  • SGLang runs on hundreds of thousands of GPUs and 332 organizations. Meta reported more than 2x GPU savings.
  • The summit also brought SGLang v1.0, Miles v0.2 and megakernels that doubled decode speed on 8x GB300.
  • RadixArk, backed by Nvidia and AMD, raised $100M at a $400M valuation and competes with vLLM.

Nvidia, AMD, and Intel compete on accelerator hardware, yet all three now help steer the same open-source inference engine. LMSYS Org announced the consortium during the opening keynote of the first SGLang Summit in San Francisco. The launch named nine members: NVIDIA, AMD, Intel, Google, Meta, Thinking Machines Lab, Cloudflare, Modal, and RadixArk. Thinking Machines Lab was included as a founding member of the consortium.

For developers who serve large language models, the consortium creates a formal channel for major hardware vendors and AI operators to influence SGLang’s priorities. Those choices affect model support, GPU utilization, distributed serving, and portability across accelerators. Maintainers continue to control technical decisions and code acceptance.

Prefix reuse became infrastructure

SGLang began as a research project focused on repeated work inside language-model requests. Prompts often share a prefix, such as a system message or few-shot examples. The model has already computed key-value cache entries for those tokens. SGLang’s RadixAttention organizes that cache in a radix tree, allowing later requests to reuse the matching prefix and reduce computation.

The project has since expanded into a full-stack engine for language, vision-language, and diffusion models. It schedules requests, manages accelerator memory, and runs the kernels that turn model weights into generated tokens. The Apache 2.0-licensed software belongs to the PyTorch ecosystem, and the consortium announcement reports deployments at Meta, Microsoft, Google, Amazon, Thinking Machines Lab, Cloudflare, Reflection, and Modal. The launch materials provide four indicators of scale:

  • Development: More than 2,000 contributors and 19,000 commits before the summit.
  • Adoption: 332 organizations listed as users, with no published counting methodology.
  • Efficiency: Meta reported “more than 2× GPU savings,” though the recap does not provide the workload or baseline.
  • Deployment: Hundreds of thousands of GPUs reportedly run SGLang across production environments.

A local deployment requires a supported accelerator environment, SGLang’s dependencies, and a model that fits available memory. The following example launches a Qwen server and sends a request through SGLang’s OpenAI-compatible API.

Start the local server
python -m sglang.launch_server \
  --model-path Qwen/Qwen2.5-7B-Instruct \
  --port 30000
Send a chat-completion request
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct",
    messages=[
        {
            "role": "user",
            "content": "Explain RadixAttention in one line.",
        }
    ],
)

print(response.choices[0].message.content)

Production deployments also need authentication, observability, version pinning, capacity limits, and an explicit tensor-parallel configuration for models spread across multiple accelerators.

Maintainers keep the merge button

Each member designates a technical representative and participates in two roadmap meetings per year. Working groups can continue between meetings, and the consortium plans to publish updates about direction and progress. Members may contribute engineering expertise, deployment feedback, testing capacity, or joint development. The initial agenda concentrates work in four areas:

  1. Support for new frontier models near their release dates.
  2. Reasoning, multimodal, and other compute-intensive workloads.
  3. Distributed inference across multiple accelerators and nodes.
  4. Hardware-specific optimization for each supported platform.
Mechanism Published terms
Representation One technical representative from each member.
Technical control SGLang maintainers retain responsibility for technical decisions and code acceptance.
Meeting cadence Two roadmap meetings each year, supported by working groups.
Community access The repository and contribution process remain open to outside participants.
Financial terms The announcement discloses no membership fees, funding commitments, or consortium budget.

Inference costs pull vendors together

Deloitte forecast that inference would consume roughly two-thirds of AI compute in 2026, up from about half in 2025. Serving efficiency therefore affects hardware purchases, data-center capacity, energy use, and cloud spending. Improvements in scheduling, caching, and kernels can support more requests on the same fleet.

RadixArk co-founder and CEO Ying Sheng, a co-author of the original SGLang paper, connected the work to data-center constraints: “With the challenges in building new data centers, software approaches to improve AI efficiency will be incredibly important.” Nvidia executive Ian Buck summarized the hardware vendor’s incentive: “Better software helps developers get more performance from NVIDIA compute.”

The summit supplied one example through SGLang’s “megakernels” for Meta’s Muse Spark Open model. A GPU kernel is a program dispatched to the accelerator. Combining a layer’s operations into one kernel reduces launch overhead and can overlap computation, memory access, and communication. On a single node containing eight GB300 accelerators, the team reported the following results:

Metric Previous path Megakernel path
Decoding throughput 517 tokens per second 1,109 tokens per second
Kernel launches per decoding step 1,450 97

The published recap omits batch size, sequence length, precision, and the full benchmark protocol, limiting direct comparisons with other models or deployments.

The summit also introduced SGLang v1.0, focused on clearer configuration errors, stronger existing features, and documented compatibility expectations. The project has not announced a release date.

Venture funding enters the roadmap

Several SGLang creators now run RadixArk, which participates in the consortium and sells managed infrastructure around the project. The company announced its seed round with $100 million in funding at a $400 million post-money valuation. Accel led the round, Spark Capital co-led it, and strategic participants included Nvidia’s NVentures and AMD.

The Apache 2.0 license preserves access to the source code and permits commercial use, modification, and forks. RadixArk plans to generate revenue through managed hosting, following the open-core infrastructure model used by companies such as Databricks and Elastic. Its founders’ roles in SGLang create commercial concentration around a community project, making transparent code review and roadmap publication especially relevant.

SGLang’s closest competitor, vLLM, emerged from the same broader UC Berkeley research community. A company commercializing vLLM has reportedly discussed raising up to $160 million at a $1 billion valuation. SGLang’s consortium adds a competitive asset through direct coordination with accelerator vendors, cloud platforms, and large operators.

Portability redistributes leverage

Five of the consortium’s nine listed participants design AI accelerators: Nvidia, AMD, Intel, Google, and Meta. Their involvement could change the engineering cost of deploying one serving stack across several hardware families.

Stakeholder Likely effect Remaining constraint
AMD, Intel, and Google First-class backends can reduce the work required to move deployments beyond Nvidia GPUs. Performance parity depends on kernels, compiler support, and continuous testing.
Nvidia Direct participation helps preserve strong performance on Nvidia hardware and provides visibility into portability work. Broader backend support may reduce CUDA-specific switching costs.
Operators A shared engine and API can simplify deployments spanning several accelerator types. Features and performance may still vary by backend.
vLLM The project faces a competitor with formal support from much of the accelerator industry. Developer adoption will still depend on reliability, performance, and model coverage.

Meta already reports running SGLang across Nvidia GPUs, AMD GPUs, Google TPUs, and its own MTIA accelerators. AMD executive Ramine Roane described the engine as software that “runs across hardware, including AMD Instinct GPUs.”

The portability effort also covers post-training through Miles, the team’s reinforcement-learning framework. Reinforcement-learning systems repeatedly generate model outputs called rollouts, which can consume much of the training budget. Miles supports Nvidia GPUs, AMD GPUs, and Google TPUs, with AWS Trainium support planned. Combined with SGLang, it could provide one open stack for rollout generation and production inference across several accelerator families.

Governance will be measured in commits

Vendor requests can pull maintainers toward competing kernels, model formats, and deployment priorities. The semiannual full-group cadence places much of that negotiation inside working groups. The announcement does not specify voting rules, conflict-resolution procedures, membership criteria, or the contents of public progress reports.

Commercial influence can also arrive through engineering capacity. Several project creators work at a venture-backed company whose products depend on SGLang’s adoption. Apache 2.0 permits forks, whose maintainers would inherit the expensive work of tracking new models, compilers, kernels, and accelerator generations.

Project credibility now depends on published evidence that the consortium improves compatibility without weakening community governance. Useful signals include:

  • Public roadmap updates with named owners and delivery status.
  • A release date and compatibility policy for SGLang v1.0.
  • Continuous-integration coverage across Nvidia, AMD, Intel, Google, Meta, and AWS hardware.
  • Reproducible benchmarks with model, batch, precision, sequence, and hardware settings.
  • Clear rules for adding members, resolving disputes, and handling maintainer conflicts of interest.

For multi-vendor deployments, direct participation from chipmakers improves the prospects for timely backend support. Actual portability will depend on feature parity, test coverage, and sustained optimization across each platform. Session recordings and talks are available on the SGLang Summit page.