Huawei and PSBC's YouZhi-LLM Doubles Bank AI Concurrency With 72% Less Memory
PSBC and Huawei convert existing GQA-based LLMs to MLA architecture, cutting KV cache by 72% and more than doubling concurrent users in production financial deployments
Takeaways
- What happened: PSBC and Huawei released YouZhi-LLM, a financial LLM that converts GQA attention to MLA to slash KV cache memory.
- Key innovation: A layer-adaptive FreqFold algorithm assigns different compression sizes per transformer layer, beating uniform baselines by up to 35% on perplexity.
- Concurrency gains: YouZhi-7B achieves 2.69x more simultaneous users and YouZhi-14B achieves 2.43x, with a 72% reduction in KV cache size.
- Accuracy also improves: YouZhi-7B scores 12.3% higher on financial benchmarks vs. its base model; YouZhi-14B gains 7.0%.
- Built on Ascend: The full pipeline runs on Huawei Ascend NPUs via vLLM-Ascend; inference gains are hardware-specific and not directly portable to NVIDIA GPUs.
- No public release: Model weights and training code are not publicly available; the paper builds on the open-source TransMLA framework.
Running an LLM at scale inside a bank is a different beast from running one in a research lab. When thousands of users are simultaneously querying a model for loan assessments, fraud detection, or customer service, the bottleneck is rarely compute -- it is memory. Specifically, the KV cache: the in-memory store of past context that every transformer layer needs to serve each active conversation. A joint team from the Postal Savings Bank of China (PSBC) and Huawei Technologies has now published YouZhi-LLM, a financial language model that attacks this problem head-on by surgically converting the attention mechanism of an existing pretrained model from GQA to MLA -- without retraining from scratch.
The memory wall blocking financial AI
To understand why this matters, you need to know the difference between two attention variants. GQA (Grouped Query Attention) is the dominant approach in most modern open-weight models like Qwen and LLaMA. It reduces KV-cache size by sharing key and value parameters across groups of query heads , which is a solid efficiency win but still stores full-dimensional key-value tensors per token. MLA (Multi-Head Latent Attention), pioneered by DeepSeek, takes a different approach. Instead of reducing KV-heads as in GQA, MLA jointly compresses the Key and Value into a latent vector -- a much smaller representation that gets cached instead. MLA requires only a small amount of KV cache, equal to GQA with only 2.25 groups, but can achieve stronger performance than MHA.
The catch? Despite the success of DeepSeek V2/V3/R1, most model providers have heavily invested in optimizing GQA-based models and therefore lack strong incentives to retrain MLA-based models from scratch. Retraining a 14B parameter model from zero is expensive and time-consuming -- not something a bank's AI team does lightly.