Skip to main content
Back

Chinese Academy of Sciences' CoG Beats Knowledge Graph AI by 10.5% Using Python

Code-on-Graph turns KG facts into Python class interfaces, letting LLMs reason with code instead of bloated prompts — beating prior SOTA by up to 10.5%

Paper, Full Access
  • Code-on-Graph (CoG) is a new framework that has LLMs write and execute Python code over knowledge graph facts, rather than injecting raw triples into prompts.
  • KG schemas are converted into Python class definitions; retrieved facts become object instances at runtime, keeping prompts lean and reasoning flexible.
  • CoG beats prior state-of-the-art on WebQSP, CWQ, and GrailQA by up to 10.5% — a large margin on benchmarks where gains are typically fractional.
  • The approach solves two core problems in LLM-KG integration: inflexibility of predefined operators and unscalability of dumping large fact sets into prompts.
  • Best suited for multi-hop, compositional questions over structured KGs; requires a well-defined schema and adds latency from iterative code execution.
  • Paper available on arXiv (2606.03705); no public code repository announced at time of publication.

Knowledge graphs (KGs) are one of the most reliable tools for keeping LLMs grounded in facts. They store structured, verifiable information as networks of entities and relationships, and plugging them into an LLM pipeline can dramatically reduce hallucinations and outdated answers. The problem is that most existing approaches for combining LLMs with KGs hit a ceiling fast , and a new paper from researchers at the Chinese Academy of Sciences proposes a fundamentally different way to cross that gap.

The framework is called Code-on-Graph (CoG), and its core idea is elegant: instead of dumping raw KG facts into a prompt, let the LLM write and execute Python code that operates on those facts as structured objects. The result is a system that is both more expressive and more scalable than anything that came before it.

The wall that existing frameworks keep hitting

The standard playbook for LLM-KG integration goes like this: retrieve relevant triples from the knowledge graph, serialize them into text, and stuff them into the model's prompt. This RAG-style injection approach has well-known failure modes , context window limits mean you can only fit so many triples, and there is no guarantee the model will actually use the facts you provide.

Beyond the context window problem, there is a deeper issue of expressiveness. While retrieval-augmented generation works reasonably well for text corpora, the information in graphs also lies in the complex interconnection between nodes , which poses a requirement for traversing and reasoning on graphs that simple retrieval cannot satisfy. Existing frameworks rely on a fixed set of predefined operators (think: get_neighbors, filter_by_type) to navigate the graph. These operators are rigid. If your question requires a combination of logic that the operator set was not designed for, the system breaks.

The CoG paper names these two failure modes explicitly: inflexibility (predefined operators lack the compositional expressiveness to handle complex KG questions) and unscalability (injecting large amounts of factual knowledge directly into prompts does not scale). Both are fundamental, not incidental.