When knowledge graphs know too little

Biomedical knowledge graphs are designed to organize complex scientific knowledge into interconnected networks of genes, diseases, drugs, proteins, and pathways. In theory, they should help researchers navigate vast amounts of information quickly. In practice, many graphs remain shallow.

They often contain only high-level relationships, missing the intermediate biological steps that explain how entities are truly connected. A drug may be linked to a disease, or a protein to a pathway, but the mechanistic context is absent. Without this depth, graphs become useful for lookup but limited for discovery.

At the same time, manually enriching these graphs with deeper knowledge is slow and difficult to scale, while fully automated systems risk introducing unsupported or inaccurate claims.

This raised the central question of my thesis: how can shallow biomedical knowledge graphs be systematically expanded into dense, high-resolution networks while preserving scientific reliability?

From sparse nodes to connected pathways

To address this, I developed KG-Orchestra, an open-source multi-agent framework that starts with a curated seed graph and progressively enriches it using biomedical literature. Instead of relying on a single AI model, the framework uses specialized agents that collaborate across different tasks: retrieving relevant evidence, identifying missing pathways, aligning extracted knowledge to graph schemas, resolving entities, and validating each new relationship before insertion.

The purpose is not to generate random new edges. It is to uncover the missing steps between existing nodes. A simple connection between two entities can therefore become a multi-hop biological pathway containing receptors, signaling molecules, cellular processes, and disease mechanisms.

Validation before integration

A key part of the framework is that newly extracted information is not accepted automatically.

Each proposed relationship passes through validation steps that assess whether it is biologically plausible, semantically coherent, directionally correct, and supported by the retrieved evidence. If a relationship appears weak or inconsistent, the system attempts to repair it or flags it for human review.

This creates a quality-control layer that helps reduce noisy or misleading graph expansion.

Every new relation carries provenance

Depth and validation are only useful if researchers can inspect the source of information.

For this reason, KG-Orchestra attaches provenance to newly added knowledge. Each relationship is linked to supporting text passages and source identifiers such as PubMed or DOI records.

This allows researchers to trace the origin of a claim, review the underlying evidence, and audit how a pathway was constructed. The result is not just a larger graph, but a graph whose growth remains transparent and accountable.

What the evaluation showed

Across multiple biomedical case studies, KG-Orchestra demonstrated that multi-agent enrichment can substantially increase graph coverage while maintaining high biological validity (+90%, depending on the LLM used in the workflow) and strong semantic consistency.

The findings also showed that combining hybrid retrieval with staged validation improves evidence quality, while schema alignment helps prevent uncontrolled ontology growth.

Why dense validated graphs matter

When shallow graphs become dense, validated, and evidence-backed, they become far more useful for science.

Researchers can trace mechanistic disease pathways, discover indirect drug effects, identify overlooked biological targets, and connect findings that were previously scattered across separate publications.

Instead of seeing isolated facts, they can follow causal chains with confidence.

Toward living scientific knowledge systems

KG-Orchestra demonstrates a shift in how biomedical knowledge graphs can be used. Rather than static resources with limited depth, they can become living systems that continuously grow richer as new evidence emerges.

By expanding shallow knowledge into dense networks with validation and full provenance, such frameworks can help researchers move from fragmented information toward explainable and evidence-driven discovery.

References

[1] Lu, Y., Goi, S. Y., Zhao, X., & Wang, J. (2025). Biomedical knowledge graph: A survey of domains, tasks, and real-world applications. arXiv preprint arXiv:2501.11632. https://doi.org/10.48550/arXiv.2501.11632

[2] Pan, S., Luo, L., Wang, Y., Chen, C., Wang, J., & Wu, X. (2024). Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering, 36(7), 3580-3599. https://doi.org/10.1109/TKDE.2024.3352100

[3] Peng, C., Xia, F., Naseriparsa, M., & Osborne, F. (2023). Knowledge graphs: Opportunities and challenges. Artificial intelligence review, 56(11), 13071-13102. https://doi.org/10.1007/s10462-023-10465-9

[4] Zhong, L., Wu, J., Li, Q., Peng, H., & Wu, X. (2023). A comprehensive survey on automatic knowledge graph construction. ACM Computing Surveys, 56(4), 1-62. https://doi.org/10.1145/3618295

[5] Luan, Y., He, L., Ostendorf, M., & Hajishirzi, H. (2018). Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In Proceedings of the 2018 conference on empirical methods in natural language processing (pp. 3219-3232). https://doi.org/10.18653/v1/D18-1360

[6] Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., ... & Gui, T. (2025). The rise and potential of large language model based agents: A survey. Science China Information Sciences, 68(2), 121101. https://doi.org/10.1007/s11432-024-4222-0

[7] Lu, Y., & Wang, J. Karma: Leveraging multi-agent llms for automated knowledge graph enrichment. arXiv 2025. arXiv preprint arXiv:2502.06472. https://doi.org/10.48550/arXiv.2502.06472