Knowledge Graphs / Preventing Hallucinations

[GiacomoPracucci]

First of all, thank you for your fantastic work.

I wanted to know if there is a possibility of integrating the use of Knowledge Graph to provide more context to the LLM.

I am experimenting using a schema with 4 tables consisting of many columns, and it happens for example that every now and then queries with nonexistent columns are generated (using Ollama with mistral or llama2), despite the fact that we have inserted DDLs of all 4 tables in the DB vector.

A knowledge graph where the nodes are schema, tables and columns I think could help the LLM a lot. For example, a chunk of distinct values could also be placed in the column properties. I imagine it could help the LLM a lot in complex schemas.

Is there in the roadmap or vannaAI integration of KG use? How difficult would it be to integrate them into the system today?

[zainhoda]

Yeah, something we're actively thinking about is having the package build up a more structured understanding of what tables, columns, relationships, and distinct values are in the database. So we're thinking along the same lines.

I think what should happen is that any time you call vn.train(...) it should append to that structured understanding in some way.
https://github.com/vanna-ai/vanna/blob/main/src/vanna/base/base.py#L1206

Then vn.generate_sql should retrieve from that structured understanding in some way to construct the prompt with more explicit instructions to use only the information provided:
https://github.com/vanna-ai/vanna/blob/main/src/vanna/base/base.py#L106-L117

I'm totally open to the specifics of the implementation, whether that's an actual knowledge graph as you suggest or a simpler method.

Please provide suggestions here!