In this post we will explore the possibilities of combining interactions between Large Language Models(LLMs) and Knowledge Graphs(KGs). As a first scenario, one can harness the power of LLMs to generate KGs from unstructured text. And a second scenario could be utilizing a verified KG as inputs for performing prompts and queries against a LLM. Third scenario could be using LLM to talk to the KG. There could be more nuanced possibilities such as using KG information in pre-training, finetuning of LLMs etc. In this post we will only discuss the three simpler and straight scenarios.
Utilizing LLM to generate KGs from unstructured text
User can create a prompt giving ontology schema and unstructured text as input to LLM, and then ask the LLM to create a KG out of this data.
In this scenario, the LLM serves as an information extraction tool. It processes the unstructured text data, identifying entities, relationships, and relevant information based on the provided schema. The extracted entities and relationships are then organized into a structured KG using Resource Description Framework (RDF) triples or other graph representation formats. This approach has significant potential for tasks like data annotation, knowledge base construction, and summarization.
Augment LLM with external KG to give more precise answers.
One challenge with LLMs is their tendency to hallucinate. It can make up answers if the information it needs is not found in its training corpus. In this case we can provide additional domain documents as context, as we showed in a previous post. There is a cost implication to feeding too many external documents. One could do a vector similarity search of the document and only use those chunks that are relevant to the question at hand. However this approach can also be very texty.
Let’s say you need to answer questions based on information contained in multiple PDF documents.
Another way to solve this problem is to extract entities and relationships from this data and then use that to construct a Knowledge graph that unifies the information contained in all these multiple documents. This could lead to economical representation as triples of a RDF versus free flowing text with noisy words even in similar chunks.
Next, when user submits a query to the LLM then it can additionally use the KG as extra context. One way to do this could be search the top-k nodes of the KG that are similar to the user question and then feed this as additional context in prompt to LLM.
For example, in a medical question-answering scenario, the LLM may generate a response based on its training data, but by cross-referencing the information in the external medical KG, the response can be validated for accuracy and enriched with additional details.
Use LLM to query against external KGs
The other use case could be using LLM to create graph queries from user defined natural text and then run these queries agianst a KG like wikidata. This is similar to the approach we used in my PandasAI post where LLM converted user’s text into Python code in order to interact with Pandas dataframes. The equivalent thing here is to simply swap the Pandas Dataframe for a Knowledge Graph and LLM translating user query to cypher query or SPARQL versus Python code.
The LLM acts as a bridge between users and the KG, allowing them to interact with complex datasets without requiring deep knowledge of graph query languages.
For instance, a user could input a question like “What are the birthplaces of famous scientists?” The LLM can convert this question into a Cypher query to retrieve birthplace information from a KG like Wikidata and present the results in a user-friendly format.
By leveraging LLMs in this manner, querying KGs becomes more accessible to a wider audience, simplifying the process of extracting relevant information from structured data sources.
Overall, these scenarios demonstrate the potential of combining LLMs and KGs in various ways, enhancing information extraction, comprehension, and interaction with structured data.
Some challenges to consider while exploring these synergies:
Data Sparsity, and Quality:
As a KG is built from diverse sources, there could be incomplete or conflicting information in the KG that can pose challenges for both data preprocessing and model performance.
Scalability:
Both LLMs and KGs can be resource-intensive and computationally demanding. As the size and complexity of the KG and the LLM increase, scalability becomes a critical concern. Efficient data storage, retrieval, and processing mechanisms are necessary to handle large-scale KGs and ensure real-time interactions with LLMs
Interpretability and Trust:
The black-box nature of some LLMs raises concerns about their interpretability. Understanding how LLMs arrive at specific answers or decisions when interacting with KGs is crucial, especially in applications where trust and transparency are paramount, such as healthcare or legal systems. LLMs oftent show the chain of thought reasoning in the verbose trace, however even this could be a result of made up hallucination!
Data Privacy and Security:
KGs may contain sensitive information, and when combined with LLMs, there could be potential privacy and security risks. Safeguarding the data within the KG and ensuring that LLMs do not inadvertently leak sensitive information during interactions is a crucial challenge.
While there may be challenges to address, the benefits of this synergy are significant, offering promising solutions across diverse domains and applications.
Credits: My post has been heavily influenced by this excellent article: here.
Note: All views expressed in this article are the author’s own.