True Knowledge Graph Capabilities

a27420e6cc562fe21ae1
Thu, 15 Aug 2024 18:43:07 +0000
Product Feedback

Ability to have a true Neo4j-like Knowledge graph capabilities within Foundry ecosystem.

Knowledge Graph elements being:

  Nodes (Entities):
        Represent real-world objects, concepts, or ideas
        Examples: people, places, organizations, events, products

  Edges (Relationships):
        Connect nodes to show how they are related
        Describe the nature of the connection between entities
        Examples: "works for," "located in," "is a type of"

  Properties (Attributes):
        Additional information associated with nodes or edges
        Provide more details about entities or relationships
        Examples: name, date of birth, job title, timestamp

Background for the request

Having experimented a lot with Traditional RAG vs Knowledge Graph RAG outside of Foundry, we are finding using Knowledge Graphs helpful in many scenarios that we want to use within Foundry. So we decided to have a go at what we could do within Foundry to emulate a Knowledge Graph using the Ontology, and Vertex.

The first challenge was how we could incorporate the main characteristics of a Knowledge Graph inside Foundry.
The approach we took was to create two Ontology Objects, Entity and Relationship.

Entity had the Node details and associated properties

Relationship has the Edge details and appropriate properties, with a link ‘from’ one entity, and a link ‘to’ another entity

We then used an LLM to parse multiple PDFs with the prompt defining the Entity values we want (ie Study, Therapeutic Area, Indication etc) and the relevant properties we want on it, and likewise for the Relationship.
From the parsed LLM response we went on to create records in the two Ontology objects, with appropriate linkage etc.
With embeddings etc as well we could do simple semantic searching and search arounds between the Node object (both the one as the ‘from’ and the one as the ‘to’ and the Relationship object that bridges them.
We went on to have a little Workshop app that allowed a side-by-side comparison of Traditional RAG and Knowledge Graph RAG results.

On from that, we wanted to use Vertex to represent it. Mixed results on this because it ultimately shows both the Entities and the Relationship objects as nodes. With styling we could set the Relationship node to be invisibly small and so only the Relationship Text remains, which made it look more ‘Knowledge Graph’ like, but it felt very crude and needed a lot of manual aligning to make it look good.

So, yes, adding Knowledge Graph capabilities, or moving towards that, would be extremely welcome.

8 posts - 4 participants