Knowledge Graph Embeddings: When Graphs and Vectors Merge for Supercharged RAG

TL;DR

Classic vector RAG excels at simple questions but fails on multi-hop questions that require connecting multiple pieces of information ("Who is the CEO of the company that acquired the startup founded by Elon Musk's brother?"). Knowledge Graph Embeddings combine the power of knowledge graphs (explicit relations) with vector embeddings (semantic similarity). Result: +35% accuracy on multi-hop questions and near-perfect entity disambiguation.

Why vectors alone are not enough

The limitations of vector RAG

The problem illustrated

Question: "Which products are manufactured in the same country
            as the headquarters of our component X supplier?"

Vector RAG:
  → Searches "component X supplier country products"
  → Finds documents about component X, others about products
  → Does NOT connect the entities
  → Incorrect answer or hallucination

RAG + Knowledge Graph:
  → component_X --supplied_by--> Company_Y
  → Company_Y --headquarters--> Germany
  → Germany --manufactures--> [Product_A, Product_B, Product_C]
  → Precise answer with full traceability

Understanding Knowledge Graph Embeddings

What is a Knowledge Graph?

A knowledge graph stores information as triples (subject, predicate, object):

(Ailog, is_a, RAG_Platform)
(Ailog, based_in, Paris)
(Ailog, founded_in, 2024)
(Ailog, supports, Shopify)
(Shopify, is_a, Ecommerce_CMS)
(Paris, is_in, France)
(France, member_of, European_Union)

Why embeddings for graphs?

Graphs store explicit relations but don't handle semantic similarity. Knowledge Graph Embeddings project entities and relations into a vector space to:

1. Predict missing relations (link prediction)
2. Find similar entities (entity similarity)
3. Combine semantic search and graph traversal

KG Embedding models

How TransE works (the most intuitive)

Idea: if (Paris, capital_of, France),
then: embedding(Paris) + embedding(capital_of) ≈ embedding(France)

           capital_of
Paris ─────────────────→ France
  h    +       r        ≈    t

Training:
- Minimize ||h + r - t|| for true triples
- Maximize ||h + r - t'|| for false triples (negatives)

DEVELOPERpython
import torch
import torch.nn as nn

class TransE(nn.Module):
    def __init__(self, num_entities, num_relations, dim=128):
        super().__init__()
        self.entity_embeddings = nn.Embedding(num_entities, dim)
        self.relation_embeddings = nn.Embedding(num_relations, dim)

        nn.init.xavier_uniform_(self.entity_embeddings.weight)
        nn.init.xavier_uniform_(self.relation_embeddings.weight)

    def forward(self, heads, relations, tails):
        h = self.entity_embeddings(heads)
        r = self.relation_embeddings(relations)
        t = self.entity_embeddings(tails)

        # Score = -||h + r - t||
        score = -torch.norm(h + r - t, p=2, dim=-1)
        return score

    def predict_tail(self, head, relation, top_k=10):
        """Predict the most likely entities."""
        h = self.entity_embeddings(head)
        r = self.relation_embeddings(relation)

        all_entities = self.entity_embeddings.weight
        scores = -torch.norm(
            h + r - all_entities, p=2, dim=-1
        )
        return torch.topk(scores, top_k)

Hybrid architecture: Graph + Vectors

Recommended design pattern

┌──────────────────────────────────────────────────────┐
│                      USER QUERY                        │
│  "Which products to recommend to customers who buy     │
│   items similar to product X?"                         │
└───────────────────────┬──────────────────────────────┘
                        │
              ┌─────────┴─────────┐
              │   QUERY ANALYSIS   │
              │  (Intent + Entities)│
              └────┬──────────┬───┘
                   │          │
        ┌──────────┘          └──────────┐
        ▼                                ▼
┌───────────────┐              ┌───────────────┐
│ GRAPH SEARCH  │              │ VECTOR SEARCH │
│               │              │               │
│ Neo4j Cypher  │              │ Qdrant/Pinecone│
│ - Traversal   │              │ - Similarity  │
│ - Relations   │              │ - Semantic    │
│ - Multi-hop   │              │ - Fuzzy match │
└───────┬───────┘              └───────┬───────┘
        │                              │
        └──────────┬───────────────────┘
                   │
            ┌──────┴──────┐
            │   FUSION    │
            │  (Reranking │
            │  + Scoring) │
            └──────┬──────┘
                   │
            ┌──────┴──────┐
            │ GENERATION  │
            │    (LLM)    │
            └─────────────┘

Implementation with Neo4j + LangChain

DEVELOPERpython
from langchain_community.graphs import Neo4jGraph
from langchain.chains import GraphCypherQAChain
from langchain_openai import ChatOpenAI
from langchain_community.vectorstores import Neo4jVector

# Connect to Neo4j graph
graph = Neo4jGraph(
    url="bolt://localhost:7687",
    username="neo4j",
    password="your-password"
)

# Create graph schema
graph.query("""
CREATE CONSTRAINT IF NOT EXISTS
FOR (p:Product) REQUIRE p.id IS UNIQUE
""")

# Populate the graph with relations
graph.query("""
MERGE (p:Product {id: 'prod_001', name: 'Widget Pro'})
MERGE (c:Category {name: 'Electronics'})
MERGE (s:Supplier {name: 'TechCorp', country: 'Germany'})
MERGE (p)-[:BELONGS_TO]->(c)
MERGE (p)-[:SUPPLIED_BY]->(s)
MERGE (s)-[:LOCATED_IN]->(:Country {name: 'Germany'})
""")

# QA chain with automatic Cypher generation
llm = ChatOpenAI(model="gpt-4o", temperature=0)

cypher_chain = GraphCypherQAChain.from_llm(
    llm=llm,
    graph=graph,
    verbose=True,
    validate_cypher=True,
    top_k=10,
)

# Multi-hop query
result = cypher_chain.invoke({
    "query": "Which products are supplied by companies "
             "located in Germany?"
})
# → Auto-generated Cypher:
# MATCH (p:Product)-[:SUPPLIED_BY]->(s:Supplier)
#       -[:LOCATED_IN]->(c:Country {name: 'Germany'})
# RETURN p.name, s.name

print(result["result"])

Hybrid Graph + Vector search with Neo4j

DEVELOPERpython
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")

# Create vector index in Neo4j
vector_store = Neo4jVector.from_existing_graph(
    embedding=embeddings,
    url="bolt://localhost:7687",
    username="neo4j",
    password="your-password",
    node_label="Product",
    text_node_properties=["name", "description"],
    embedding_node_property="embedding",
)

def hybrid_graph_vector_search(query: str, top_k: int = 5):
    """Combine vector search and graph traversal."""

    # 1. Vector search: find close entities
    vector_results = vector_store.similarity_search_with_score(
        query, k=top_k
    )

    # 2. Enrich with graph: relations and context
    enriched_results = []
    for doc, score in vector_results:
        node_id = doc.metadata.get("id")

        # Get node neighborhood
        neighbors = graph.query(f"""
            MATCH (n {{id: '{node_id}'}})-[r]-(m)
            RETURN type(r) as relation,
                   labels(m)[0] as type,
                   m.name as name
            LIMIT 20
        """)

        enriched_results.append({
            "entity": doc.page_content,
            "score": score,
            "relations": neighbors,
            "context": format_graph_context(neighbors)
        })

    return enriched_results

def format_graph_context(neighbors: list) -> str:
    """Format graph context for LLM."""
    context_parts = []
    for n in neighbors:
        context_parts.append(
            f"- {n['relation']} -> {n['type']}: {n['name']}"
        )
    return "\n".join(context_parts)

Building the Knowledge Graph

Automatic entity and relation extraction

DEVELOPERpython
import anthropic

client = anthropic.Anthropic()

def extract_knowledge_graph(text: str) -> dict:
    """Extract entities and relations from text."""
    response = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=2000,
        messages=[{
            "role": "user",
            "content": f"""Analyze the following text and extract:
1. Entities (people, organizations, products, locations, concepts)
2. Relations between entities

Return a JSON with:
{{
  "entities": [
    {{"id": "e1", "name": "...", "type": "Organization|Person|Product|Location|Concept"}}
  ],
  "relations": [
    {{"source": "e1", "target": "e2", "type": "...", "properties": {{}}}}
  ]
}}

Text: {text}"""
        }]
    )

    return json.loads(response.content[0].text)

def build_graph_from_documents(documents: list):
    """Build graph from a list of documents."""
    all_entities = {}
    all_relations = []

    for doc in documents:
        kg = extract_knowledge_graph(doc["text"])

        # Deduplicate entities
        for entity in kg["entities"]:
            key = f"{entity['type']}:{entity['name'].lower()}"
            if key not in all_entities:
                all_entities[key] = entity
            else:
                all_entities[key]["properties"] = {
                    **all_entities[key].get("properties", {}),
                    **entity.get("properties", {})
                }

        all_relations.extend(kg["relations"])

    insert_into_neo4j(
        list(all_entities.values()),
        all_relations
    )

Benchmarks: KG-only vs Vector-only vs Hybrid

Test on HotpotQA (multi-hop questions)

Test on enterprise questions (internal corpus)

When to use each approach

Advanced use cases

Entity disambiguation

DEVELOPERpython
def disambiguate_entity(mention: str, context: str) -> dict:
    """Disambiguate an entity mention via the graph."""
    candidates = graph.query("""
        MATCH (n)
        WHERE n.name CONTAINS $mention
           OR n.aliases CONTAINS $mention
        RETURN n, labels(n) as types,
               [(n)-[r]-(m) | {rel: type(r), node: m.name}]
               as relations
        LIMIT 10
    """, {"mention": mention})

    if len(candidates) <= 1:
        return candidates[0] if candidates else None

    scores = []
    for candidate in candidates:
        context_embedding = embed(context)
        candidate_context = " ".join(
            [f"{r['rel']} {r['node']}" for r in candidate["relations"]]
        )
        candidate_embedding = embed(candidate_context)
        score = cosine_similarity(context_embedding, candidate_embedding)
        scores.append((candidate, score))

    return max(scores, key=lambda x: x[1])[0]

Multi-hop reasoning with chain of thought

DEVELOPERpython
def multi_hop_reasoning(question: str, max_hops: int = 3):
    """Multi-hop reasoning on the knowledge graph."""
    # Step 1: Identify starting entities
    entities = extract_entities_from_question(question)

    # Step 2: Explore graph step by step
    reasoning_chain = []
    current_entities = entities

    for hop in range(max_hops):
        neighborhood = graph.query("""
            MATCH (n)-[r]-(m)
            WHERE n.name IN $entities
            RETURN n.name as source, type(r) as relation,
                   m.name as target, labels(m)[0] as type
        """, {"entities": current_entities})

        if not neighborhood:
            break

        next_step = select_relevant_relations(
            question, neighborhood, reasoning_chain
        )

        reasoning_chain.append(next_step)
        current_entities = [step["target"] for step in next_step]

    # Step 3: Generate answer with reasoning chain
    answer = generate_answer_with_chain(question, reasoning_chain)
    return answer

Tools and frameworks

FAQ

Knowledge Graph or Microsoft's GraphRAG, what's the difference?

A Knowledge Graph is a database of structured facts (entities + relations). Microsoft's GraphRAG is a specific approach that automatically builds a graph from documents, creates hierarchical summaries by community, and uses them to answer questions. GraphRAG is better suited for global questions ("corpus summary"), while classic KG excels for precise multi-hop questions. See our guide on GraphRAG.

What does building a Knowledge Graph cost?

Automatic construction with LLM costs about $0.50-2 per document (entity/relation extraction). For 10,000 documents, expect $5,000-20,000 for initial construction. Neo4j production costs are ~$65/month for a dedicated instance. The open-source alternative is Neo4j Community Edition (free) or Amazon Neptune serverless.

Are KG Embeddings necessary for hybrid RAG?

No, for a simple graph+vector hybrid RAG, Cypher queries suffice. KG Embeddings become useful when: 1) you have millions of triples and traversal is slow, 2) you want to predict missing relations (link prediction), 3) you combine entity similarity with semantic search. For most use cases, start without KG Embeddings and add them if needed.

How to keep a Knowledge Graph up to date?

Three approaches: 1) Incremental extraction when new documents are added, 2) Periodic validation of existing relations, 3) Versioning the graph to track changes. Incremental extraction is the most common: each new document goes through the extraction pipeline and new entities/relations are added to the existing graph.

Is a Knowledge Graph necessary for my use case?

A KG is recommended if: you have frequent multi-hop questions, your domain has complex relationships between entities (org charts, supply chains, regulations), or you need explainability (reasoning traceability). If your questions are mainly simple factual ones, classic vector RAG is sufficient. See our guide on retrieval strategies to choose the right approach.

---

Knowledge Graph Embeddings represent the next frontier of RAG: beyond simple semantic similarity, toward true reasoning about relationships between entities. Try Ailog to see how our hybrid graph+vector approach answers your most complex questions.