After seeing yet another Graph RAG demo using Neo4j with no ontology, I decided to show what a real semantic Graph RAG looks like.

How RDF, SPARQL, and OWL Transform Graph RAG from Pattern Matching into Intelligent Knowledge Retrieval (github repo included at the end)

14 min readNov 25, 2025

The Viral Reality Check

My recent LinkedIn post on this got 65,000+ views, 1,500 likes, 1000+ saves and 1,000 clicks to the GitHub repository. The reaction told me something important: the AI community knows something is missing from current Graph RAG implementations.

Everyone’s building Graph RAG with Neo4j, calling it “knowledge graphs,” and claiming semantic understanding. But here’s the uncomfortable truth: without formal ontologies, you don’t have a knowledge graph — you just have a graph database with fancy labels.

This article is the deep dive behind that viral post. I’ll show you exactly why formal semantics matter, walk through a complete working implementation, and demonstrate with a concrete example why Labeled Property Graphs (LPG) fundamentally cannot deliver what RDF-based knowledge graphs can.

Part I: The Problem Nobody Talks About

The Graph RAG Explosion

Graph RAG is everywhere in 2025. The pattern is elegant: combine the reasoning power of Large Language Models with the structured knowledge in graph databases. Instead of pure vector search over unstructured text, you get:

Precise retrieval of structured facts
Relationship traversal that matches queries
Reduced hallucination through grounded data
Explainable results with clear provenance

But there’s a critical flaw in most implementations.

The “Jaguar Problem”

Consider this sentence:

“Yesterday I was hit by a Jaguar.”

What does this mean? Without concept awareness, it’s impossible to know:

🐆 Were you attacked by a big cat (Panthera onca)?
🚗 Were you struck by a luxury automobile?
🎸 Did someone throw a Fender Jaguar guitar at you?
🏈 Were you hit by a Jacksonville Jaguar on a football field?

The word “Jaguar” is identical in all four cases. In a typical LPG database like Neo4j, you’d represent all four as nodes with the same label. They’re just strings with different properties attached.

All three nodes have the same label: Jaguar.

When you feed this to an LLM for extraction or querying, how does it know which concept you mean? It doesn’t — unless you add extensive application-level logic, manual disambiguation, and post-processing cleanup.

What’s Missing: Formal Semantics

The fundamental issue is that LPG databases lack formal semantic definitions:

❌ No machine-readable ontologies
❌ No class hierarchies (RDFS/OWL)
❌ No property domain/range constraints
❌ No reasoning or inference capabilities
❌ No standardized vocabulary (W3C standards)

In an LPG, node labels and relationship types are application-specific strings. They have no inherent meaning outside your specific implementation. There’s no way for an external system — or an LLM — to understand what these concepts actually represent.

Part II: The Solution — Semantic Knowledge Graphs

What Are Formal Ontologies?

Formal ontologies (RDFS/OWL) are machine-readable specifications of domain knowledge. They define:

Classes — Concepts and their hierarchies
Properties — Relationships and their constraints
Axioms — Rules and logical constraints

In the jaguar conservation ontology used in this project, we define a clear hierarchy: Animals are the top-level concept, which contains Mammals, which contains Big Cats, which finally contains Jaguars specifically referring to Panthera onca. Each level adds specificity and context.

Properties are equally explicit. When we define “hasGender,” we specify that it only applies to Jaguars (the domain) and that its value must be text (the range). When we define “occursIn,” we’re stating that Animals can occur in Locations, and the system understands the semantic meaning of this relationship.

Why This Changes Everything

With formal ontologies, an LLM now has semantic context:

✅ Domain understanding — A Jaguar is explicitly defined as Panthera onca, not a car or guitar

✅ Hierarchical knowledge — Jaguars are BigCats, which are Mammals, which are Animals

✅ Property constraints — Gender properties only apply to Jaguars and must be text values

✅ Relationship semantics — Location relationships connect Animals to Places with specific meaning

✅ Reasoning capabilities — If X is a Jaguar, the system can infer X is also a Mammal and an Animal

This isn’t just metadata — it’s machine-understandable domain knowledge that LLMs can reason over.

Part III: The Demonstration — Ontology-Aware Extraction

The Experiment Setup

To prove that ontologies enable concept-aware extraction, I created a deliberately challenging corpus containing mixed content:

The corpus includes:

🐆 Wildlife conservation stories (El Jefe, Macho B, jaguar populations)
🚗 Jaguar automobile history (E-Type specifications, production details)
🎸 Fender Jaguar guitar descriptions (model details, surf rock history)

All three use the word “Jaguar” hundreds of times. A naive extraction system would pull everything indiscriminately.

The Ontology-Driven Process

The notebook demonstrates feeding GPT-5 both the formal ontology defining the jaguar conservation domain and the mixed corpus containing all three types of “jaguars.” The ontology acts as a semantic filter — a formal specification of what concepts are relevant to this domain.

The LLM isn’t given any additional instructions about filtering out cars or guitars. The ontology itself provides all the semantic guidance needed.

The Results: Pure Semantic Filtering

GPT-5 automatically extracted ONLY wildlife-related entities:

The extraction identified individual jaguars like El Jefe and Macho B, correctly associating them with:

Gender information (Male)
Geographic locations (Arizona)
Monitoring organizations (Arizona Game and Fish Department)
Timeline data (first sighting dates, last seen dates)
Conservation status (killed vs. alive)
Cause of death where applicable

Zero mentions of:

Jaguar E-Type automobiles
Fender Jaguar guitars
Any non-wildlife content

No Post-Processing Required

This isn’t cherry-picked data or manual cleanup. The LLM:

Understood the domain from the formal ontology structure
Disambiguated entities based on semantic class definitions
Extracted only relevant information aligned with the ontology
Generated valid RDF with proper structure and relationships
Inferred implicit relationships from contextual understanding

This is impossible with LPG databases because they lack the formal semantic structure that guides the LLM’s understanding.

Part IV: The Architecture — Building Semantic Graph RAG

Technology Stack

The complete implementation demonstrates a full semantic stack:

Knowledge Layer:

Ontotext GraphDB serves as the RDF triple store, providing a SPARQL query engine that understands semantic queries
RDFS/OWL Ontologies provide formal domain semantics that machines can interpret
RDF Turtle format represents knowledge in a standardized, interoperable way

AI Layer:

Microsoft Agent Framework orchestrates the AI conversation and manages state
OpenAI GPT-4/5 handles natural language understanding and can generate semantic queries
Function Calling integrates the LLM with knowledge graph tools seamlessly

Application Layer:

DevUI provides a built-in development interface for rapid prototyping without building custom UIs
Python 3.10+ serves as the core application runtime
HTTP communication connects all components

System Architecture

The architecture demonstrates three key layers working together:

User Interface Layer:

The DevUI provides an interactive chat interface that automatically opens in a browser, includes built-in debugging tools, and handles all UI rendering without requiring custom frontend code.

Agent Layer:

The Jaguar Query Agent sits at the heart of the system. It contains a carefully crafted system prompt that provides Graph RAG instructions, SPARQL generation guidelines, and ontology awareness. The agent uses OpenAI’s GPT models to transform natural language into semantic queries, leverages function calling to execute those queries, and maintains conversation context across multiple interactions.

Data Layer:

GraphDB stores the RDF knowledge graph with full ontology support. It executes SPARQL queries with semantic understanding, validates queries against the ontology, and returns structured results that preserve all relationship information.

The Graph RAG Flow

The system demonstrates a complete semantic query cycle:

Step 1: Natural Language Input

Users ask questions in plain English, like “How many jaguars were killed in Arizona?” The system doesn’t require users to know anything about the underlying data structure or query language.

Step 2: Semantic Query Generation

The agent analyzes the user’s intent and consults the ontology to understand what concepts are involved. It generates a SPARQL query that explicitly requests Jaguar entities (as defined in the ontology), filters for those marked as killed, restricts to Arizona locations, and retrieves relevant labels and cause of death information.

Step 3: Knowledge Graph Execution

GraphDB executes the semantic query against the RDF triple store, using the ontology to ensure proper interpretation. It returns structured results with full URI references and typed data values.

Step 4: Natural Language Response

The agent interprets the structured results, generates a human-readable explanation, includes relevant context and details, shows the SPARQL query used for transparency, and formats everything with clear markdown presentation.

For the Arizona question, the response identifies Macho B as the single jaguar killed in Arizona, provides his complete history from first sighting in 1996 to his tragic death in 2009, explains the circumstances around his capture and euthanasia, and maintains full attribution to the database.

Key Design Principles

Agent Configuration:

The agent is designed with a clear system prompt that emphasizes using the GraphDB tool for all jaguar-related queries. It instructs the model to base all queries on the provided ontology, ensures SPARQL queries are shown for transparency, and enforces markdown formatting for readable responses.

The agent settings specify GPT-4 or GPT-5 for advanced reasoning, configure authentication with environment variables, and register the knowledge graph query tool for function calling.

Knowledge Graph Tool:

The GraphDB query tool demonstrates how to embed ontological knowledge. Its description includes the complete formal ontology specification, showing all classes, properties, and their relationships. This enables the LLM to generate ontology-aware queries.

The tool handles query execution against the configured GraphDB endpoint, manages authentication and headers for SPARQL requests, validates and formats results as JSON, and provides detailed error messages when queries fail.

Main Application:

The entry point demonstrates the simplicity enabled by Microsoft Agent Framework. It creates the configured agent, starts the DevUI server with auto-opening browser functionality, and lets the framework handle all complexity of conversation management and state persistence.

This minimal setup contrasts sharply with traditional web applications, requiring no custom UI code, no manual session management, and no complex routing or state handling.

Part V: RDF vs LPG — The Technical Reality

The Fundamental Differences

The project highlights critical architectural differences:

Schema Definition:

RDF uses formal ontologies (RDFS/OWL) that are machine-readable specifications. LPG uses informal application schemas that exist only in code or documentation.

Semantic Meaning:

RDF semantics are explicit and standardized — any system can interpret them. LPG semantics are implicit and application-level — only your code knows what labels mean.

Reasoning Capabilities:

RDF provides built-in reasoning through RDFS/OWL inference engines. LPG requires application-specific logic for any reasoning.

Standards:

RDF is based on W3C standards (RDF, SPARQL, OWL) that work across all implementations. LPG uses vendor-specific languages (Cypher for Neo4j, GSQL for TigerGraph) that don’t interoperate.

Interoperability:

RDF uses universal identifiers (URIs) and shared namespaces for seamless integration. LPG uses proprietary formats that require custom integration work.

World Assumption:

RDF uses Open World Assumption where absence of information means “unknown.” LPG uses Closed World Assumption where absence means “false.”

LLM Guidance:

RDF provides rich semantic structure that LLMs can reason over. LPG provides only labels and properties with no inherent meaning.

Why LLMs Need Ontologies

The extraction notebook demonstrates what LLMs can do with proper semantic guidance:

With RDF Ontologies, LLMs understand:

Formal class definitions showing that Jaguar is a subclass of BigCat
Property constraints specifying that gender applies to Jaguars and must be text
Relationship semantics defining that “occursIn” connects Animals to Locations
Hierarchical understanding through taxonomic structures
Validation rules for data types, cardinality, and domains

With LPG, LLMs only see:

String labels like “Jaguar,” “OCCURS_IN,” “Location”
No formal meaning or semantic context
No hierarchical structure to reason over
No validation constraints to guide extraction
No machine-readable semantics whatsoever

The Open World vs Closed World Problem

The project demonstrates why Open World Assumption matters for knowledge:

RDF’s Open World Approach:

If we don’t know a jaguar’s gender, that doesn’t mean the jaguar has no gender — it means we don’t have that information yet. This enables gradual knowledge accumulation as new data arrives. It supports inference from partial information, naturally representing real-world knowledge that’s always incomplete.

LPG’s Closed World Approach:

If a jaguar node lacks a gender property, queries interpret this as “has no gender” rather than “gender unknown.” This assumes complete knowledge and requires all facts to be explicitly stated. While suitable for operational data, it doesn’t work well for evolving knowledge bases.

For Graph RAG over continuously evolving knowledge, the Open World Assumption is essential.

The Standardization Advantage

The project leverages W3C standards throughout:

RDF Benefits:

Universal identifiers (URIs) enable global interoperability. The standard query language (SPARQL) works across all RDF stores. Formal ontology languages (RDFS, OWL) enable reasoning. Shared vocabularies allow cross-organizational knowledge sharing. Any system can interpret the semantics without custom code.

LPG Limitations:

Neo4j uses Cypher, TigerGraph uses GSQL, Amazon Neptune supports both but inconsistently. There’s no standardized way to share schemas or queries across systems. Each implementation requires custom integration work. Knowledge can’t easily be shared between organizations using different graph databases.

This matters when building systems that need to integrate knowledge from multiple sources, share ontologies across organizations, build reusable semantic tools, or enable LLMs to work with any knowledge graph.

Part VI: Why This Matters for AI

Knowledge Graphs as “Data for AI”

LLMs are trained on massive text corpora, but they struggle with:

❌ Factual accuracy — Hallucinations and outdated information
❌ Structured reasoning — Complex multi-hop inferences
❌Domain expertise — Specialized knowledge not in training data
❌Explainability — Understanding why an answer is correct

Semantic knowledge graphs solve these problems:

✅ Grounded facts — Every statement is traceable to its source

✅ Explicit relationships — Complex reasoning through graph traversal

✅ Domain ontologies — Formal definition of specialized concepts

✅ Provenance — Clear lineage of where information comes from

The Graph RAG Revolution

Traditional RAG follows a simple pattern: User queries trigger vector searches that retrieve text chunks, which are fed to an LLM for response generation.

Problems with this approach include vector similarity not capturing semantic relationships, text chunks losing their original context, lack of structured reasoning capabilities, and difficulty explaining why specific results were retrieved.

Semantic Graph RAG transforms this: User queries trigger semantic query generation that retrieves structured results from knowledge graphs, which are then interpreted by LLMs for natural language responses.

The benefits are substantial: Semantic query matching through ontologies ensures conceptual understanding. Structured data preserves all relationships and context. Graph traversal enables sophisticated multi-hop reasoning. Both queries and results are fully explainable with clear provenance.

Real-World Applications

Healthcare:

Medical ontologies like SNOMED CT, ICD-11, and Gene Ontology enable diagnosis systems with complete evidence trails. The system can traverse from symptoms to diseases to treatments while maintaining semantic relationships.

Legal:

Legal domain ontologies and case law hierarchies support contract analysis systems that understand regulatory compliance requirements through semantic reasoning.

Enterprise:

Industry-specific ontologies for finance, manufacturing, and logistics enable supply chain optimization with causal reasoning about dependencies and impacts.

Research:

Domain-specific scientific ontologies facilitate literature review systems and hypothesis generation tools that understand conceptual relationships across papers.

In every case, formal ontologies transform raw data into machine-understandable knowledge that LLMs can reason over effectively.

Part VII: The Project Structure

The repository demonstrates a complete semantic Graph RAG implementation with four key components:

1. The Ontology (jaguar_ontology.ttl)

This file defines the formal semantic structure for jaguar conservation. It establishes a class hierarchy from Animal down to specific Jaguar species, defines properties with explicit domains and ranges, specifies relationships between entities, and includes comments explaining semantic intent.

The ontology serves as the “schema” that both the knowledge graph and the LLM use to understand the domain. Unlike database schemas that just define storage structure, this ontology defines meaning.

2. The Knowledge Extraction Notebook (text2knowledge.ipynb)

The Jupyter notebook demonstrates ontology-aware extraction:

Initialize the OpenAI client for GPT-5 access, showcasing that extraction requires advanced language models with deep reasoning capabilities.

Load the formal ontology, demonstrating how semantic structure guides the entire extraction process.

Load the mixed-content corpus containing wildlife jaguars, Jaguar cars, and Fender guitars — the deliberately challenging dataset.

Perform the extraction by providing both ontology and corpus to GPT-5, demonstrating pure concept-aware filtering without any manual post-processing.

Explain why this approach requires RDF and formal ontologies, breaking down the technical reasons LPG cannot replicate this capability.

The notebook’s key insight is that the LLM uses the ontology’s formal class definitions, property constraints, and hierarchical structure to understand which “jaguars” are relevant, automatically filtering out cars and guitars.

3. The Agent Implementation

The agent files demonstrate an MVP semantic Graph RAG:

jaguar_query_agent.py shows how to configure an agent with ontology awareness. The system prompt instructs the agent to always consult the knowledge graph for jaguar queries, base all SPARQL generation on the provided ontology, show queries for transparency, and format responses with markdown.

jaguar_tool.py demonstrates embedding ontological knowledge in tool descriptions. The function docstring includes the complete ontology specification, enabling the LLM to generate semantically valid queries. The tool handles SPARQL execution against GraphDB, manages authentication and error handling, and returns structured JSON results.

main.py shows the minimal entry point enabled by Microsoft Agent Framework, proving that semantic Graph RAG doesn’t require complex application infrastructure.

4. The Data Files

jaguars.ttl contains actual RDF instance data about specific jaguars — El Jefe, Macho B, Sombra, and others. Each instance conforms to the ontology structure, demonstrating how formal semantics work in practice.

jaguar_corpus.txt contains the mixed-content text used for extraction experiments, deliberately including automotive and musical content to prove the disambiguation capability.

Part VIII: The Broader Implications

Semantic Web’s Time Has Come

The Semantic Web vision from the early 2000s was ahead of its time. The tools existed (RDF, OWL, SPARQL), but the reasoning engines weren’t powerful enough.

LLMs changed the game.

Now we have AI systems that can understand natural language queries, generate formal semantic queries from user intent that reason over ontological structures, and explain the reasoning in natural language.

Formal ontologies + LLMs = The Semantic Web, realized.

The Path Forward for Graph RAG

This project demonstrates five critical principles for building effective Graph RAG systems:

1. Start with domain ontologies

Use existing standards where possible (Schema.org, industry-specific ontologies). Define clear class hierarchies that capture domain knowledge. Specify property constraints that validate data. Document semantic relationships with formal definitions.

2. Choose RDF-based storage

Options include Ontotext GraphDB for commercial applications with excellent reasoning, Apache Jena for open-source Java-based solutions, Amazon Neptune for managed cloud services, or Stardog for enterprise-focused deployments.

3. Leverage SPARQL for queries

SPARQL is more expressive than Cypher for semantic queries, standardized across all RDF stores, includes built-in reasoning support, and handles complex graph patterns naturally.

4. Design for ontology evolution

Knowledge grows continuously over time. Ontologies should be versioned like code. Use modular ontologies for different domains. Plan for cross-ontology alignment as systems integrate.

5. Enable LLM-ontology collaboration

Provide complete ontologies in tool descriptions so LLMs understand the domain. Let LLMs generate SPARQL dynamically based on user intent. Use semantic validation to catch errors before execution. Build feedback loops for continuous improvement.

Beyond the Hype

Graph RAG is powerful, but only if we do it right. Calling an LPG database a “knowledge graph” doesn’t make it semantic.

The AI community needs to understand the difference between LPG and RDF, recognize when formal ontologies are necessary, use W3C standards for interoperability, build semantic infrastructure that scales, and teach these concepts in AI/ML curricula.

This project provides a working template that demonstrates all of these principles in practice.

Conclusion: Knowledge Graphs Are Data for AI

The viral response to my LinkedIn post revealed a hunger in the AI community for real semantic infrastructure. People know intuitively that there’s a difference between:

Graph databases (storage of connected data)
Knowledge graphs (machine-understandable domain knowledge)

The difference is formal ontologies.

With RDF, SPARQL, and OWL, we can build Graph RAG systems that:

Understand domain concepts (not just keywords)
Reason over explicit semantics (not just patterns)
Explain their answers (not just generate text)
Interoperate across systems (not just work in isolation)

This isn’t theoretical — it’s working code, proven with the “Jaguar Problem” demonstration. The LLM successfully extracted wildlife conservation data from a mixed corpus, filtering out cars and guitars, purely through ontological understanding.
That’s the power of semantic Graph RAG.

The project is open source at

https://github.com/nemegrod/graph_RAG

The techniques are proven. The standards are mature. Now it’s time to build.

Key Technologies:

Microsoft Agent Framework — AI orchestration platform
Ontotext GraphDB — RDF triple store with reasoning
W3C Standards — RDF, SPARQL, RDFS, OWL specifications