Let me start with a scene you will recognize. The PagerDuty alert fires at 2:07 AM. You join the incident bridge half-asleep. The dashboards are screaming. Five services are red. The on-call engineer, who has been with the organization for all of eight months, is staring at metrics like she is looking at modern art. She has no idea where to start. She starts checking everything. Randomly. She is not lazy or incompetent. She just does not have the institutional knowledge to know that when the payment service starts throwing 5xx errors at this rate, it is almost always the downstream authentication cache that has gone stale, which happened once before during a traffic surge eighteen months ago. The only person who knows that is you. And you are not always available at 2 AM.
Now replace that on-call engineer with an AI Agent. Does the problem go away?
Not even close. Not unless you give that AI Agent a Knowledge Graph to reason with.
This is the fundamental challenge sitting at the center of AI-powered DevOps and SRE today. And it is the challenge I want to explore in this post, because I think the industry is making the same mistake with AI Agents in operations that it made with the first wave of AI chatbots: throwing generative AI at a problem that requires structured, contextual knowledge, and then wondering why the outputs are inconsistent, unreliable, or just wrong when it matters most.
Before we go further, let's be precise. Buzzwords get diluted fast in this industry. Don't even get me started on "DevOps" as being the canonical example, and "AI Agent" being well on its way. So let's define things from first principles.
A Knowledge Graph is a structured representation of entities and the relationships between them. It is a graph in the mathematical sense - it has nodes and edges. Nodes represent entities (a service, a team, a deployment, a configuration item, an incident, a policy) and the edges represent the typed, directional relationships between those entities ("service A depends on database B," "pipeline X deploys to environment Y," "incident Z was caused by config change Q").[1]
What makes a Knowledge Graph different from a database? A relational database stores facts in rows and columns. A Knowledge Graph, while it could be implemented using a database, stores meaning. It captures not just the data, but the semantic context of how data elements relate to each other across domains. It is fundamentally designed for multi-hop reasoning, that is the ability to answer questions that require walking across multiple relationships: "If I change service A, what downstream services are affected, which teams own them, and what SLOs are at risk?" That is a four-hop traversal. A relational schema will make you cry trying to query it. A Knowledge Graph, likely implemented leveraging a graph database, makes it a first-class operation.[1][2]
What makes a Knowledge Graph different from a Service Graph? This is an important distinction, especially as service graphs have become a core part of observability platforms. A service graph maps runtime interactions between services—who calls whom, request flows, latency paths, and dependencies observed from telemetry data. It is excellent for answering “what is talking to what right now” and for visualizing live system behavior during incidents. But it is limited to observed interactions and typically lacks deeper context. A knowledge graph, on the other hand, models relationships across a broader set of entities - services, infrastructure, ownership, policies, incidents, and even business context - using explicit, structured relationships. It doesn’t just show that service A calls database B; it can represent that database B is owned by team X, governed by policy Y, impacted by incident Z, and deployed in region R. For an AI agent that needs to reason, correlate signals, and take informed action across systems, not just observe traffic, that richer, connected understanding is what makes the difference.[1][2]
What makes a Knowledge Graph different from a vector database or RAG system? This is the most critical distinction to make in 2026, because RAG has become the default answer to "how do I give my AI agent knowledge?" Retrieval-Augmented Generation (RAG) retrieves relevant text chunks based on semantic similarity and feeds them to an LLM as context. It is fantastic for answering "what" and "when" questions by surfacing relevant documentation, runbook text, or past incident notes. But it cannot reliably tell you how things connect. RAG does not understand that service A depends on database B. It retrieves text that mentions both of them, which is not the same thing. For an AI Agent that needs to take autonomous action in production and not just answer a question, but actually do something, that distinction is everything.[2]
Here is the dirty truth about DevOps and SRE knowledge management: it is a disaster in almost every organization.
Think about where the actual knowledge lives that governs how your systems operate. Your runbooks, if you have them, are in Confluence. Some of them are three years stale. Your architectural decision records (ADRs) are in a wiki that nobody has touched since the architect who wrote them left the company. Your service dependency maps are in a CMDB that was last accurately populated by a consultant in 2022 and has been slowly drifting from reality ever since. Your real incident knowledge, the hard-won tribal expertise of what actually breaks and why, lives in Slack threads from past incidents, in postmortem documents that get written and never read again, and more often than not, solely in the heads of your senior engineers who are burning out.[2][3]
Let’s look at an analogy from Coding. AI coding assistants improving in-IDE code generation by 30% only makes the developer 6% more productive if they are only spending 20% of their time in the IDE. The same math applies to SREs. Your SREs are not spending all their time resolving incidents. They are spending enormous amounts of time reconstructing context: figuring out which service owns which component, which team is on-call for a dependency, what changed in the last 24 hours, and what the blast radius of a potential fix might be.[3]
That context reconstruction is what AI Agents should be eliminating. But they can only do that if the context is complete and properly structured and queryable in the first place.
DevOps environments are inherently relationship-heavy: pipelines, services, environments, teams, approvals, policies, artifacts, and dependencies all interact tightly. The relationships are not optional decoration on top of the data. The relationships are the knowledge. And only an up to date Knowledge Graph captures those relationships explicitly enough for an AI Agent to reason over them autonomously.[1]
Let me describe what happens when you deploy an AI Agent for incident response without a comprehensive Knowledge Graph underneath it. You get one of two failure modes, neither of which is acceptable.
The agent pulls context via RAG, retrieves some loosely relevant documentation and past incident summaries, and then uses the LLM to synthesize a root cause hypothesis. The hypothesis sounds authoritative. It cites specific services and possible causes. It is completely wrong because the RAG retrieval surfaced documents about a service with a similar name, not the service that is actually failing. The engineer follows the recommendation, wastes 20 minutes chasing a ghost, and then loses trust in the AI agent entirely.
Failure Mode 2: Paralysis by Uncertainty
The agent, perhaps more carefully engineered with guardrails, recognizes that it does not have enough structured context to act with confidence. It escalates to a human. Every time. The human is still woken up at 2 AM. The agent has added zero value. You have a very expensive on-call pager system with extra steps.
The difference between these failure modes and a genuinely useful AI SRE agent is the quality of the underlying knowledge structure. A senior SRE does not randomly check dashboards. They use their internalized understanding of the system to form hypotheses based on symptoms. It is the kind of expert intuition that comes from having a rich mental model of how the system works and how it has failed before. A Knowledge Graph externalizes that mental model and makes it machine-queryable.
So what does a Knowledge Graph for DevOps and SRE actually look like? Let me get architectural for a moment, because the implementation details matter.
At its core, a DevOps Knowledge Graph models five categories of entities and the relationships between them:
Infrastructure Entities: Infrastructure as Code (IaC), representing your physical and virtual servers, containers, pods, clusters, databases, network devices, cloud resources. These are your configuration items (CIs), the atoms of your CMDB.
Software Delivery Entities: the artifacts and processes of the pipeline: repositories, branches, commits, builds, artifacts, pipelines, deployments, releases. These entities connect the code to the infrastructure.
Operational Entities: the runtime behavior of the system: services, APIs, endpoints, SLOs, alerts, incidents, postmortems. These connect what is deployed to how it is performing. These should also represent what is the expected ‘noemal’ behavious of the system, as opposed to behavious that should raise an alert.
Human and Organizational Entities: teams, individuals, on-call schedules, ownership records, approval chains. An agent acting in production needs to know who owns what, who can approve what, and who to escalate to when it encounters a situation outside its autonomous scope.
Policy and Knowledge Entities: runbooks, ADRs, compliance requirements, change management policies, security controls. These encode the rules that constrain what the agent can and cannot do.
The CMDB has been trying to be this knowledge graph for decades. The difference is that traditional CMDBs are maintained manually and decay rapidly. A true DevOps Knowledge Graph is built to be continuously synchronized from live telemetry, pipeline events, deployment signals, and observability data. It reflects the current state of the system, not what someone thought the system looked like the last time they ran a discovery scan.
There is a dimension of the Knowledge Graph story that goes beyond structural topology, and I want to spend some time here because it is where the real long-term value accumulates.
Foundation Capital has articulated what they call the "context graph" as a layer that extends beyond knowing what your system looks like to knowing how decisions have been made about it over time. The insight is sharp: rules tell an agent what should happen in general. Furthermore, “Decision traces” are components of context graphs that capture what happened in this specific case, which policy version was active, what exception was granted, who approved it, and why.
This matters enormously for SRE. Think about your change management process. You have a written policy that says no changes to production on Fridays. But everyone on the team knows that there is an implicit exception for security patches above a certain severity threshold, and another implicit exception for revenue-critical hotfixes that get CTO approval. These exceptions live in people's heads and in buried Slack threads. They are never written into the change policy document. When an AI Agent encounters a situation that requires invoking an exception, it needs that institutional memory, not just the written rule.
Organizations that capture these decision traces are building something that does not exist in most enterprises today: a queryable record of how decisions were made, not just what decisions were made. Over time, these records form a context graph: entities connected by decision events, the moments that matter, and the "why" links that explain them. Every automated agent decision adds another trace to the graph. The feedback loop compounds. The agent gets smarter with every incident it handles because the graph gets richer.
This is the same compounding pattern we see in mature SRE organizations today, but happening at the speed of software rather than the speed of human memory. Postmortems contribute to a growing institutional knowledge base. But right now, that knowledge base is mostly in documents that are hard to query, hard to connect to live system state, and invisible to AI Agents making real-time decisions. A context graph extends it, making it machine-readable and traversable.
For SREs, this has a very concrete implication. The burnout problem in SRE is real. Nearly 70% of SREs report on-call stress as a direct cause of burnout according to the Catchpoint SRE Report 2025. A significant contributor to that stress is the cognitive load of reconstructing context during incidents. A mature context graph systematically reduces that cognitive load by externalizing it. The agent carries the context. The human focuses on the judgment calls the agent cannot yet make.
Let me be direct: not every organization needs a fully built knowledge graph on day one. But every organization that is serious about AI-powered operations needs a path toward one. Here is how I think about the progression, grounded in First Principles.
Start with the entities you already have. Most organizations have some form of service registry, CMDB, or service catalog. It is probably incomplete and partially stale. That is OK. The goal is not to boil the ocean. It is time to begin modeling the relationships between what you already track. Connect your service catalog to your monitoring system. Connect your deployment pipeline to your service catalog. Connect your incident history to your services. Even a partial graph with accurate relationships is dramatically more useful than no graph at all.
Make telemetry a source of truth. The reason CMDBs decay is that they depend on manual maintenance. A Knowledge Graph for DevOps should be continuously populated from live telemetry - service mesh data, OpenTelemetry signals, deployment events, and infrastructure state changes. Beyla, Istio, Linkerd, and similar tools can feed this automatically. When the graph reflects the live system, it stays accurate by design, not by discipline.
Instrument your agents before you scale them. Before your AI Agents start making autonomous decisions in production, instrument them to emit their reasoning traces back to the knowledge graph. What context did the agent retrieve? What hypothesis did it form? What action did it take? What was the outcome? These traces are not just audit logs; they are training data for the context graph. Every agent run makes the graph more accurate.
Treat the graph as a first-class engineering artifact. This is the cultural shift that matters most. The Knowledge Graph is not an IT ops tool or a compliance artifact. It is infrastructure for your AI agents, as fundamental to their operation as the LLM or the MCP server. It needs owners, SLOs, and a maintenance culture. Organizations that treat it as a project artifact will end up with the same stale CMDB problem they have always had. Organizations that treat it as a living system will compound its value continuously.
Let me ground this in some real numbers, because I believe some teams are still treating the Knowledge Graph as a "nice to have" rather than a prerequisite for serious operational AI.
A 2025 SolarWinds report found that AI-assisted incident response saves an average of 4.87 hours per incident, with leading implementations achieving MTTR reductions of 30 to 70%. Across hundreds of incidents per year, that is an enormous amount of engineering time reclaimed from firefighting. The biggest MTTR bottleneck in 2026 is not alert detection — that is essentially a solved problem. The bottleneck is coordination overhead and context assembly: the time spent figuring out who owns what, what changed, and what the blast radius is before actual troubleshooting even begins.
That context assembly time is precisely what a Knowledge Graph eliminates. And consider the onboarding dimension: a new engineer often takes three months or more to build enough institutional knowledge to be effective during a major incident. A Knowledge Graph that has ingested years of incident history, postmortems, architectural decisions, and system topology compresses that timeline dramatically. The knowledge is no longer locked in the heads of senior engineers who can only be in one place at a time.
There is also a structural asymmetry worth noting. DevOps and SRE exist as organizational functions precisely because no single system of record owns the cross-functional workflow. Development, operations, infrastructure, security, compliance - these domains intersect continuously in modern software delivery, and the context that governs their intersection lives nowhere explicitly. Someone has to carry it. Today, that someone is a human, usually a senior engineer who is on-call too frequently and is burning out. Tomorrow, that carrier of cross-functional context should be a Knowledge Graph.
I will not leave you with unqualified enthusiasm for Knowledge Graphs because that would not be useful. There are real challenges here that deserve acknowledgment.
Building and maintaining a Knowledge Graph is non-trivial. Ontology design, deciding what entities and relationships to model, requires expertise and iteration. Get it wrong, and you build a graph that is technically correct but not useful for the questions your agents need to answer. Fortunately, the telemetry-driven approach (building the graph from live signals rather than manual entry) reduces this burden significantly, but it does not eliminate the need for thought about what relationships matter.
Data quality problems do not disappear. A Knowledge Graph built on top of inconsistent, duplicated, or incorrect source data will propagate those inconsistencies into agent reasoning. Garbage in, garbage out, but now the garbage is confidently delivered by an autonomous agent acting in production. This is why the observability and eval layers are not optional additions. They are how you catch graph quality problems before they manifest as agent failures.
The context graph is still an emerging standard. The decision trace layer, capturing why decisions were made, is less mature than the structural topology layer. Standards for decision trace schemas are still being developed, and cross-system precedent queries are hard today because every platform captures decisions differently. The analogy to OpenTelemetry is apt: OpenTelemetry standardized observability traces across tools and vendors; we need something similar for decision traces. It will come. But in the interim, organizations that build in proprietary formats are accepting future migration risk.
AI has shifted toil, not always eliminated it. The Catchpoint 2025 SRE Report recorded that toil actually rose to 30% in 2025, the first increase in five years, as teams wrestled with model tuning and what some are calling "AI babysitting". The implementation strategy matters as much as the technology. Deploying AI Agents in operations without proper grounding, evaluation frameworks, and knowledge infrastructure does not reduce toil. It creates new toil in the form of debugging agent misbehavior. The Knowledge Graph is not a shortcut around this work, but a prerequisite for doing this work correctly.[10]
If you are a DevOps or SRE leader reading this and you have not started building a Knowledge Graph, here is the honest advice: you are already behind. But the gap is closeable, and the pragmatic first step is not to build a perfect knowledge graph, it is to stop letting your operational knowledge evaporate.
Start with postmortems. Make them machine-readable. Map the entities they reference, the services, teams, infrastructure components, to a service catalog. Connect that catalog to your observability tooling. Enable automatic relationship discovery from your service mesh telemetry. You do not need a graph database on day one. You need the discipline to treat operational knowledge as a first-class engineering artifact, and the architectural intent to make it queryable by your agents.
The organizations that will lead in AI-powered operations over the next three years are not the ones with the best LLMs. The LLMs are now largely a commodity. They are the organizations whose AI Agents have the richest, most accurate, most current knowledge graph to reason over.
Build the graph. Build it now. Your next 2 AM on-call rotation will thank you.
Disclaimer: No AI Agent was paged at 2 AM in the writing of this post. Though one did help review the draft.