It's 2 a.m. Alert manager fires twelve pages for what turns out to be one root cause. The on-call engineer opens Grafana to find the root-cause dashboard the one built for exactly this moment timing out, because the query it needs to run crosses a cardinality wall nobody budgeted for. By the time the dashboard loads, the RCA has already been done by hand, the slow way: kubectl logs, three tabs of PromQL, a Slack thread full of guesses.
If that scene is familiar, you already know the uncomfortable truth about running your own observability stack: Prometheus, Grafana, and Loki are not neutral bystanders during an incident. They're systems you operate, with their own capacity limits, their own upgrade risk, and their own failure modes, and those failure modes have a habit of showing up exactly when you need the stack most.
Most of the AI SRE tools getting funded and talked about right now weren't built for this problem. They're built to sit on top of whatever observability backend you already have and reason over whatever data it hands back. That's a reasonable bet if your pain is "too many alerts, not enough hypothesis-testing." It's the wrong bet if your pain is "the observability system itself has become a reliability risk, and a line item nobody can explain." This post is about that second problem and how Aiden for SRE, running natively on Aiden for Observability (ObserveNow), is built specifically for teams who are staying on Prometheus, Grafana, and Loki rather than abandoning them.
Prometheus, Grafana, and Loki earn their popularity honestly. No per-host licensing, full control over retention, and query languages — PromQL, LogQL — that map cleanly onto how engineers already think about systems. The tradeoff shows up later, once you're past a few dozen services, and it shows up in two places at once: toil and money.
The toil. A few patterns repeat across nearly every self-hosted team we talk to:
(For a deeper look at cardinality and federation specifically, see our breakdowns of setting up a managed Grafana stack for Kubernetes monitoring and the top managed open-source observability solutions in 2026.)
The money. This is the part that rarely makes it into the architecture diagram. Teams in this position consistently end up dedicating one to several full-time senior SRE engineers hired and paid to improve application reliability to keeping the monitoring system alive instead. Run the math on a team carrying two senior SREs at a loaded cost north of $200K each, and self-hosted observability quietly becomes a seven-figure line item that never shows up on a vendor invoice, which is exactly why it doesn't get scrutinized the way a renewal bill does. Add the storage and compute cost of retaining high-cardinality metrics and log volume at scale, and "free" software is often the most expensive tool in the stack, just billed in headcount and opportunity cost instead of a subscription.
None of this shows up in a demo. It shows up eighteen months in, when service count has quadrupled, and the team that built the stack is now the team keeping it on life support, work that has nothing to do with the reliability of the systems it was built to watch.
Look at the current AI SRE landscape, and a pattern emerges quickly. Tools built as investigation overlays are explicitly designed to be vendor-neutral: they connect to whatever observability platform you already run Prometheus, Datadog, Elastic, a dozen others and reason over the telemetry those systems return. That's a deliberate design choice, and a good one for a specific problem: reducing the manual work of hypothesis testing during an investigation.
But it comes with a structural limitation: these tools are usually upfront that output quality depends entirely on integration coverage and the underlying telemetry. If your Prometheus instance is timing out, missing series past a cardinality wall, or federating across shards in a way that drops context, an agent reading through that integration inherits the exact same blind spot a human on-call engineer has. These tools reason brilliantly about your incident. They do nothing about the monitoring system's own operational health, because that was never in scope they treat Prometheus, Grafana, and Loki as a data source to query, not as infrastructure to manage. And none of them touch the cost side of self-hosting at all; they assume your observability backend is a fixed cost, not a variable one you're actively trying to bring under control.
At the other end of the spectrum, Kubernetes-native, eBPF-based AI SRE tools take the opposite approach: they collect their own telemetry directly from the kernel, sidestepping the observability backend entirely. That's genuinely powerful inside Kubernetes. It's also the source of the main limitation: value drops sharply for anything running outside Kubernetes, and kernel-level instrumentation requires privilege and compatibility assumptions not every environment can grant.
Somewhere in between sit tools with deep, proprietary, platform-native access- genuinely strong products, but tied to a specific commercial observability platform's data model, which doesn't help a team that has deliberately chosen to run and control its own OSS stack.
What's missing from all three categories is a tool built for the actual shape of this problem: teams staying on Prometheus, Grafana, and Loki, not migrating off them, not replacing them with a proprietary backend, who need both the toil and the cost of running that stack to stop landing on their SREs.
Aiden for SRE runs on top of ObserveNow, StackGen's managed control plane for Prometheus, Grafana, and Loki. The distinction matters: instead of treating your observability stack purely as a read-only data source, Aiden operates with native context about how that stack is built, sharded, retained, and priced because ObserveNow is the layer managing it. If you're coming at this from the infrastructure side rather than the incident-response side, it's worth also reading how Aiden for Infrastructure and Aiden for SRE share the same underlying reasoning engine, just pointed at different parts of the lifecycle.
That gives Aiden three things a purely observability-agnostic tool doesn't have by default:
"Every upgrade to our monitoring stack feels like a production deployment. Because it is one." The kind of line every platform team running self-hosted Prometheus eventually says out loud.
The diagram below shows the shape of the system: your existing Prometheus, Grafana, Loki, and Alertmanager stay exactly where they are. ObserveNow sits underneath as the managed control plane, handling federation, cardinality, retention, and upgrades. Aiden for SRE sits on top, doing the reasoning and remediation work: correlating alerts, walking the query layer to a root cause, and either suggesting or executing a fix within guardrails your team defines.
Walk through a single incident to see how the pieces connect.
1. Alert ingestion and correlation. Aiden subscribes to Alertmanager's routing tree directly, rather than polling a generic webhook. When a service degrades, it doesn't see twelve unrelated pages; it sees twelve alerts sharing a label lineage, and collapses them into one incident with a shared root before anyone gets paged.
2. Cross-signal root cause analysis. Aiden queries PromQL for the metric anomaly, LogQL for the surrounding log context in Loki, and any available trace data, then builds a ranked set of hypotheses rather than a single guess. Because it's querying the same PromQL and LogQL your team already writes, the reasoning is legible an SRE can read the exact queries Aiden ran and verify them in Grafana in seconds.
3. Stack-aware filtering. Before surfacing a hypothesis, Aiden checks the observability system's own state: is this metric gap real, or a cardinality-related sampling artifact? Is this dashboard timeout a symptom of the incident, or a known federation bottleneck that's been there for weeks? Generic tools working purely off returned data can't reliably tell the difference; Aiden can, because it has visibility into ObserveNow's operational state.
4. Remediation. Depending on the guardrails your team sets, Aiden either drafts a remediation PR, executes a pre-approved runbook action, or hands a ranked, evidence-backed hypothesis to the on-call engineer. This is intentionally configurable: teams earn their way from read-only suggestions to automated action as trust builds, which is the same trajectory most AI SRE deployments follow across the industry.
5. Knowledge capture. Every investigation feeds back into institutional memory: what the root cause was, which query sequence found it, whether the automated fix held. Over time, this closes the "only two people understand this system" gap that shows up repeatedly as teams scale past their original observability design.
For a real example of this flow, our own SRE team has documented how they run Aiden against StackGen's own multi-cloud production estate, including how Aiden closed out a Kubernetes "Node Not Ready" alert without a human touching kubectl.
No tool erases the operational reality of running open-source infrastructure. A few things worth stating plainly:
Industry data on GenAI-driven SRE gives a useful reference point for the category, even outside StackGen specifically: internal deployments at companies like Uber report tens of thousands of engineering hours saved annually, and RCA-focused tools have published reductions in investigation time in the range of 50–70% for teams that adopted correlation-first workflows. The mechanism is consistent across these results: the time saved rarely comes from one dramatic fix. It comes from removing compounding, chronic toil: alerts that no longer need manual deduplication, root-cause investigations that start from a ranked hypothesis instead of a blank dashboard, and a monitoring stack that stops generating its own incidents.
For teams running Prometheus, Grafana, and Loki specifically, the shift is both operational and financial: SREs stop being the people who keep the monitoring system alive, and the cardinality and retention decisions that quietly inflate self-hosted observability spend get surfaced before they hit the next capacity review, not discovered in it. Our own team has broken down the specific cost levers this touches cardinality audits, tiered retention, and tool consolidation in more depth in how to reduce observability costs with managed open-source tools.
If you're evaluating AI SRE tools and your pain looks like alert fatigue plus a self-hosted observability stack that's become its own maintenance project and its own budget line, ask any vendor a direct question: does this tool understand my monitoring system as infrastructure I pay to run, or only as a data source to query? Investigation-focused, observability-agnostic tools are strong at the latter. Kubernetes-native tools are strong inside Kubernetes. Aiden for SRE, running on ObserveNow, is built for the specific and common case of teams who are keeping Prometheus, Grafana, and Loki and want both the toil and the cost of running them to stop landing on their SREs.
Key takeaways:
Ready to see how Aiden for SRE handles your Alertmanager routing tree and your cardinality bill? Schedule a demo, try Aiden for SRE Community Edition free, or read more on how ObserveNow manages Prometheus, Grafana, and Loki at scale.