“We’re killing Datadog.”
A VP of Engineering said that to a room full of nodding engineers three months ago. Series C, big planning meeting, everyone relieved that they were finally going to move the whole mess onto Grafana Cloud and stop feeding the bill that had been quietly wrecking the budget every quarter. I checked in last week. They’re now running Datadog, Grafana Cloud, and somehow New Relic on top of both, and observability spend is up 40% from the day they decided to cut it.
That’s not an outlier. It’s the median outcome.
Every team I’ve worked with in the last two years calls itself “actively consolidating,” and every single one is still running two or three observability stacks in parallel. Consolidation is the project that’s permanently in progress and never finished, and the reason it never finishes has almost nothing to do with the migration itself, which is annoying but genuinely a solved problem, the sort of grind a decent platform engineer gets through in a sprint or two. It stalls because tool sprawl is a people problem wearing a technical costume, and you cannot close a people problem with a migration ticket.
Sunk cost, dressed up as diligence
“We already spent six months setting up Datadog” is the line I hear in every consolidation meeting, and it’s almost always said by the engineer who personally spent those six months, the one who wired up the dashboards and the custom queries and the panel layouts that quietly encode a year of hard-won team knowledge nobody ever wrote down. I get it. Nobody wants to delete their own work. But that six months is spent whether you keep paying or not, the only question that matters now is which single tool you want to be living in two years from today, and the part people won’t say out loud is that you’re going to spend another six months learning whatever you move to anyway, so the real choice isn’t keep-versus-throw-away, it’s run one bill or run two while you do the exact same learning curve either way.
Engineers fall for this harder than anyone else does, because the sunk thing is their own craft.
Politics nobody says out loud
Here’s a real Slack thread, lightly anonymized:
#platform-eng
Sarah: What are we using for tracing again?
Mike: Datadog APM
Chen: I thought we moved to Honeycomb?
Mike: Backend did. Frontend uses Sentry.
DevOps: Prometheus, obviously. It's free.
Sarah: ...so all of them?
Everyone in that thread has a defensible reason. Backend likes Honeycomb’s high-cardinality querying, frontend standardized on Sentry years ago and won’t relearn anything, DevOps refuses to pay for what Prometheus already does for nothing, and each of them is optimizing for their own day while not one person is optimizing for the company. Leadership lets it ride under the banner of “we trust our engineers,” which is a lovely sentiment that holds up exactly until an incident crosses three teams at once and every team is heads-down in its own tool with no shared trace ID and no common timeline, three groups in the same war room shouting different numbers because their dashboards can’t even agree on what “now” means. That one took four hours. One stack with one trace ID and it’s a twenty-minute incident, probably less.
The “what if we need it later” hoard
What if we need that data someday. What if the new tool is missing this one feature. What if we’re wrong about all of it. So you keep everything, forever, just in case.
One Series B I worked with was quietly spending eight thousand dollars a month on tools nobody had logged into in months: standby tools, backups for the backups, monitoring for problems that might theoretically show up one day. We switched every one of them off in a single afternoon and then sat around waiting for something to fall over, and nothing did, and honestly nobody even noticed. That particular pile of “what if” was costing them ninety-six thousand dollars a year to feel slightly safer.
Consolidation that breeds more sprawl
This is my favorite version of the failure, because it’s so dumb and so common I can almost set a watch by it. A team kicks off a migration onto Grafana Cloud to save money, the migration runs long the way every migration runs long, and meanwhile a brand-new team spins up, needs monitoring this week and not next quarter, and reaches for Datadog because it’s the single fastest thing on earth to stand up. Now you’re paying for both, plus the New Relic trial somebody started back in March and never cancelled, and your cost-saving project has manufactured more sprawl than it removed. You tripled your lock-in instead of escaping it. The Datadog dashboards won’t port, the Honeycomb queries are in a syntax that travels nowhere, the Sentry integrations all have to be rebuilt by hand.
Pick one. Go deep on it. If you genuinely have to switch again later, fine, at least you only pay the switching tax once instead of carrying three of them forever.
The cost that never lands on a slide
The licenses cost real money, somewhere between fifty and two hundred grand depending on scale, and that’s the number finance circles in red. It is not the expensive part.
The expensive part is your on-call engineer at 2am. The page fires. They open Datadog, check APM, nothing jumps out, then over to Sentry to check the errors, maybe related and maybe not, then CloudWatch for the Lambda metrics, then Honeycomb for the traces that actually show the request path, and every one of those jumps is two or three minutes of re-authenticating their own brain and rebuilding the whole mental model from scratch in a UI that names everything differently, which adds up to roughly ten minutes of pure overhead across four tools before the debugging they got paged for has even started. At 3am with customers screaming, those ten minutes feel like an hour. You pay that tax on every incident, with your most expensive people, on the worst night of their month.
What actually works
Stop running this as a tool evaluation. It’s a leadership call, and it needs someone who’s actually allowed to make one.
Pick an owner: one person, not a committee, with the authority to say “we use X, full stop” and have it stick. Give them a real date instead of “when we have bandwidth,” and when that date lands you turn the old tools off even if someone’s still mid-migration, because a deadline that slides was never a deadline. Accept up front that you’re going to lose dashboards people loved, thank the people who built them, delete them, move on. Then track the numbers that prove it worked, MTTR before and after, context switches per incident, time to first useful signal, so that the next time someone lobbies to bring back their favorite tool you can put the cost on the table.
I’ve watched this actually succeed exactly twice. Both times a VP owned it personally, made it their explicit OKR, and ate the complaints when teams whined about losing their toys. Every other attempt I’ve seen is, as of this week, still running four tools and calling it “in progress.”
So if you want to be one of the two: name the owner now, put a kill date on the shared calendar, and quit pretending it’s an engineering project. It never was.
I wrote more about the hidden costs of observability chaos here.
— Youn