Travis Barlow
Subject matter expert
Overview
A few years ago the pitch was everywhere. AI would read the queue, clear the noise, surface the handful of alerts that actually mattered, and give your analysts their nights back.
I heard it at every conference. I made a version of the argument myself, more than once, to boards who wanted to believe it.
So here is a fair test. Walk into a working SOC tonight and ask whoever is on shift whether AI fixed the queue.
The queue is still there. The dashboard is still orange. There is a new tab open next to it, the AI assistant, and most of the time nobody is looking at it.
This is not a story about AI being dangerous. It is a story about AI being disappointing, which in some ways is the harder thing to talk about. The tool gets bought, it gets deployed, the rollout is reported as a success, and the value never shows up on the floor.
The short version
What adoption says
- Many SOCs have bought or deployed AI and ML tooling.
- Security leaders still expect AI to absorb a major share of SOC work.
- AI has become a boardroom answer to staffing pressure and queue fatigue.
What the floor says
- Analyst satisfaction remains weak where tools are inaccurate or unowned.
- The hardest alerts still need human judgment.
- A tool nobody trusts becomes another console to supervise.
The problem is not that AI has no value in the SOC.
The problem is that many teams bought it like a finished product, skipped the integration work, and then expected analysts to trust it during the hardest hours of the shift.
What the numbers actually say
Start with the gap between who likes these tools and who uses them.
In the SANS 2025 SOC Survey, roughly 40% of security operations centers reported using AI and machine learning tools. That sounds like adoption winning. But in the same survey, those AI and ML tools ranked at the bottom of the technology satisfaction list, below the boring, decade-old tools nobody writes blog posts about.
Forty percent are using it. Almost nobody is happy with it.
That split is the whole problem, and it matches what I have seen. The people who bought AI for the SOC and the people who work the SOC are describing two different products.
Meanwhile, the expectations have not come down to meet reality. Prophet Security's 2025 work on the state of AI in the SOC found that security leaders still expect AI to handle around 60% of SOC workloads within three years. So the people signing the contracts are forecasting that AI will soon do most of the job, while the people doing the job today cannot get it to reliably do a slice of it.
That is not a small gap. That is a leadership team and a night-shift team living on two different planets.
None of this is unique to security. MIT's Project NANDA looked at enterprise AI across the board in 2025 and found that roughly 95% of generative AI pilots returned nothing measurable. The SOC is one room in a very large building where the same thing is happening.
The difference in our room is that a tool nobody trusts does not just waste money. It leaves people believing something is being watched when it is not.
Why the value never showed up
“AI is overhyped” is the lazy version of this, and I am not interested in it. The useful question is why the value never made the trip from the slide deck to the desk.
In the cases I have seen up close, the failures were human, not technical.
We bought the capability and skipped the work
A lot of these tools went in straight out of the box. No tuning, no customization, no owner. We treated AI the way we treat antivirus: license it, install it, move on.
But a model that does not know your environment, your naming, or your normal is a confident stranger making guesses.
We automated the easy part and forgot who owns the rest
The promise was that AI would clear the low-value alerts so people could focus on the real ones. Sometimes it does. But the alerts it cannot resolve do not disappear. They pool.
Now there is a second job nobody budgeted for: deciding whether the AI got it right.
Analysts work out the first problem inside a week. Once they learn the tool is wrong often enough that checking it costs more than doing the job themselves, they stop checking. The licence renews anyway. The tool just sits there.
I have watched teams come out of an AI rollout more tired than they went in, because they are doing their old work plus supervising a junior analyst who never learns and never goes home.
We forgot that trust is earned at 3 a.m., not in procurement
A tool that cries wolf gets muted. This is the oldest pattern in the business. It is why nobody reads the SIEM rule that fires two thousand times a day.
Every false summary, and every confident “this is benign” on something that was not, spends trust the tool cannot easily win back.
SOC analysts are professional skeptics. Being skeptical is the job. You do not convert them with a demo. You convert them by being right when they are too tired to double-check, and most of these tools have not been right often enough.
The value is unrealized, not unrealizable
Let me argue against my own headline for a paragraph, because the distinction carries weight.
There are narrow jobs where AI earns its place in a SOC right now:
- Collapsing a long incident timeline into something a human can read in thirty seconds.
- Writing the first draft of the report nobody wants to write.
- Pulling context from ten consoles so the analyst is not tabbing between them at midnight.
- Finding a pattern across more telemetry than a person can hold in their head.
That is not 60% of the workload. It is closer to 10%, and it is the 10% that makes the rest of the shift survivable.
The teams getting real value out of AI have one thing in common, and it is not the logo on the contract. They treated the tool as something that makes their people better, not something that replaces them.
They gave it an owner. They tuned it to their environment. They checked whether it was actually right before they trusted it with anything that mattered. They bought it the way I keep arguing you should buy an MSSP or an MDR: skeptically, with questions, and with someone accountable for the answer.
What I would do about it
If you are looking at an AI line item and wondering why the SOC is no calmer than it was last year, a few honest questions will tell you more than another pilot.
Ask your analysts, not your vendor and not your dashboard, whether the tool makes the shift better. If they have quietly stopped opening it, you already have your ROI report.
Ask whether it was ever tuned to your environment or whether it is still running the way it shipped. If it shipped and stayed that way, you do not have an AI problem. You have an implementation you never finished.
Pick the small jobs, the summarizing and the enrichment and the first drafts, and measure those honestly instead of forecasting a future where AI runs the floor. Earn the trust on the small things before you bet anything large on it.
And stop counting deployment as a win.
“We rolled it out” is not a result. A tool everyone has and nobody trusts is worse than no tool, because it puts an AI-shaped gloss over a queue that is still humming at three in the morning.
AI is not failing in the SOC because it is fake. It is failing because we bought it like a product and never did the work to make it ours.
That work is still there to do. We just have to stop pretending we already did it.
Sources
- SANSSANS 2025 SOC Survey
- Prophet Security6 Key Takeaways from the AI in SOC Survey Report
- MIT Project NANDAThe GenAI Divide: State of AI in Business 2025