In IT, the “playbook” is our security blanket. It’s a checklist for what to do based on past experiences of a similar issue. Unfortunately, these playbooks only consider human operators and the extent to which humans can work sequentially.
When there’s an IT problem, the business ultimately only cares about the outcome. If an agent can fix it, great. If not, can the human team help? Agents can parallel-process billions of logs at once, rather than 10 at a time. It’s time to rewrite playbooks with an agent-first, self-healing mindset.
Agents Do What Humans Won’t, or Even Can’t
Imagine a human trying to look at billions of logs at 3 a.m. when something’s broken. It’s incredibly difficult, mind-numbing, error-prone work. An AI agent won’t ever be tired and can do it in minutes.
AI is great at pattern matching, finding mistakes, and figuring out correlations. I’ve witnessed agents, with the right context, flag config changes that passed human reviews but would have led to something catastrophic.
20 years ago, IT was dealing with one infrastructure, a fixed set of apps, and a permutation combination that was manageable. The IT surface area is now exploding so quickly that it’s arguably becoming unmanageable for humans. How are you going to monitor agents? You can’t have humans monitor agents. Like poring through billions of logs, agents monitoring agents is another no-brainer use case.
It normally takes a day or two for humans to put together a monitoring system. They have to log into all the tools, parse everything, and make it human-readable. AI can do all that for you, plus map your infrastructure’s topology, from your servers to your switches to your routers. Nobody on your team knows this. It’s a black box. A well-grounded AI tool can not only decipher all your IT infrastructure connections, but it can also tell you what’s problematic about your traffic flows and offer fixes.
Notice that I didn’t once mention a dashboard, each of which is created with the premise that a human will be looking at it. If you do want a dashboard, you can still ask your AI tool in plain language, “Hey, what’s going on in this alert map, and can you please generate a dashboard?” This means you no longer have to wait two weeks for IT to spin up your dashboard request, and your IT team is no longer inundated with them.
When Machine-to-Machine Makes Sense
Do you really need humans flipping through dashboards when network packets are dropping or your latency is spiking? Is that the best use of human eyeballs?
Agents will help us usher in a world of self-healing. Say you’re an app developer and some ISP provider is having network issues. An agent should be able to figure out the problem and have another agent redirect the traffic. Or, imagine an agent notices that your website is loading slowly in a certain part of the world. Another agent should be able to figure out the cause and fix it.
Network losses, network traffic, application latency, metrics, logs, and traces all ultimately come down to machine-generated data. Because it’s all machine-generated, it’s also all machine-resolvable. This is what agents were built for.
In truth, even if agents weren’t as helpful as they are, humans’ abilities no longer suffice. With AI generating more code and requiring more compute, humans can’t verify everything. AI is not only a lot more effective at detecting patterns across trillions of metrics to find the root cause and fix it, but humans can’t even keep up.
Compounding Time Wasted and Time Saved
Consider the July 2024 outage caused by one CrowdStrike change. Once the faulty update was pushed live, it was too late. The cascading effect was massive: roughly 8.5 million Windows PCs crashed and couldn’t restart properly.
Picture the classic war room during an outage: 200 people get on a call, and everybody professes their innocence: It’s not the application, it’s not the network, and it’s definitely not my stack! There’s even a metric to describe this phenomenon: mean time to innocence.
After coding, troubleshooting in production could be the next major growth area for AI. Troubleshooting is ripe for disruption because AI can avoid having 200 people gather on the same call. It will tell you, with 95% accuracy, “Here’s the root cause, go investigate that path.”
If it’s right, you’ve saved 200 people two days of work. In the CrowdStrike example, remember that every team had to do its own investigation. It wasn’t just CrowdStrike or Microsoft investigating, but every IT team managing Windows machines with CrowdStrike around the world had to do an investigation.
Imagine the compounding effect of the productivity lost with that one outage and how AI could have slashed the time and resources needed to resolve it for everyone.
Don’t Wait for Perfection to Start
The industry has overpromised a bit on AI; clients tell me they’re skeptical of tools like Microsoft Copilot. Our job as AI tool vendors is to make it straightforward to see the before (human playbook) and after (agent playbook). Here’s the investigation your team is running without AI. Do you really want to hire 100 engineers to do this? Do you think that’s a good use of their time, and what’s the risk of getting the investigation, let’s say, 5% wrong?
There’s a high bar for AI to always be right. Instead of asking if the accuracy is 100% all the time, you should be asking, “What’s the trade-off?”
You’re running a business. If humans can be wrong, an AI-generated answer can be wrong too. Your human team is likely wrong more often than the AI tool, especially if you consider that they can’t get to most of the work anyway. Agents will get to all the work and let their human counterparts address any mistakes.
This requires a mindset shift. Successful companies are the ones willing to take the plunge and iterate quickly. Throw out your human playbook and stop waiting for AI to be perfect. Deploy now, get the business benefit right away, start writing your agent playbook, and remember that AI will only get better.
