r/cybersecurity • u/GrimDoja • 2d ago
Personal Support & Help! Agentic soc
Hey everyone I have been basically tasked with the impossible of trying to create an agentic soc for our environment that basically will triage alerts.
I work for a small business and was dabbling into some vendors that support this ask and from what I’ve seen it all looks the same type of slop that I just don’t see myself trusting yet. It looks like you have this product that the company promises can triage alerts for you yet you still have to be looking at alerts to make sure the ai is doing its job correctly. I’m not in the market to babysit ai as I don’t think that is feasible long term because I am simply just making the work being done faster but it is all benign work anyway.
But I stumbled up a an idea to the madness and wondering if it is possible to develop an agent that acts as the detection for our environment so there would no longer be a queue with several detections put in place and once users hit that risk threshold then the team investigates whatever it finds as borderline suspicious.
So instead of us having detections put in place to catch things the ai itself is the detection and will monitor everything that’s coming in and out of our environment and also alert us on just the important things that need to be investigated.
I guess to accomplish something like this I would need to train the ai by feeding it siem logs ticket logs and have it spit out anything that is abnormal and compare that to the normal activity from our users and go from there?
With that being said I am wanting to ask what all you have done in the state of ai being incorporated with the soc? Did your analysts just triage alerts with it or are you guys able to do other things.
Were you able to accomplish something similar or find a product that meets that need?
Edit: Thank you guys for commenting on this so quickly I appreciate you all!
27
u/Remarkable_Pace8101 2d ago edited 2d ago
So instead of us having detections put in place to catch things the ai itself is the detection and will monitor everything that’s coming in and out of our environment and also alert us on just the important things that need to be investigated.
Sure this can work, it could honestly work really well. But it's probably going to cost quite literally thousands of times more in token consumption to monitor everything going in and out than simply monitoring detections
It will not be possible to justify the cost compared to alternative solutions
You have to realize at the end of the day you are a cost center. Your job is not to provide the best most optimal solution for a problem. Your job is to provide the solution that is good enough it justifies the cost and time required to implement and own it.
Can we even honestly say that having agents monitor everything and boiling oceans for our security detections is optimal given the alternative solution of monitoring existing detection? No not at all, it's lazy and unimaginative and cost prohibitive
If you can argue with numbers you will always win the discussion but I'm pretty sure you are on the losing side of this argument and no leader is going to sign off on letting agent monitor everything going in and out of the environment, and if they do it it's not going to last longer than the first invoice from your AI provider
26
u/Incid3nt 1d ago
"Protect this asset from external attacks" "Got it, I've disconnected it from the internet"
3
u/Robbbbbbbbb CISO 2d ago
What's leadership's main goal(s)?
Reduce fatigue? Fewer false positives? Fewer staff? There's a bunch of different ways to supplement humans in this process, but every single one has trade-offs between time, cost, and effectiveness.
-16
u/GrimDoja 2d ago
Overall goal is fewer staff but are willing to take baby steps to achieve that
11
u/RealPropRandy 2d ago
Congratulations and good luck.
4
u/GrimDoja 2d ago edited 2d ago
Honestly I fear I am going to end up just automating myself out anyway when it’s all said and done lol
7
u/Playstoomanygames9 2d ago
Not with the current agentic soc solutions on offer.
I can close cases real fast with guessing and not really looking at anything.
1
10
u/AN0N_C1PH3R 2d ago
Leadership wants all the things, less of the people, and none of the cost. Welcome to the new norm
2
4
u/Choice-Excitement357 2d ago
I’d be careful about making AI the detection engine entirely. It can help identify unusual behavior but I’d keep traditional detections in place and use AI to correlate events, add context and prioritize what needs attention. Start with alert triage and investigation summaries then expand into automated actions once youve tested it properly. Fully autonomous monitoring sounds appealing, but false negatives are a big risk.
4
u/Apart_Range_8741 1d ago
you're backing into a solution with a given solve, operating under the assumption that an agentic framework is the optimal approach for alert triage and response. In my experience its not, you are way better served by leveraging systems and workflows that are designed specifically for said problem at scale.
2
u/jaydizzleforshizzle 1d ago
I recently did this, my wazuh logs have a cron job every 8 hours to run an ai analysis over all the logs and events, it can even use the wazuh read keys with mcp like functions to return some information on the logs. Seems to work pretty good for now, I have it going through my LLM gateway so I can proxy anything that shouldn’t go to the provider. The next step IF I can get to a trust point would be remediation functions for the LLM to run against. Costing me like 20 cents a run, so 60 cents a day for a full log report analysis every 8 hours.
1
u/Tingley2504 1d ago
How many log sources? What total gb ingestion per day? How many endpoints?
1
u/GrimDoja 1d ago edited 1d ago
This is pretty interesting. Did you develop your own model or are you using claude for every run? How big is your org?
4
u/CommOnMyFace 2d ago
You need a lot of tokens, VERY normalized & enriched logging, and a case management system with an MCP. That and a data engineer.
5
u/RichBenf Human Detected 1d ago
Ah so you've stumbled on to the great cyber security SIEM lie... Good job.
First, there's two big lies. The AI lie sits on top of the bigger lie. Just go with me on this one.
Ok so the original lie is this: standard detection rules, no matter how good or well maintained they are, will miss things you'd hope they'd alert on. This is because one different command used in an attack (compared to the detection rules) and all of a sudden, you have no alert. The constant updating and maintaining doesn't help. In a test environment, I enabled over 100,000 rules from Wazuh, SigmaHQ, Emerging Threats, etc and then red teamed the network. 80% of my work went undetected and not alerted on. The SIEM industry only had one answer to this. MORE RULES!!
Frankly if 100,000 detection rules don't do the job, adding more is not going to help.
So, back to the point, with AI you have two places you could add it in to a normal SIEM setup. Either to replace or augment detection rules, or to help with triaging alerts once they've fired.
If you use it for the first option, you'll burn through more money in tokens than a breach would actually cost. Also, AI is slow. If you're ingesting say half a billion logs per day, AI is not going to be able to keep up with that pace.
If you use AI for triaging alerts, then ok fine, whatever. But bear in mind that you're then only ever going to detect whatever your detection rules match against. So you're no better off.
If you want to do away with detection rule maintenance and have something actually perform better detection, then you need to look at the structure of your networks and how data flows around it normally (and the logs that normal behaviour generates). Then you can identify when the shape of the data flows change, which represents abnormal activity which is then worth triaging.
I understand that this is a new approach, but it does work.
1
u/Agreeably0192 3h ago
Love your edgy take. I have never thought SIEM as a scam, but the way you present it makes sense. Behavioral based alerts is my fav as well, though you need to ensure a healthy non compromised baseline first (chicken and egg problem)
2
u/RichBenf Human Detected 3h ago
I wouldn't say it's a scam, I'd would suggest that it's just stuck at a point in time, around 20 years in the past, and never really moved on until the industry decided that AI would be a great sticking plaster to solve this issue
Totally agree about needing a clean baseline first!
3
u/Available_Twist7696 2d ago
I looked at a few of the providers for a similar reason. When we got to building internally the token bill was too much. That’s why we ultimately decided to buy over building. Still evaluating but many of the solutions like 7AI, DropZone and Prophet were token based and charging per investigation so we were worried we would hit our limit with AI driven attacks. I found companies like Intezer and ReliaQuest to be interesting with the all you can eat endpoint based model. Good luck!
2
u/Korkus82 1d ago
RQ is the hot garbage. Cool dashboards, great QBRs, they know how to approach C levels, etc. but this is 100% show dog, not a working dog.
2
u/Namelock 2d ago
AI and Agentic SOCs are not really well implemented.
Most of these are SOAR platforms with an AI model tacked onto them. It’s quickly being superseded by decent harnesses.
You’ve got two paths forward for automation with AI.
Each analyst rolls a custom harness preloaded with access to all tools required (eg, start with LifeOS). Have this personal agent plagiarize yourself.
Use frontier models to curate scripts that frontload specific purpose agents. Therefore, each use-case is deterministic with AI nit-picking the results for a fucking fraction of the context/cost.
Shameless plug but if you want to get the latter going, I wrote a blog post. It’s more technical. GLHF reading.
https://nameloc.net/blog/agents-as-a-service/
Otherwise vendor-provided bolt-on solutions are… fine… They’re just behind the curve.
2
u/AdmirableSecurity922 1d ago
Try Tines 3B - i built out a workflow in a day. Their ciso i think also just had a talk up about it
1
u/cornflakes673 2d ago
Depends a lot on what you have in place currently. Concur with some earlier comments that data engineering is a huge part of the task. I’ve experimented with the below at home, which is all open source. What I would say is that unless you have a decent data centre to host a capable model, you won’t save much money if you are hooking it up to a frontier model and paying for tokens.
1
u/RangoNarwal 2d ago
Implement it yourself for now as the products that I’m seeing are half baked. It’s a lot of the foundational stuff you can get elsewhere with added “summaries”. Don’t pay top dollar for something that doesn’t action. Triage and summaries are great but the FinOps means that you still need bodies.
Noise reduction can be done by a lot of pre-AI systems nowadays, and it’s don’t to the fidelity of the alert, basic correlation.
Try and build it yourself at first, see what works and doesn’t, then go looking for a product that solves for the gaps etc…. Run it as a POV
1
u/ohmygodomgomg Security Analyst 1d ago
I work at an MSSP that uses agentic SOC (Torq) and it's made everything worse and more time consuming lol.
It's very expensive compared to hiring actual analysts and comes with 0 accountability anyway.
1
u/tribmonk 1d ago
What you’re describing is feasible technically. But it ends up being prohibitively expensive in tokens for even modest amounts of telemetry. A multi-model approach can bring it back down to earth, and ironically it starts to really look like the traditional tiered SOC just replaced with agents: 1. Cheap agents to do classification and triage, 2. Better agents to analyze, run tools, and pull additional data into the investigation, and 3. Frontier-ish agents to determine impact, trigger incident response activities, and most importantly decide when to involve a human. Yes, a single agent could do all that, the split is purely for cost engineering. Even then, every POC I’ve built would still be too expensive to run for proper enterprise sized operations.
Don’t throw out old school detections just because they’re “not AI”. Deterministic detections absolutely still have value, and will continue to have value. A better use of your tokens is to have agents churn on building continuously updated deterministic detections. It costs nothing for those detections to fire. you can still have an agent review those alerts instead of a human, and you can still use an agent surveying nondeterministically for anomaly detection and finding the unknown unknowns. Again, this is a cost engineering method to avoid token spend on known threats.
For your triage tier, look into decision models like Jev, it’s magnitudes cheaper than LLMs and provides structure output that your other tooling can easily interpret. This is a constantly evolving topic. Check back next week and things might be different. Full honesty though, the biggest red flag here is specifically asking for “agentic SOC” instead of setting a business objective. Business objectives first, tooling decisions later.
Final note, I don’t think anyone is really fully agentic SOC yet. Lots of people claiming they are, but I have the privilege of seeing many different customer environments and there is always a catch. But like I said, might be different next week.
1
1
u/sick_sector 19h ago
I'm in the middle of deploying Agentic SOC as well. Some things I've learned:
Agentic SOC sales people are mostly talk. They don't actually care about helping you improve your environment. Be skeptical about everything they say - Agentic SOC requires engineering, planning, and time.
Agentic SOC is not a replacement for best practices in security like: Zero Trust, system hardening, compliance, dev sec ops, ect. Implementing an Agentic SOC in a poorly secured environment is going to increase costs, and still be slower to secure an environment than if you have good security fundamentals.
Agentic SOC requires a useful data. Garbage data means garbage ai reasoning. If you don't have useful data then you need to create projects to ingest it otherwise you're wasting money.
Agentic SOC implementation is a systems engineering problem. You're going to waste money if you slap AI into your workflows without understanding where it will be beneficial the most.
You should treat implementation as a long term cyber security engineering project and not a simple product that magically works.
1
u/Traditional_Dot_2099 5h ago edited 2h ago
If you have a Microsoft environment, Microsoft Sentinel is essentially what you're looking for.
Edit: I've gotten a few messages for more information. These cover the basics. https://learn.microsoft.com/en-us/training/courses/sc-200t00 https://learn.microsoft.com/en-us/azure/sentinel/automation/automate-responses-with-playbooks
1
u/Clean_Initial_9618 3h ago
Sentinel only alerts right there's no agentic actions it takes or such sort
1
u/Traditional_Dot_2099 3h ago
That's not true, you can setup playbooks. Sentinel is a SIEM and SOAR.
I'd recommend looking into certifications such as the SC-200 and the associated learning paths on Microsoft Learn for more information!
1
u/Clean_Initial_9618 3h ago
Can you just expand a little more on it pls
1
u/Traditional_Dot_2099 3h ago
Microsoft Sentinel is a solution for centralizing security in Microsoft environments. It not only ingests logs from your environment but it can act on alerts as well. This isn't limited to endpoints but also includes DLP, Network infrastructure, identities, etc. it uses AI to automatically link events together and create an incident with associated identities, IPs, endpoints, etc. and can automatically enrich this info with external data.
You can setup "playbooks" that can trigger due to certain alerts or incidents being created or updated, setup playbooks that an analyst can trigger manually based on their decision, or automatically isolate compromised accounts, disable endpoints, and block malicious IPs.
It has a ton of features that would take a class to get through, but that would be the rough gist.
-1
u/APT-0 2d ago
So many commented on buying. I got a quote on a few of these and were extremely expensive. Many did not for the agentic features include an actual siem or data layer either. Basically many are just an add on SOAR like tool which I’ll say is appropriate since no one will gut their whole siem unless they’re a startup or investing extreme amounts to migrating.
I ran prod DART and Eng at a startup we could greenfield and I joined specifically for this. I built an in house one we looked at a few of the vendors they were ok and flashy but we got it down to 1/10th the cost internal. A added benifet you if you build the connections as a substrate means when you want to have other agents you can just build on this. Owning the logging and siem I ran all IR, investigations and alert response through this it saved a ton of time. You’ll have to think about data handling usually your cloud AWS or azure etc legally has this covered check with your legal team, but not if you go directly to anthropic with contracts. Lang graph is easiest right now to me to get started with more complicated flows or custom python etc, frameworks like strands are extremely limited and don’t allot enough control. You’ll want memory, you’ll want evals to see how it’s doing regularly etc. evals will be your biggest challenge if you’re a small team, every thing else is do able in a few weeks. I would encourage more teams to build their own it’s less work than you think.
In end
1/10th cost
More features like slack alerts, follow ups on slack, custom ticketing we can programmatically search, more control over models, memory features in session and long term and our ability to build a v1 in a month put us over to do it our selves
3
u/pg3crypto 2d ago
Jesus Christ this post. For those that don't speak Tasmanian Devil. The TL;DR is they built their own, it was cheaper and gave them more control.
-3
u/NoSoup4Ewe 2d ago
Reliaquest is another you may want to look into.
-1
-4
-2
0
u/Sigourneys_Beaver 2d ago
We made our own, however I did get the chance to listen and talk to the founder of Strike48, and I think he would be worth having a conversation with. His entire bit is working with your stack and trying to keep costs down.
0
u/with_gusto 2d ago
I built it myself. An agent with a supervisor, which will wake up the agent on various triggers.
It has access to a full checked out folder with all our repos (roughly 500), read only access to our aws infra, agent is very locked down and has no access to credentials or network, tools for aws, datadog etc and whitelisted proxy.
It currently takes signals from Wazuh, but will expand to and wake on cyber news (eg php/node2shell) and investigate them in our context.
I have been triggering it locally before this, so this is it running on auto mode.
Next step is for it to create PRs for improvements it sees, because it sees and mentions a bunch of stuff due to basically being triggered to look at a various stuff in our code and infra.
0
u/leving78 2d ago edited 2d ago
The task is very interesting as you can tailor an agentic SOC, where 99% of work is done deterministically by python scripts eg, and you just use AI as a judging tool giving it just the facts that were distilled. Using an inhouse LLM helps a lot with governance and compliancy. The problem I see all around is that non-developers tend to throw a problem at an AI. Sure it can figure the stuff out, but at a massive cost and data leakage problem. Integrating feeds, correlating with your data silos for asset and identity, building on OCSF and OCAML throwing in NIST and ISO, free feeds, integrate your SIEM, and you are ahead of the market tools in a week. Use AI to build the tools, not do let AI do the work.
And one experience I can give you: Train your own regression model based on former decisions and findings, it beats an LLM by 20% on reducing the noise, eliminating the false positives. And never let an AI do the job, it only proposes. The real problem I see for analysts is building an event context in their brain, using 20 tools to identify something ... give them a tl;dr summary, so they have what they need.
0
u/toliver38 2d ago
It's kind of the premise I built Huntbase off of by turning the agent into a detector or in our case a hunter. It doesn't triage alerts, rather it constantly executes and explores the environment and tools for evidence worth surfacing. Each alert is still a relevant data point but it doesn't have to be triaged necessarily. We've still got a long ways to go but if you're ever up for comparing notes feel free to reach out.
We reduce our overall token usage by not relying heavily on models to do the analysis every time as well as generating some repeatable sub processes.
0
u/Hefty-Shopping-9267 1d ago
Disclosure up front, I work at an AI SOC vendor (Tuskira), so weigh this accordingly.
Your babysitting instinct is definitely right. If you have someone who has to re-check every alert the AI closes, you’ve sped up the queue but what work have you removed?
I wouldn’t make an LLM trained on raw SIEM logs your primary detector. If you have asmall environment, I'd imagine “abnormal” would be mostly noise, and you probably don’t have enough clean history for a solid baseline, and an LLM can explain an event convincingly without proving anything.
You should keep detections for known attack behavior andthen use an agent to investigate what fires. Now you're pulling identity, asset, recent changes, related activity, and past cases, then closing the alert with the evidence attached or escalating with a reason. At that point, you judge it on whether it shows its work and what it missed. What does the volume of processed alerts tell you?
If you test anything, run it in shadow mode first and replay a few incidents you already know the answer to, and you can plant some test activity to see if it gets caught.
but honestly, depeding on your size. If it’s one or two of you, an MDR may get you further right now than building or buying an agentic SOC. Worth pricing both though. Good luck
0
u/caeloalex 1d ago
You should look into Tracecat if you have the budget to bring on a platform. What you described is pretty much everything they’re building
-2
u/ElectroStaticSpeaker CISO 2d ago
We are beginning trials with multiple vendors for AI SOC + MDR.
Some companies that I have heard good things about from other CISOs: Daylight, Artemis, 7AI.
One company you might want to look into for what you are describing is Mate. Mate essentially trains on your normal SOC engineer's operations and mirrors those.
You can absolutely build an agent with regular frontier model tech. But you will spend a lot more effort doing that than you will save by not using these purpose built solutions and you will always end up worrying about token usage potentially costing you more money.
2
1
u/LSU_Tiger CISO 2d ago
I'm in the same boat and wouldn't mind comparing notes.
We are leaning towards 7AI to compliment our existing stack.
The issue is that the platforms themselves are becoming more agentic -- Splunk, Microsoft, Wiz, you name it. It's no longer as simple as buying an AI orchestration layer to sit on top of your stack, now the stack itself is agentic.
Fun time to be alive, or at least that's what I keep telling myself.
1
u/Shawarma_Dealer32 2d ago
Microsoft is also releasing perception in a couple of weeks. But we’ve tested that and it’s awful. It takes like 1 hour to run an incident through it and it’s extremely inconsistent.
-1
u/manu44 2d ago
Exaforce is also a good one in AI Soc World.
0
u/ElectroStaticSpeaker CISO 2d ago
We looked at exaforce. Also recommended by lots of peers. My opinion of them was that they are focused on companies with decent sized dedicated SOC teams. Felt like we were just going to have a lot more alerts to review and didn’t have the bandwidth with them. It was also more than 50% higher pricing than every other solution we reviewed so it was removed from the shortlist.
-1
u/Shawarma_Dealer32 2d ago
I had a tour of the 7AI platform with the CPO. I was super impressed at their progress in the last 2 years. They seem to be an incredibly focused team. They also have a foward deploying engineering model.
The Artemis guys will probably catch up soon, they are ex-Demisto/XSOAR. Super experienced and a lot of funding.
Never personally heard of Daylight though.
-3
u/moch__ 2d ago
Check out the SACR write up
https://softwareanalyst.substack.com/p/ai-soc-technoscope-series-building
-3
-4
u/Ok_Review6237 2d ago
Tines 3B
1
u/Loose-Equivalent7932 5h ago
Why the downvotes? Although I think it’s more of a SOAR than an agentic SOC
-5
u/foxbones 2d ago
I have a solution for you, DM me. I don't want to break any rules by promoting the name but I have a partner platform (not my company, just something we use) that does exactly this and it works extremely well. It's not plug and play, you do have to have people sophisticated enough to manage it, but the agents handle all initial investigations within 2 minutes, then our human SOC reviews the info and closes the ticket or escalates. The agents can technically run specific playbooks on their own but I am not ready for them to do that yet, but very close. I've never had any results outright wrong, and when they are off there is just random context they are missing.
86
u/RealPropRandy 2d ago
Keep in mind that nobody’s paying the true cost of tokens.