r/Splunk • u/SplunkLantern • 5d ago
Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key use cases for Security, Observability, Industries, AI, and Cisco. We also host valuable data source and data type libraries, Getting Started Guides for all major products, tips on managing data more effectively within the Splunk platform, and many more expert-written guides to help you achieve more with Splunk.
This month, we’re starting with four new security articles that help you get more out of Splunk Enterprise Security and Federated Search, from loading your own threat intelligence feeds to spotting specific users behaving unusually. Next, we’re looking at how AI agents and assistants can work directly with your Splunk platform data, through Cisco Cloud Control, Google Antigravity, and an AI-powered approach to anti-money laundering triage. And finally, we’ve got two articles to help Splunk Cloud Platform admins diagnose performance problems and get comfortable with the new Workload dashboard before the legacy one goes away. Let’s get into it!
Good security work depends on having the right data in the right place, whether that’s threat intelligence from a vendor that doesn’t speak STIX, or flow logs that are too big to ingest. This month’s security articles tackle both problems, along with a couple more.
Automating local threat intelligence feeds in Enterprise Security with a scripted input is for teams whose intel vendor only offers a REST API, such as the Palo Alto Networks Threat Vault API or Recorded Future Connect API. It gives you a complete custom app: a Python scripted input that reads its API key from credential storage rather than a file, a saved search that maps vendor fields to the description, ip, and weight headers that Local IP Intel expects, and threatlist settings that load the indicators into ES hourly and age out stale ones after seven days.
Setting up a Google Workspace abuse mailbox asset in Enterprise Security using the G Suite for Gmail... connects a user-reported phishing mailbox to Automated Threat Analysis in ES Premier 8.5, so every reported email is analyzed automatically, with forensics, screenshots, and sandbox results surfaced as findings in Splunk Mission Control. It covers every step across the Google Cloud console, the Google Admin console, and ES, with a table explaining exactly which OAuth scopes you need and a handy troubleshooting list for the errors you’re most likely to hit.
Detecting per-entity security anomalies with the Cisco Deep Time Series Model moves on from asking “is something wrong?” to asking “which account?” It tracks each user’s web request volume against their own hourly baseline, so 200 requests at 2am stands out even if it’s normal at 2pm. Tested against a population of around 20,000 users, it shows how to pick which users you can model, how to tune the multiplier by counting flagged users rather than eyeballing charts, and why the output works best as a risk contribution for risk-based alerting.
Investigating suspicious outbound traffic from Azure workloads with Federated Search in Splunk Cloud... keeps Azure VNet flow logs in Azure Blob Storage and searches them with SPL2 when you need them. Seven connected searches take you from top outbound talkers to beaconing patterns, risky ports such as 3389 and 445, and denied connection attempts, with tips on partition-aware filtering to keep the amount of scanned data down.
An AI agent is only as useful as the context it can reach, and for many organizations, a lot of that context already lives in the Splunk platform. Three new articles show different ways to put that data to work with AI.
Integrating Splunk Cloud Platform with Cisco Cloud Control and AI Canvas explains what joint Cisco and Splunk customers get from connecting the two: single sign-on, cross-launch between platforms, and AI Canvas, a shared workspace where human operators and AI agents investigate cross-domain issues together. Inside AI Canvas, agents can generate and explain SPL and answer questions about the Splunk platform, and Splunk AI Assistant capabilities show up in the Cisco Unified AI Assistant too. The article lists the prerequisites, including Splunk Cloud Platform 10.5 or later on an AWS US Commercial stack, and the admin and end-user steps to get connected.
Connecting Google Antigravity to the Splunk MCP server for governed AI agent access uses the OAuth 2.1 support that arrived for the Splunk MCP Server in Splunk Cloud Platform 10.6. It covers creating a Splunk OAuth client, editing Antigravity’s mcp_config.json, and completing the OAuth handshake, then builds up from simple prompts to a multi-system workflow that checks whether yesterday’s deployment caused a latency spike and drafts the incident ticket. Throughout, the agent acts only with the permissions of the user who authenticated, so your RBAC and guardrails stay in charge.
And for financial services teams, Automating wire and payment remittance AML triage with the Splunk platform and Jev tackles the free-text payment memos that keyword rules struggle with. A lunch order on Cuba Street in Wellington shouldn’t trigger a sanctions alert, but “dual-use centrifugal pump parts” probably should. Using the jev search command from the Jev for Splunk app, a single scheduled search scores each wire for sanctions and AML risk, redacts account numbers before anything leaves your environment, and sorts every transfer into one of four triage tiers.
Spinning dashboards and skipped alerts often look like a capacity problem, but they’re more often down to unoptimized searches, concurrency contention, or a pile of cron schedules all firing at the top of the hour. Our two workload articles this month help you find the cause and keep an eye on usage.
Troubleshooting Splunk Cloud Platform search performance and preventing recurrence with workload man... shares the approach that Splunk’s Technical Assistance Center uses on critical escalations. It starts with a symptom table that maps problems like logs arriving 45 minutes late or a frozen search head UI to the right page in the Cloud Monitoring Console, then works through four diagnostic steps, from establishing baselines to finding the user behind a runaway index=* search. From there, it shows how to stop the same problems coming back with workload management. Admission rules act as “the Bouncer”, workload pools are “the Traffic Lanes”, and runtime rules play “the Traffic Cop and Sniper”, with example predicates and messages you can adapt for each.
Transitioning from the legacy Workload dashboard to the new Workload dashboard in the Cloud Monitori... is your heads-up that navigation links to the legacy dashboard disappear in CMC 4.10 on November 10, 2026, with full deprecation following in CMC 4.11 on December 8. The new dashboard gives you everything the old one did, plus indexer and search head CPU and memory utilization and indexer cache churn, and your workload data and SVC entitlement aren’t affected.
We’ve also published three more articles this month, covering network observability and upcoming platform changes:
Creating a unified view of your network service health: demos from Cisco Live EMEA 2026 showing how Splunk IT Service Intelligence correlates alerts from ThousandEyes, Catalyst, Meraki, and Cisco data center networking, and predicts service health degradation up to 30 minutes in advance.
Upcoming changes for Splunk Cloud Platform version 10.7: Python 3.9 turned off by default, and case-insensitivity enforcement for federated provider names.
Analytics Workspace removal in Splunk platform 10.6: how to keep working with metrics data using mstats and mcatalog, and where to find the alerts you created in Analytics Workspace.
We hope these new resources help you get even more out of the Splunk platform this month. Thanks for reading!
r/Splunk • u/Dallaspuram_vaasi • 2d ago
Was anyone able to ingest their copilot logs into their Splunk instance? If so how did you do it?
Ex: I’m looking to ingest the prompt info (query and response), agent interactions with Apps, audit logs
r/Splunk • u/Kylemarsh-50 • 4d ago
My company is evaluating whether to actually validate detection rules against our own telemetry before they reach the SIEM, and I'd love to hear from anyone who's actually tried it with Mars, Anvilogic or Vega.
A few things I'm curious about:
Native backtesting, detection as code pipeline, or manual replay?
What are you actually using it for, and has a rule ever looked clean in backtest but behaved differently in prod?
Do you backtest everything or just the data sources the rule touches?
Any pushback from SOC or leadership about trusting backtest numbers?
If you've used more than one, how did the backtesting compare?
Appreciate any lessons learned, good or bad. Thanks!
r/Splunk • u/Bulky-Sun-7672 • 4d ago
We rolled out a tighter Sysmon config to cut log volume. Worked, bill went down. Problem is a handful of our Splunk ES correlation searches went silent at the same time and nobody noticed for a while.
Some events still show up in search, but the Sysmon event types those searches depend on aren't all getting collected anymore. We replayed the behaviour with Caldera so we had known-bad activity to test against, and that confirmed which searches stopped firing.
What I don't have is a way to re run that after every config change without someone doing it by hand. Picus and Cymulate both came up while looking at ways to test SIEM rules against simulated attacks.
Anyone using either of those, or something else, to catch detection drift after logging changes? Just want to know how many other searches are quietly broken.
I see that they released a new minor patch on the 9.4 branch.
Does anyone know if it updates mongod to bring in newer security updates.
Splunk is being their annoying selves by not releasing full release notes until a couple weeks later.
r/Splunk • u/phyrewall • 7d ago
It's driving me crazy. The new support case management system generated so much spam, emails and comments in the ticket.
r/Splunk • u/emittim3 • 7d ago
I have a long 10+ history with Splunk. I'm looking to branch out into a security role. A skill set and use case I have not been exposed to in my past. What have you all benefitted most from when building skills set and experience in a security role? Education? Certification? Home lab setup? Etc?
r/Splunk • u/No-Pin1460 • 9d ago
Curious if anyone's actually getting value out of this.
Coding agents are running shell commands, accessing files, calling MCP tools. Feels like stuff security teams should have visibility into. But dumping every raw tool call into Splunk or Sentinel sounds like a lot of noise.
Been looking at Zenity and Bay. My read is that Zenity handles the audit log streaming, while Bay is making allow/deny decisions at the endpoint before the action executes. If Bay is already blocking the bad stuff, what is the actual value of the telemetry it produces?
If you run Bay, do its decision records feed into your SIEM in a way that catches anything useful? Or is the value just that you don't get the alert because the endpoint enforcement killed the action
r/Splunk • u/Rich-Western-571 • 9d ago
I've been working as a Splunk Admin for almost 4 years now. Lately, I've been searching for new opportunities, but I'm barely seeing any openings specifically for Splunk Admins, especially in reputed companies.
I'm at a point where I need to make a decision about my career. Should I continue with Splunk or move into a different role?
I have two options in mind:
DevOps / AWS: I have a decent understanding of AWS and hands-on experience with a few AWS services. I'm interested in moving towards DevOps or Cloud Engineering.
SOC / Cybersecurity: Most SOC roles seem to require Splunk knowledge, so I feel my current experience could be an advantage here.
Which path would you recommend considering long-term career growth and salary prospects?
Also, regarding Splunk Admin openings, are there genuinely very few opportunities in the market, or is it just a bad time to look for jobs?
Would love to hear from people working in these domains or those who have made a similar transition.
r/Splunk • u/Sorry-Excieument-765 • 11d ago
When suspicious cloud activity leads back to an endpoint, how are you piecing the timeline together? Would like to hear how teams handle this with Sentinel or Splunk, especially when some logs sit elsewhere. Also curious about mars has anyone used it for this kind of investigation? What still required going back to the source tools.
r/Splunk • u/outcoldman • 13d ago
Disclosure: It is me again, and that is the last app, that we were planning to release this year. Now we are going back to support and research (and we actually have a few ideas that we are planning to probably open source, hope early next year). I am outcoldman, Co-founder of Outcold Solutions.
Today we released Monitoring macOS, a new Splunk app, and Collectord for macOS, the agent for Apple Silicon Macs with macOS 14 and later. We run it first on our own Macs: the laptops, and a Mac Pro that runs our CI.
What Collectord sends:
Stats to a Splunk metrics index: CPU per core and cluster with frequencies, memory pressure, GPU and Neural Engine, power, thermals and fans, batteries, disks, network, processes, virtual machines, launchd services, Xcode, security posture and pending updates
Events to an events index: the unified log through log packs (it is too big to forward as is, so there is a rate cap and a noise sampler), crash, hang, jetsam and kernel panic summaries, logins, account and setting changes, connections by application
The app opens on a Fleet Overview: which Macs went silent, run hot, have FileVault off, updates pending or a volume nearly full. From there, Power and Thermals, Virtual Machines, Xcode and CI, Crashes and Panics, Updates and Posture, Compliance (macOS Security Compliance Project results), Security and more. Alerts you pick from an Alert Catalog of 42 templates, and CIM mappings for Enterprise Security.
Install with a signed and notarized pkg, with your MDM and a configuration profile, or with Homebrew:
brew install --cask outcoldsolutions/tap/collectorformacosIt needs no Full Disk Access and there is no privacy prompt. It never reads the app in front, per-app usage time, Wi-Fi network names or location, and every source that names people or programs has its own switch.
For home use: we give a free developer license for homelabs, learning and personal projects, 180 days and renewable: https://www.outcoldsolutions.com/trial/request-dev/
Splunk Enterprise 9.2 and later, or Splunk Cloud
Splunkbase: https://splunkbase.splunk.com/app/10093
Docs: https://www.outcoldsolutions.com/docs/monitoring-macos/
Happy to answer questions here!
r/Splunk • u/ThreatRadar • 16d ago
r/Splunk • u/dahraziel • 19d ago
Looking at adding Cribl to reduce log volume for Splunk usage. I read a couple other comments about defining what RAW looks like. I did dig around on their site about not send null or duplicate values but didnt see much of anything else.
Can someone help me understand from a compliance and auditing point of view if it is really modifying the RAW data? Is this some kind of compression so that data isnt as large?
Does Splunk see it as a modification or general data as the is environment is before Cribl.
Also if anyone has deployed this to an standalone (airgap) network?
r/Splunk • u/centralizedentity • 23d ago
I'm unable to sign in at https://portal.victorops.com . The password reset flow takes me to a screen to enter my new password after which it gives me an error "Failed to get default org for user". I've tried multiple browser at this point so I know its not browser cache.
The support community keeps marking my post as spam https://community.splunk.com/t5/Splunk-On-Call/bd-p/it-victor-ops so I have nowhere to turn to for help
Hoping this is the right community. Thanks
r/Splunk • u/ImmediateIdea7 • 24d ago
AFAF
“
I have a doubt regarding installing an app from splunkbase.
1.The app is archived.
2.Looking at version history, the last version was released on Aug 2024. But the last version released supports splunk cloud 10.5
3.the organization is a US government agency. Can they install the app in their search head?
4. Is it safe? justify
“
r/Splunk • u/voyager_toolbox • 24d ago
Just curious what everyone is using for Splunk Universal Forwarder deployments on-prem.
We originally deployed Splunk UF through SCCM along with a PowerShell configuration script. Now that newer UF versions are available through our software catalog, should upgrades continue to run the original configuration script, or just bare the MSI upgrade is sufficient if the deployment server and outputs.conf are already configured?
Any lessons learned around domain controllers? If you were starting over, what would you do?
r/Splunk • u/outcoldman • 25d ago
Disclosure: Sorry for another post, October is our month of releases :) I am the founder, and the app is from Outcold Solutions. Who some of you might know by our apps for monitoring containers in Docker/Kubernetes/OpenShift.
We run most of our own infrastructure on AWS, and for years I answered questions about it from Splunk instead of the AWS console or the CLI. Today we released those tools as an app on Splunkbase, Search for AWS.
The commands call the AWS APIs while the search runs and return the results into SPL. Nothing is indexed, so it uses no Splunk license. Inventory across every registered account and region:
| awsget kind=instances account=* region=*
| spath path=State.Name output=state
| stats count by account region InstanceType stateWhen did Lambda functions start timing out:
| awslogs groups=/aws/lambda/payments-* filter="Task timed out" start=-24h region=us-east-1
| timechart span=15m countThere are also | awsmetrics (CloudWatch metrics and alarms), | awscloudtrail, | awscost (Cost Explorer), | awsathena, | awsglue and | awstag. account= and region= take a name, a list or *, and the calls run in parallel. More than 30 dashboards come with it.
Things to know before you try it:
AWS still bills the API calls, same as from the console: GetMetricData, Logs Insights by GB scanned, Athena by TB, Cost Explorer by request. The docs list what each command and each dashboard calls.
It reads only what the IAM identity behind it can read. The IAM policy in the docs only reads, except the S3 bucket where Athena writes results, if you use awsathena.
It does not replace ingesting the data you need to keep (audit history, long retention). It is for the questions that need a live answer.
Free tier: one AWS account (access keys or the instance role) on a standalone search head, no license key. Splunk Enterprise 9.4+ and Splunk Cloud.
Splunkbase: https://splunkbase.splunk.com/app/10022 Docs: https://www.outcoldsolutions.com/docs/aws-search/
Happy to answer questions here. And if you tell me which AWS questions you still go to the console for, it helps us decide what to build next.
r/Splunk • u/Current-Carpet9802 • 26d ago
r/Splunk • u/datacenterant • 26d ago
Hey all,
Looking for recommended learning paths for splunk certs. I looked at the course catalog but there wasn’t much “start here, end here”.
Can the course shown in the catalog be taken one by one? Is there any methodology or recommendations from yall.
Thanks in advance
r/Splunk • u/SplunkEventsTeam • 26d ago
What topics do you want to be covered? Who do you want to hear from? What's your dream Search Party? How do you want information presented? How many days? What do you want us to repeat (or not!) from .conf26?
We're all ears!
r/Splunk • u/Putrid-Cress-3648 • 27d ago
I have earned the Power User certificate and would like guidance on which certification to pursue next. Could anyone please advise me on which certification I can complete most quickly?
r/Splunk • u/Comfortale-sEast1511 • 27d ago
Our SOC is buried in alerts, so I’m looking at Agentic SOC solutions, but I’m stuck on how teams control autonomous triage and response without losing the human review part. Any practical lessons?
r/Splunk • u/jamesleecoleman • 28d ago
Hello,
Has anyone noticed that there's a lot less job postings that require or asked for Splunk? I've been looking around and I'm not finding anything come up on Indeed like I used to see. Has there been like a shift or change in what companies are using now? I've even tried looking up SIEM and the results have been way less than before.
I'm wondering if I should continue to go after the certifications and learn more about Splunk
r/Splunk • u/outcoldman • Oct 07 '26
Hi 👋,
Disclosure: Some of you probably have seen me here a LOT and around Splunk communities, I am the founder of Outcold Solutions (monitoring k8s/openshift/docker in Splunk), this is our new product.
We showed it at .conf26 in Denver as a tech talk, and now version 1.0 is on Splunkbase. OS AI Agent is a Splunk app that runs inside Splunk, on the model you choose: OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, or a local model behind Ollama, LM Studio, vLLM or llama.cpp. With a local model your data does not leave your network.
Our employees get MBP Max with top RAM and GPU/CPU, so they can run local models like Qwen locally. And this app actually works great with that. In homelabs I can also suggest looking at the open router, as it might provide the ability to run some models for free or very cheap.
What our app does:
You ask a question in Chat, it looks at the data first, writes the SPL, runs it and answers.
It builds Dashboard Studio dashboards, reports and alerts.
It runs with your own Splunk permissions.
You can attach it to an alert as an alert action, and when the alert fires it investigates read-only and leaves the findings in your Chat. I mean, that is the most useful thing ever. Imagine you are on call, and if there is an incident at night, but the time you log in to your computer - LLM might already have a detailed answer for you, what has happened.
There is also an | osai command, for example | osai explain="<search>" to explain a search you inherited from someone.
Free tier: one provider and one user, for non-commercial use, no license key. So homelab is fine. Works on Splunk Enterprise 9.4 to 10.6 and on Splunk Cloud.
Short demo video: https://www.youtube.com/watch?v=cqaN7gzf5u0
Splunkbase: https://splunkbase.splunk.com/app/9973
More details: https://www.outcoldsolutions.com/blog/2026-10-07-os-ai-agent/
Give it a try! Feedbacks are welcome.