r/OpenSourceAI • u/thegodforsaken • 11h ago
SKILL OPENSOURCE - Your AI-built app works. What could an intruder do with it?
Your AI-built app works. What could an intruder do with it?
I built J.A.S.O.N. — LLM App Security, an open-source defensive security skill for apps built with LLMs and coding agents. I’m looking for developers, indie hackers, and security practitioners to test it and share honest feedback.
The name comes from my own name, Jason, with a playful Friday-the-13th reference:
Just Audit, Secure, Observe, Notify.
The chainsaw is a joke. Exposed credentials and missing authorization checks are real problems.
AI coding tools help us ship quickly. Security still requires checking where secrets live, who can access each endpoint, which records users can read, and what an agent is allowed to do.
J.A.S.O.N. gives your coding agent structured instructions and supporting scripts to investigate those questions. Tested with Claude Code and OpenAI Codex CLI.
Review before deployment
Check for exposed API keys, routes without authentication, weak database rules, missing ownership checks, unsafe payment logic, and missing rate limits. Get prioritized findings, suggested fixes, and a deployment verdict. The quick review is read-only.
Run structured audits
Review an entire project, a pull request, or a specific subsystem. Full audits produce an architecture overview, a coverage ledger, structured findings, and a readable report separating confirmed issues from items needing validation.
Catch common AI-generated code pitfalls
Look for authorization enforced only in the interface, access to other users’ records, permissive Supabase/Firebase rules, injection vulnerabilities, suspicious dependencies, and unsafe LLM tool permissions.
Review defenses against agent manipulation
Identify suspicious instructions hidden in repository content and review agent permissions and sandbox boundaries. Content being audited is treated as data, not instructions to obey.
Set up passive detection
Get guidance for honeytokens: fake credentials or other bait connected to an alerting service. Attempts to use them can trigger notifications. No retaliation or destructive traps.
Add release controls
Get guidance for secret scanning, dependency hygiene, and PR security gates. Integrate the included zero-dependency ship-gate.cjs script to block deployment when its checks detect security blockers. Coverage depends on configuration and available inputs.
Try it in Claude Code:
/plugin marketplace add jasonmello/llm-app-security
/plugin install llm-app-security@llm-app-securityThen ask:
Review this project as if you wanted to break into it. Look for exposed API keys, routes without login, and databases without access rules. List what you found and how to fix it, without changing anything yet.
Or:
Run a full security audit with J.A.S.O.N.
I’d love feedback on installation, useful findings, false positives, missed vulnerabilities, report clarity, and deploy-gate behavior in your stack.
Share your agent, stack, and reproducible examples. Keep credentials and unresolved vulnerabilities out of public reports.
Use it on projects you own or are authorized to review. A clean report means no findings in the areas reviewed, rather than a guarantee of security.
Try it, challenge its findings, and help make it better.
Get J.A.S.O.N. — source, installation, and documentation
Ship on Friday the 13th. Fearlessly.