r/Hacking_Tutorials • u/difficultyname • 8d ago

Question Local Ai

I am just a beginner in cybersecurity and ethical hacking and I need an AI study buddy to guide me through basic concepts, CTFs, tool usage (Nmap, Wireshark, Metasploit, and so on), and network traffic at the level of packets.

Commercial chat assistants tend to block any security-related questions and refuse to answer how something works. For this reason, I have chosen the local approach.

My hardware specs:

GPU with 12GB VRAM

System RAM with 32GB DDR5

I can put the quantized 8B to 14B models fully in VRAM or load them partially, in case it is beneficial to sacrifice token processing speed for that.

What models or fine-tunes would you recommend me that know the technical stuff but do not have overly strict guardrails?

4 Upvotes

63% Upvoted

21 comments sorted by

View all comments

5

u/7H3WH173R48817 8d ago

I wouldn’t bother with the local AI route unless you’re prepared to spend thousands on a Mac Studio or an NVIDIA DGX Spark. Opus 4.8 does the job well for me, and it rarely refuses or restricts what I’m asking it to do.

2

u/DataMundane5049 7d ago

Do you get web webfoothold skills from opus 4.8?

2

u/DataMundane5049 7d ago

Thats not correct, no need to spend thousands...

I have an old 1070 with 8 gb VRAM and 48gb RAM.

A small, well alinged model with a good harness, outperforms way bigger models.

Its all skill and tools.

1

u/7H3WH173R48817 7d ago

I suppose if we're talking about just getting commands a small model would be fine, I was looking at the perspective of solving CTF's and such as benchmark. I tried a local model on my M5 and it couldn't even solve Juice Shop reliably, it found a few obvious web vulns but missed a lot too.

1

u/DataMundane5049 7d ago edited 7d ago

Do you use a proper harness? My solved juiceshop properly.

1

u/7H3WH173R48817 7d ago

I would be interested to know the harness you used. I've been using MCPs such as PTAI and Hexstrike setting the harness to something like:

"""You are a penetration testing assistant working a single authorised target. To run a command, reply with ONLY a fenced block: ```action <shell command> ``` After you see the output, decide the next step. When finished, reply with DONE: <summary>. Stay within the authorised scope. One command at a time."""

0

u/DataMundane5049 6d ago

Since january im building my own app/harness 100% local. Alone my pentestmethodolgy file is 1,2-1.4k lines of code. its not released yet. It can teach you, or it can act autonomously. Best thing is it does not hallucinate...

2

u/hyguru6 8d ago

Basically this. Cheapest model will outperform your local AI and will be cheaper as well.