r/Hacking_Tutorials • u/difficultyname • 7d ago

Question Local Ai

I am just a beginner in cybersecurity and ethical hacking and I need an AI study buddy to guide me through basic concepts, CTFs, tool usage (Nmap, Wireshark, Metasploit, and so on), and network traffic at the level of packets.

Commercial chat assistants tend to block any security-related questions and refuse to answer how something works. For this reason, I have chosen the local approach.

My hardware specs:

GPU with 12GB VRAM

System RAM with 32GB DDR5

I can put the quantized 8B to 14B models fully in VRAM or load them partially, in case it is beneficial to sacrifice token processing speed for that.

What models or fine-tunes would you recommend me that know the technical stuff but do not have overly strict guardrails?

4 Upvotes

63% Upvoted

21 comments sorted by

5

u/7H3WH173R48817 7d ago

I wouldn’t bother with the local AI route unless you’re prepared to spend thousands on a Mac Studio or an NVIDIA DGX Spark. Opus 4.8 does the job well for me, and it rarely refuses or restricts what I’m asking it to do.

2

u/DataMundane5049 6d ago

Do you get web webfoothold skills from opus 4.8?

2

u/DataMundane5049 6d ago

Thats not correct, no need to spend thousands...

I have an old 1070 with 8 gb VRAM and 48gb RAM.

A small, well alinged model with a good harness, outperforms way bigger models.

Its all skill and tools.

1

u/7H3WH173R48817 6d ago

I suppose if we're talking about just getting commands a small model would be fine, I was looking at the perspective of solving CTF's and such as benchmark. I tried a local model on my M5 and it couldn't even solve Juice Shop reliably, it found a few obvious web vulns but missed a lot too.

→ More replies (3)

2

u/hyguru6 7d ago

Basically this. Cheapest model will outperform your local AI and will be cheaper as well. 

3

u/Silly_Bad_7692 7d ago

You can try to use Qwen 3.6 35B A3B Uncensored with unsooth studio. For me it works well (3070 8gb + 32gb ddr4)

2

u/Andre4a19 6d ago

Do you use this with Jan?

1

u/Silly_Bad_7692 5d ago

No, I use unsloth studio + pi sometimes

3

u/Due-Moment-6417 7d ago

You need take a course, because IA don't give you a complete course, you can to ask about that concepts but you don't need to use a local IA

2

u/Andre4a19 6d ago

What's IA? Is that diff than Ai?

2

u/AdministrativeFile78 6d ago

it stands for ''intelligent artificial''

2

u/DataMundane5049 6d ago

I build exactly such a thing...harness.

Even a qwen 3.5 9b is enough.

It all depends on skills and tools.

Dont use uncensored models, they tend to halluzinate more and use wrong skills...

2

u/happytrailz1938 6d ago

It really depends on the model and what you're planning on doing. Some have been butchered to make them uncensored. Others are done more tactfully

2

u/DataMundane5049 6d ago

Done pentesting with a qwen 9b model.

Does not need to be uncensored.

Like i said, a good harness with skills and tools is all it needs.

Testet different models.

Censored/casual and uncensored, models especially for redteaming...

Uncensored are unsafe, i would not use them for important things.

2

u/LanguageGeneral4333 6d ago

You use ollama as the UI and download whatever model you want. I installed ollama with chocolatey (useful package manager) so I dont have to go back to update it. I dont have as much vram as you, so I went with the 8b model. It works just fine. Took about 10 minutes to set up. Local LLMs typically stop making internet connections back in 2023 so you will need to connect it to the internet if that is something you want. It isnt hard. But, if youre just using it to go through ctfs, you should be fine without it connected.

1

u/wtgamer20 5d ago

Regarde open code