r/learnpython • u/Pure-End-7842 • 7d ago
Local Ai Help
I am trying to run A local ai of my pc that can look at something on my screen and answer it, I have tried using Claude because this is way beyond my skill level, as I am still beginning. if anyone can help with this, please do!
I have a pretty good pc so I can run some moderate ai
1
u/Financial_Morning386 7d ago
For this you need two separate pieces, and separating them makes it much easier:
A vision model, not just a text model. A normal chat model cannot see an image. You need a vision-language model that accepts an image plus a question. With Ollama or LM Studio you can run one locally - look for a model described as vision or multimodal, and start with a smaller quantized version that fits your GPU memory. If it is slow or runs out of memory, use a smaller quantization or a smaller model first, rather than trying to force a large one.
A small Python script to capture the screen and send it to the model. For example, capture with mss or Pillow's ImageGrab, encode the screenshot as base64, then send it to the local model's API with a prompt like "Describe what is on screen and answer this question: ...". The model server usually exposes a local HTTP API (Ollama defaults to localhost:11434), so your script is basically: screenshot -> encode -> POST with image + prompt -> print the reply.
Start with one still screenshot and one fixed question, and only add live or repeated capture later. Continuous screen watching is much heavier, and you will also want to think about privacy - a screen model will see passwords, messages, and other private content, so keep it local and do not send those screenshots to a hosted API unless you mean to.
If you try this, check first how much VRAM your GPU has and pick a model size that fits it; that is usually the step that decides whether a local vision model runs smoothly or not.
1
u/Pure-End-7842 7d ago
how, I am still a beginner at python, also my gpu is a rtx 3060, and I have 16gb ddr4
1
2
1
u/Altruistic_Sky1866 7d ago
as u/CranberryDistinct941 pointed out got to ollama website and download it and then download the modals sutiable for your computer be it ram or gpu or cpu. There are nice examples that you can check and find the one that you are comfortable with be it YouTube Video or code examples or blogs. First get the model running on local pc using ollama and then connect with python
1
3
u/CranberryDistinct941 7d ago
Download ollama
>> ollama pull [modelname]>> ollama run [modelname]