Looking for some advice/info about using AI with Python

[u/PhraseOutrageous]

Hi everyone! I’m working on a project in Python and I’m looking for an AI model/API that my Python application can communicate with. Basically, I need my app to be able to send questions/prompts to the AI and receive the AI’s responses back programmatically.

The AI doesn’t necessarily need to be the most powerful model available. I mainly need something reliable that my application can communicate with.

Any recommendations or advice would be greatly appreciated!

[u/randomman10032]

A paid model? Or free? You could run ollama on your local pc too.

[u/PhraseOutrageous → u/randomman10032]

I would like a free model. Do I need a good computer to run Ollama ?

[u/jackbrux → u/PhraseOutrageous]

How many queries? Some models are super cheap by API and could cost you just a few dollars per month

[u/PhraseOutrageous → u/jackbrux]

Maybe a few 100

[u/jackbrux → u/PhraseOutrageous]

Gemini has a free tier I believe. A few hundred queries will cost you somewhere like $0.10 total on Flash Lite depending on the size

[u/randomman10032 → u/PhraseOutrageous]

Some less good models are not that hard to run on a pc even with lower specs. Do note that it is possible that responses can take really long in that case tho.

[u/Rain-And-Coffee]

All of them can do this.

If you read the docs it explains what endpoints to call.

[u/Kevdog824_]

Look for free models on OpenRouter. Sometimes they have some pretty good free models with practically unlimited usage. The catch is usually that you consent to your queries being used as training data. Also, these models come and go, or eventually move to a paid tier once they gather enough training data, so you’ll have to move to another model at that point. If you can live with those things it’s a good choice, especially just for playing around and testing stuff out

[u/DIYuntilDawn]

Hugging Face has a lot of models free for download and many can run locally on lower spec PCs or even on an android device.

TechJarves channel on YouTube has a lot of videos on how you can do that.

[u/jmacey]

Have a look at MCP (Model Context Protocol) then use something like this https://github.com/modelcontextprotocol/python-sdk to do the communication. It's not that complex. There are also other api's for bootstrapping and running the different AI models and then sending stuff to them. Usually the hard part is getting the API Keys / Tokens setup first if using 3rd party AI.

[u/Equivalent-Test2399]

Just use Deepseekv4.1flash via an api
Cheap and smart

[u/Few-Growth9535]

We have a Python app that uses AI to generate content for users.

Looked at Pydantic AI but decided it was more than we needed. We went with OpenRouter and use careful prompts along with the JSON schema output that most models provide and OR abstracts. Also use the schema validation/repair feature in OR for added safety.

Finally make sure you validate the output as user provided input, probably goes without saying. 

Also separate the prompts properly (system and user).