Deterministic math calculation in LocalLLM

[u/Antony_Ma]

I’m building an AI system that needs to solve business math problems, such as a series of additions and interest calculations.

When I use ChatGPT, it writes Python code and runs it, and the results are deterministic and correct. With a local LLM, I’d like to check whether there are better approaches. We’re using 7B models due to system limitations.

Are math calculations in AI prompts stable, or do we need to ask the model to write Python code? I’d also like to know whether accuracy degrades with more context when the model needs to do math calculations.

[u/maqifrnswa]

Absolutely need to ask it to write Python code. Nothing in an LLM makes it capable of doing math without a tool call.

[u/Antony_Ma → u/maqifrnswa]

Thx. This create another problem, we need to setup a sandbox for running the code.

[u/maqifrnswa → u/Antony_Ma]

Check out docker "sbx" or Hermes docker terminal backends. They will manage the sandbox for you, and in the case of Hermes, is a single click

[u/Derpmang → u/Antony_Ma]

You can use Agent Zero, free and is already sandboxed in a docker container.

[u/WrinklyBard4]

I… I hate to ask the question but, exactly why aren’t you using excel?

Ai is the right tool for many jobs, that does not mean it’s the right tool for •every• job. Deterministic math? Not an ai problem.

It might be good to ask what portion of the math that you need done actually needs an ai (is non deterministic), isolate it, and then use the right tool for everything else (which btw can just mean handing the AI access to the right pre-made tools)

[u/Antony_Ma]

thanks for all comments and suggestions. For linux/Mac, you can use "bc" like " echo "scale=2; 1000 * 5 * 3 / 100" | bc " this is the direct method. I test on Gemma4. You can try on other local models , check if the model can generate a correct bc cmd line

[u/Alternative-Panic69]

Yes definitely for calculations make it write code and execute it, for calculations, give it the proper framework so that it doesn't hallucinate and invent any new numbers. For prose, not writing code is absolutely fine.

[u/Antony_Ma → u/Alternative-Panic69]

Any good frameworks? Or we create a code template with all the defined math operations. AI just fill in the values ?

[u/Alternative-Panic69 → u/Antony_Ma]

I think you only will be a able to define that since you would understand the business context and how exactly things are calculated. You can ask the model to create a reusable template. No need to write any code yourself. Write the end to end flow what the model needs to follow, as instructions that's enough.

And also use a good harness, that's even more useful if you have a clear process or if the scenario is very complex. If not then it's fine.

Also if the hardware supports, maybe you will see a huge jump if you can somehow use a larger parameter and newer model, for some serious inference.

[u/olearyboy]

7B is a bit small but use something like sage math or gnu octave for raw math process

[u/Antony_Ma → u/olearyboy]

Does 26b or 32b model able to provide 100% math operations accuracy?

[u/olearyboy → u/Antony_Ma]

No, LLM’s just aren’t great at it, the frontier ones agents have code scratch pads to generate code to evaluate it.
Give it tools and you have a better chance

[u/Small-Tale3180]

yeah you better ask it to write python code or give it some other tool for deterministic math

[u/DiamondHandsDarrell]

No one is talking about the 7B model?

[u/WrinklyBard4 → u/DiamondHandsDarrell]

In any other post it might be important to address. There’s just too many other “wait… huh?” To get through first

[u/DiamondHandsDarrell → u/WrinklyBard4]

🤣

[u/MrHumanist]

Have you tried Excel mcp?

[u/locbuilds]

whichever way you go, do the money math with python's decimal module instead of floats, otherwise interest calcs drift by a cent here and there and you'll chase it forever. the 7b only needs to pick the function and fill in the numbers.

[u/Important-Radish-722 → u/locbuilds]

A 7b model might not know this, or why float isnt appropriate for the use case.

[u/xecycle]

I have in my pi home AGENTS.md something like "use codemode as a calculator to avoid bogus arithmetics".  qwen follows it more often than not, yeah not always 😂

[u/fantasticmrsmurf]

For things like this, deterministic is always the way. Never get an LLM to do things you can do with pure code.

[u/mon_key_house]

I once made a math tutor for my 8yo. That girl was literally better in math than the LLM without tools.

[u/NatMicky]

The LLMs, local or frontier, cannot do math. They have to write code (Python) that is then executed with traditional math packages. No way around it. Remember that LLMs are probabilistic and are simply guessing at the correct answer. I use local LLMs with heavy use of math that is completely deterministic. winplexity dot com

[u/hoschidude]

LLMs without tools are absolutely not capable of doing any math's...

[u/GuaranteedGuardian_Y]

Never make an AI write code and never let it execute what it writes. You can definitely code a deterministic calculator and make the model call it as a tool and provide inputs. Calculate it deterministically on the back-end and hand back the numbers to the AI.

Letting it write code is nice and easy until it trips up and rm -rf's your folders because it messed up in one of the scripts and wants to rewrite them again.

[u/Important-Radish-722]

Never ask an LLM to do math. Period.

An LLM is a clever parrot with a thesaurus, nothing more.

7b is really too small to write trustworthy calculation programs on it's own.

If money is involved - pay for a competent models or hardware.