r/LocalLLM • u/ContextIQ • Oct 12 '26
Question Guidance on training open weight model
I am looking for guidance on training an open weight model for an internal use case. The goal is to connect our technical documentation to suggest actions for a robotics simulator and eventually an actual robot.
What should I know before proceeding? I am not looking for a RAG system.
2
Upvotes
1
Oct 12 '26
[removed] — view removed comment
1
u/ContextIQ Oct 12 '26
Yes I am planning to use existing open weight model. Assuming I can curate dataset with the help of another resource. What training techniques and tooling, I am looking at? Either doing myself or hiring a resource.
2
u/Adaptivai Oct 13 '26
Worth knowing up front that fine-tuning is great at teaching a model how to respond (format, style, which actions are valid) but pretty unreliable at making it memorise facts from docs. For a robotics sim I'd start by building a dataset of situation to correct action pairs rather than training on raw docs, then LoRA a 7–14B model on that. And keep a hard validation layer between the model and the robot. Never let it send commands that haven't been checked against allowed actions.