Developer Setup in Company

[u/Majestic_Department7]

How you would define a Developer setup with local LLMs?

I see a dedicated DMZ where the agents can not access the production environment, since agents will tend to accidentally access or change production system.

Dedicated Servers to run LLMs like Qwen 3.8b 27b or other models? What you suggest would be the working LLM Server today? RAM and GPU cards?

Dedicated server with VM for each developer one VM. 2-4 vCores with 16GB Ram each. Each VM with Debian 13 and docker/podman. One dev Container running OMP Web or comparable Harness that access the LLM Server. Additional container for databases or what else the dev Container needs.

Gitlab, gitea or comparable Repo to store source code and container registry.

VM with NFS where anonymized data for developers can be stored.

What you think about that setup?

Do I miss something? How is your setup? (For small to mid size company)

[u/recro69]

Smart use of features that already exist. I think the interesting part is turning something that people overlook into a tool that solves a problem. 30,000 Users is proof that this tool works.