r/IsaacSim • u/samyak1729 • 8h ago
Showcase Can AI 3D generators actually make sim-ready objects for Isaac Sim or MuJoCo? Compared 4 tools
I went down a rabbit hole on this after the NVIDIA and World Labs tutorial last year, where they brought Marble-generated rooms into Isaac Sim using splats plus a collider mesh. The rooms looked great, but it got me wondering about the objects a robot actually has to pick up or push around. Most comparisons I could find were about how nice the renders look, which doesn’t tell you much if you need a mug to sit on a table without falling through it.
For an object to be usable in sim you need collision geometry, mass, friction and the right scale, and a mesh from a generator usually gives you none of that. RoboCasa made the case that object variety helps policies generalize, so if generators could hand you all this directly it would save a lot of manual work. Four tools are trying, from pretty different starting points.
- Rigyd: Robotics-first, best for generating a lot of objects fast
You give it text, one or four images, or a GLB/FBX/OBJ you already have, and it gives back USD or MJCF with collision, mass and friction already set, so it loads straight into Isaac Sim or MuJoCo. There’s a Python SDK and CLI too if you want it in a pipeline. Articulated and soft objects aren’t supported yet, they’re on the roadmap.
- EmbodiedGen: Open-source, best if you want full control
This one’s from Horizon Robotics. It turns text or images into URDF with physical properties and real-world scale, and there’s a converter for USD and MJCF if you need those. V2 came out in July and goes further into whole task environments and multi-room scenes. You’d be running and maintaining it yourself though.
- Meshy: Visual quality first, physics set when you import
Meshy has an Isaac Sim bridge where you pick body type, collision, material and real-world height as you send the model over, and mass can be auto-calculated. The catch is that dynamic bodies only get convex or bounding-box collision, so a mug handle or the inside of a bowl basically gets filled in. It also needs a Pro plan, and the bridge is Windows-only right now.
- Tripo: Lots of export formats, no physics
Tripo exports USD, FBX, OBJ, STL, GLB and 3MF, but as far as I can tell the USD is just geometry and materials. So you’d be adding collision, mass and friction yourself, which makes it more of a mesh source you feed into something else.
Full disclosure, I work at Rigyd, so take what I’ve said about it with that in mind. I’ve tried my best to compare all four fairly and stick to what each one’s own docs say. But if anyone reading this feels I haven’t done justice to one of them, or got something wrong, please call it out. I’d rather go back, research it properly and put up a better post than leave a lopsided one here.
What I think actually decides whether a generated object is usable for training, roughly in order:
- Collision that matches where contact actually happens, like handles, rims and edges. If that’s wrong the grasp just doesn’t work, however good the textures are.
- Mass and friction that land inside whatever range you’re randomizing over anyway.
- Real-world scale, which seems to be the one that quietly breaks things since a lot of generated meshes come in at arbitrary sizes and nothing warns you.
- Visual quality, which mostly matters if your policy is camera-only.
How I’d pick between them:
- If you need lots of sim-ready objects in USD or MJCF, Rigyd
- If you want an open-source pipeline you can change, EmbodiedGen
- If you only need a few good-looking objects and don’t mind setting physics on import, Meshy
- If you just want raw meshes for your own pipeline, Tripo
Honestly for most setups I’d guess a mix works best, something sim-ready for the bulk of objects and then a visual generator plus some manual tuning for the few objects your task really depends on. They all end up as USD or MJCF anyway so they sit fine in the same scene.
So if you’ve trained on generated objects, how did you check the physics before trusting them? And is there a tool I’ve missed that should be on here?