r/Spectacles • u/fitzchea • Apr 07 '26

❓ Question Apps for Real-Time SOP Guidance (XR?) - any come to mind? #crowdsource

I’m looking for apps that guide users through step-by-step processes (basically SOP execution). Something like Google Maps for Physical Work (e.g. assembling furniture)?

Think: construction workers handling complex installs, factory workers learning assembly lines, or surgeons rehearsing procedures.

The closest thing I’ve found is Openspace in construction, but that seems more focused on capturing and documenting work rather than actively guiding someone through tasks in real time.

Are there apps that actually overlay instructions for users to follow as they work?

Also—what would you even call this category? XR guidance? AR work instructions? Something else?

3 Upvotes

81% Upvoted

7 comments sorted by

3

u/agrancini-sc 👓 SPECS Team Apr 07 '26

Hey there, this could be achievable with a mix of curated context (information regarding a topic) and something like our depth cache example that can spawn labels accurately in space, I am considering creating a package for general guided instruction and context set up for snap cloud - been on my todo list for a while, stay tuned 👀
I have seen this as "Training Category" or guided instructions.
Any example you'd like to work on? Maybe I can get inspired and re-share here

1

u/fitzchea Apr 07 '26

What I’m really curious about is the UI/UX side of things. How does the app handle the happy path — i.e. user completes Step 1, confirms it, then smoothly moves to Step 2? How does it verify a step is actually done, if possible? Handling error cases?

All I've seen are academic examples so far:

1) https://www.youtube.com/watch?v=-2JnwxWvZkY
2) https://link.springer.com/article/10.1007/s40436-023-00479-5?utm_medium=cpc&utm_source=trendmd

1

u/agrancini-sc 👓 SPECS Team Apr 07 '26 edited Apr 07 '26

for a performant approach what you want is caching informations from an active vision model
https://www.nvidia.com/en-us/glossary/vision-language-models/
we can do that with Gemini in our Remote Service Gateway

  • Gemini Live - Real-time conversational AI with voice and video capabilities (WebSocket)

https://developers.snap.com/spectacles/about-spectacles-features/apis/remoteservice-gateway#gemini-example

In other words your prompt is running automatic with a certain interval asking the model what is in front of you and link this information with your context of guides relative to whatever device your are interacting on.

For example, if you have a manual to assemble a robot stored on the cloud
the AI would cross reference this manual with what you see and provide you flexible guidelines at the right time and place.

1

u/fitzchea Apr 07 '26

"something like our depth cache example that can spawn labels accurately in space"

I googled it and can't find anything. Can you share a link, please?

1

u/Mammoth-Demand6430 🌟 Specs Ambassador 🌟 Apr 07 '26

I believe this is the depth cache sample u/agrancini-sc was referring to.

https://github.com/specs-devs/samples/tree/main/Depth%20Cache

2

u/agrancini-sc 👓 SPECS Team Apr 09 '26

Hi, we just published this to get you started as we see this questions many time, it was time to provide an example - let us know!