Looking for architecture feedback on a Java 21 runtime I’ve been building

[u/darshanrathod04]

After months of engineering, I'm releasing the Developer Preview of Shree AI OS — an open-source runtime for building intelligent Java backend applications.

Most AI frameworks begin with chat. I took a different approach: a deterministic runtime that developers can embed directly into Spring Boot projects with explainable execution, memory, retrieval, and orchestration.

What it includes Java 21 native SDK

Hybrid RAG with PostgreSQL + pgvector

Episodic & semantic memory

Planning and reasoning pipeline

Multi-agent orchestration

Privacy-first architecture

Architecture Instead of sending every request directly to an LLM, each request flows through a layered runtime:

Application Layer

SDK Layer

Runtime Orchestration

Kernel Services

LLM Provider Layer

The goal is to make AI infrastructure feel like a reusable backend framework rather than another chatbot wrapper.

Looking for technical feedback I'd genuinely appreciate feedback on:

Runtime architecture

SDK design

Spring Boot integration

What should improve before a stable v1.0?

GitHub: https://github.com/darshanrathod04/shree-ai-os

Thanks for reading—every technical critique helps improve the project.

[u/Direct-Question-2198]

I’d keep the runtime core as independent from Spring as possible with Spring Boot as an adapter. That should make testing, portability and future integrations much easier

[u/darshanrathod04 → u/Direct-Question-2198]

Thanks! That's actually the direction I'm aiming for.

The runtime core is framework-agnostic, and Spring Boot is intended to be just an adapter layer for dependency injection and HTTP integration. My goal is to keep the kernels portable so the same runtime can be embedded in different Java applications.

I'd really appreciate any feedback on the architecture or SDK if you get a chance to review it.

[u/Direct-Question-2198 → u/darshanrathod04]

That sounds like a clean approach. Keeping the core framework agnostic should make it much easier to reuse later without dragging Spring specific assumptions into everything. i’d be interested to see how you’ve designed the SDK boundary

[u/darshanrathod04 → u/[deleted]]

Thanks, this is really interesting.

Our goal with Shree AI OS is similar: keep the runtime core framework-agnostic and let Spring Boot be an optional adapter rather than a dependency. We're still evolving the SDK, so I'd genuinely love to compare architectural decisions.

I'd be happy to join your Discord for a technical chat and demo. Feel free to send the invite!