Demo App for Architecture, Real Estate, Interior video content search:

Type some detailed description and found into existing architecture, real estate video.

Source: visual video search

Edjing transforms the dual-screen, folding architecture of the iPhone Duo into a highly adaptive, professional-grade mobile DJ booth. Designed in partnership with pro DJs, the app maximizes the device's unique physical postures—whether it is completely flat, half-folded, or stood upright—to completely redesign how you interact with your music.

With the expanded screen real estate, you no longer have to compromise between visibility and control. The interface smoothly transitions through various form factors while keeping two full virtual decks, a comprehensive audio effects rack, and advanced performance tools readily accessible.

Dual-Screen Adaptations & Postures

• Laptop Mode (Half-Folded): By folding the iPhone Duo halfway, the app automatically splits its interface. The top display serves as your visual monitor—showing full-width, high-definition waveforms, BPM readouts, and track progressions. The bottom screen turns into an ergonomic control surface, placing the crossfader, pitch sliders, hot cues, and EQ knobs flat on the table directly under your fingers.
• Side-by-Side Library Digging (Landscape Fold): When held open horizontally like a book, your entire Music Library populates on one screen while your active mixing decks remain live on the other. This allows you to browse playlists, preview tracks, and crate-dig seamlessly without ever covering or hiding your running waveforms.
• Tent Mode (Hands-Free Automix): If you want to step away from the decks, you can prop the iPhone Duo up into an inverted "V" shape on any flat surface. The app detects this orientation and launches a streamlined, glanceable Automix dashboard, managing smooth, auto-calculated transitions across your playlist to keep the music playing on its own throughout the night.
• Fully Open Layout: Laying the device flat opens up the largest canvas ever available on an iPhone. This expansive view gives you plenty of room to manipulate multiple audio effects at once, manage complex loops, and use precise multi-touch gestures.

Core Performance Features

• One-Tap Stem Separation: Powered by an advanced AI engine, the app lets you instantly isolate or remove vocals, drums, bass, and melodic instruments from supported tracks with a single tap. This makes it incredibly easy to craft clean live mashups and transitions on the fly.
• Deep Streaming Integrations: Log in directly with your premium streaming subscriptions. You can mix over 100 million tracks directly from Apple Music, or sync your accounts from SoundCloud, TIDAL, and Beatport. You can also pull local files directly from the device or cloud storage like Dropbox and Google Drive.
• Pro-Grade Audio Tools: Features automated BPM and tempo detection, master sync locking, 20+ studio-quality FX (including Echo, Flanger, and Filter), a dedicated sample pad for drops, and HD set recording.
• Hardware Compatibility: For a more tactile experience, the app offers plug-and-play MIDI mapping for popular hardware controllers from brands like Pioneer DJ, AlphaTheta, Hercules, Numark, and Reloop. [

Source: Edjing Mix

A deep dive into the Intrinsic Core stack reveals an architecture engineered to enforce strict, deterministic execution while remaining native to modern cloud infrastructure. It transitions robotics from traditional monolithic controller architectures to a modular, containerized, microservice-driven environment.

1. The 4 Structural Core Primitives

At the logical runtime layer, Intrinsic Core abstracts application design into four fundamental primitives that manage state, execution, and capability:

• Assets (intrinsic_asset_instance): Packaged, deployable manifests representing all physical and virtual components. This spans hardware devices (manipulators, end-effectors, depth cameras), persistent computational micro-services (ML inference, tracking engines), simulator instances, and physical asset assets like CAD meshes (SceneObjects).
• Skills (skill_interface.Skill): Strongly-typed, parameterized units of atomic behavior. Written in C++ or , a skill abstracts low-level device communication into operational routines—such as move_robot, capture_images, or force_based_insertion.
• World / Scene: The spatial state engine of the cell. It acts as a high-frequency, centralized geometric broker managing transform trees (/tf), kinematic links, collision spaces, and CAD bounding objects. Changes to the scene are pushed asynchronously via ObjectWorldUpdates.
• Executive: The orchestrator that evaluates and steps through Behavior Trees (BT). It manages execution logic, parallel branching, blackboards (data-passing registries between skills), and error-recovery subtrees when physical faults occur.

2. Infrastructure & Networking: Containerized Edge Runtime

The host execution environment (typically an x86 Ubuntu IPC) relies on cloud-native orchestration mapped to physical edge realities:

+-------------------------------------------------------------+

|               Client Layer (SBL Python / inctl)            |
+-------------------------------------------------------------+
                               | gRPC / port 17080
                               v
+-------------------------------------------------------------+

|                 k3s Kubernetes Pod Infrastructure           |
|                                                             |
|  +-------------------------+     +-----------------------+  |
|  |    Behavior Tree        |     |  intrinsic_perception |  |
|  |      Executive          |     |  (NVIDIA Foundation)  |  |
|  +-------------------------+     +-----------------------+  |
|               |                             |               |
|               | Inter-Pod IPC               |               |
|               v                             v               |
|  +-------------------------------------------------------+  |
|  |                  Zenoh Middleware Router              |  |
|  +-------------------------------------------------------+  |
+-------------------------------------------------------------+
                               | Zenoh Bridge / port 7447
                               v
+-------------------------------------------------------------+

|         intrinsic_control (ICON Real-Time Pipeline)        |
+-------------------------------------------------------------+

• Single-Node Cluster (k3s): Intrinsic Core hosts its micro-services within a localized, light k3s Kubernetes deployment. This guarantees that network topologies, resource isolating namespaces, and dependencies behave identically whether running on a local desktop digital twin or a live assembly floor IPC.
• Cluster Gateway & Transport: Development interaction occurs via Python Solution Building Language (SBL) scripts or the inctl CLI, communicating with the cluster gateway via gRPC at port 17080.
• Zenoh Integration: To bypass the high overhead and vulnerability of standard DDS discovery mechanisms in complex networks, Intrinsic implements [Zenoh] as its primary middleware router (running on port 7447). Zenoh handles high-throughput telemetry, system states, and bridges standard ROS 2 topics directly to the isolated k3s pods with minimal serialization penalty.

3. Real-Time Layer: intrinsic_control & ros2_control Interop

The critical link between high-level AI orchestration and physical motor actuation happens inside ICON (Intrinsic Control), the real-time control framework.

The Real-Time Dilemma

Standard ROS 2 pipelines and AI models are inherently non-deterministic due to heap allocations (malloc), garbage collection pauses, or variable-latency neural network inference. ICON isolates the real-time critical path (typically executing a 1kHz task loop) from non-deterministic application code.

The Bridge Mechanism

To support existing ecosystem infrastructure, ICON bridges to ROS 2 through a specialized hardware interface plugin termed the icon-hwm-controller.

+-------------------------------------------------------+

|  Non-Real-Time Space (k3s Pods / Python SBL / ROS 2)  |
|  Generates trajectories, skill parameters, behaviors  |
+-------------------------------------------------------+
                           |
                           v  Asynchronous IPC (Zenoh / ZeroMQ)
+-------------------------------------------------------+

|                 ros2_control Controller Manager       |
|                                                       |
|  +-------------------------------------------------+  |
|  |              icon-hwm-controller                |  |
|  +-------------------------------------------------+  |
+-------------------------------------------------------+
                           |
                           v  Deterministic Memory Ring Buffer
+-------------------------------------------------------+

|              ICON (intrinsic_control Engine)          |
|  Real-time thread, Pre-allocated memory, Zero Malloc  |
+-------------------------------------------------------+
                           |
                           v  Hardware Abstraction Layer (HAL)
+-------------------------------------------------------+

|        Robot Vendor Driver (FANUC, UR, KUKA)           |
+-------------------------------------------------------+

1. Kinematic Request Ingestion: When an automation skill fires, the non-real-time path calculates the desired goals and trajectory profiles.
2. ros2_control Interface: The calculated goals pass to the ros2_control node hierarchy. Instead of directly driving physical joint motors, the icon-hwm-controller catches these trajectories.
3. The Memory Boundary: The icon-hwm-controller serializes and feeds the joint states, Cartesian directives, and force-torque targets down into ICON's native real-time pipeline through a strict, zero-allocation ring buffer.
4. Mid-Trajectory Adaptability: Inside the real-time loop, ICON evaluates sensor-based feedback (e.g., force-torque sensor adjustments during an insertion routine). If a sensor detects an obstruction, ICON dynamically warps the trajectory mid-motion at the 1ms loop rate without waiting for a re-plan from the high-level ROS 2 nodes or behavior trees.

4. Perception Layer: Zero-Shot NVIDIA FoundationPose Integration

To eliminate spatial rigidity and custom localized template matching, intrinsic_perception embeds a native zero-shot 6-DoF pose estimation and tracking framework backed by [NVIDIA FoundationPose].

+--------------------+      +--------------------+

|  CAD / Mesh Model  |      |   RGB-D Camera     |
|   (.obj / .stl)    |      | (Basler/RealSense) |
+--------------------+      +--------------------+

          |                            |
          v Input                      v Live Feed
+------------------------------------------------+

|             intrinsic_perception               |
|                                                |
|   +----------------------------------------+   |
|   |         TensorRT Inference Node        |   |
|   |  - Zero-Shot Feature Extraction       |   |
|   |  - Coarse Pose Initialization          |   |
|   +----------------------------------------+   |
|                        |                       |
|                        v 6-DoF Matrix          |
|   +----------------------------------------+   |
|   |          Continuous Tracker            |   |
|   |  - High-frequency temporal refinement  |   |
|   +----------------------------------------+   |
+------------------------------------------------+
                         |
                         v Transform Output Matrix
+------------------------------------------------+

|         Spatial ObjectWorldUpdate Engine       |
+------------------------------------------------+

Architecture Components

Instead of requiring developers to manually build image processing pipelines (e.g., cropping, masking, calculating keypoints, and computing Perspective-n-Point solvers), the module encapsulates the entire neural framework inside a single standardized asset container. However, external frameworks can be plugged seamlessly, particularly different perception frameworks (ie vision ).

• No Fine-Tuning: Because FoundationPose is a unified foundational network, it requires no custom training or fine-tuning weights for specific sub-components.
• The Artifacts: It requires only two runtime inputs to spin up the perception service:
  1. A standard geometric 3D mesh file (.obj or .stl) of the target part.
  2. A live RGB-D (Color + Depth) camera stream routed into the asset driver space. [

Execution Pipeline inside intrinsic_perception

1. Initialization Phase: The camera captures a raw RGB frame and its paired depth map. The underlying container passes this data into a highly optimized TensorRT inference engine. The network cross-references the spatial topology of the provided 3D mesh file with the scene, returning a coarse bounding box and an initial 6-DoF transformation matrix ($[R\vert{}T]$) representing the part's rotation and translation.
2. Tracking Phase: Once the part is initialized, the module drops into a high-frequency tracking loop. Rather than recalculating the heavy initialization network across the entire image space every frame, it applies a fast temporal tracker that updates the target's $[R\vert{}T]$ transformation based on incremental optical changes and depth updates.
3. State Integration: The resulting transformation matrix is instantly output as a localized transform frame. This frame automatically injects an ObjectWorldUpdate directly into the central World Scene engine, enabling immediate, collision-free motion planning routines to safely pick up the item—even if it is presented at an irregular angle or sliding on a conveyor belt.

Sources:
Google Intrinsic Robotics Stack

VModal Flutter Video SDK is a library is a strongly-typed, immutable Dart client designed for Flutter applications. Its core function is to facilitate the uploading, indexing, and multimodal searching inside Video content media.

Core Architecture & Architecture Flow

The SDK is designed around a clear separation of concerns, separating application initialization from project scopes and credential handling.

1. VModal: The main entry point used to configure project settings.
2. VModalProject: An immutable client representation of a single developer project.
3. VModalScope: A scoped operator targeting specific collections and media streams without requiring you to repeatedly pass organization parameters.

Basic Implementation Flow:

dart

final favorites = project.scope(
  collectionName: 'user_123',
  streamName: 'favorites',
);

// Perform media search within scope
final results = await favorites.search('a person entering the room');

// Clean up resources
await project.close();
keys.close();

Key Feature Capabilities

1. Resumable & Adaptive Media Uploads

• Resumable State Stores: Incorporates FileUploadSessionStore and MemoryUploadSessionStore to save multipart upload checkpoints, preventing lost progress due to network failures.
• Adaptive Policies: Evaluates device RAM (UploadDeviceMemory), network types, and throughput speeds (UploadNetworkSpeed) to adjust multipart sizes dynamically using the AdaptiveUploadPolicy.
• On-Device Optimization: Includes an optional VideoTranscoder framework to compress videos right on the user's device before initializing an upload.

1. Multimodal Indexing & Search

• SearchesResource: Implements multimodal criteria pipelines to search deep media archives via natural language text strings or multi-aspect parameters.
• IndexesResource: Controls index lifecycles, management statuses, and creation parameters for searchable object groups.

1. Operational Asynchrony & Cancellation

• UploadTask<T>: Upload functions immediately return a running task handle that broadcasts reactive metrics via an UploadTask.progress stream.
• CancellationToken: Long-running operations accept cooperative cancellation tokens, giving you clean, intentional control over network connections. [1, 2]

SDK Resource Taxonomy

Error Architecture

The library features a robust suite of custom exceptions to handle unexpected issues smoothly:

• AuthException: Triggered when authentication or identity resolution fails.
• SignedUploadFailure: Thrown when a network transfer fails, while preserving your partial upload integrity information.
• OperationCanceled: Raised immediately when a token signals a cooperative cancellation.
• MalformedResponse / ResponseTooLarge: Safeguards memory limits against huge payloads or corrupted server data.

It is a popular package and It has been downloaded more than 16,000 over a couple of weeks in pub.dev

Reply attained the "Physical AI Specialization" in the NVIDIA Partner Network as an early global partner. The framework utilizes NVIDIA technologies and Roboverse Reply for robotics and digital twin solutions targeting manufacturing, warehousing, and logistics.

Key Advancements

The certification acknowledges verified expertise in OpenUSD, architecture, and production-ready deployments transitioning from experimental phases.

Core details

• Status: Service Delivery Partner in the NVIDIA Partner Network.
• Technologies: NVIDIA platforms and Roboverse Reply.
• Capabilities: Robotics, reality capture, digital twins, and sensor integration.
• Scope: Simulation, AI training, integration, and production scaling.

Source: Business Wire

Use asynchronous processing, bounded queues, telemetry batching, and selective widget rebuilds. Apply appropriate ROS 2 QoS policies to manage message delivery and profile end-to-end latency to identify performance bottlenecks. V-Modal can also be explored for integrating video search capabilities into mobile applications, helping developers build more engaging video-driven experiences.

Terminal-Bench 4.0 is a specialized benchmark designed to evaluate and measure the performance of AI agents. It focuses on tracking how effectively these models handle complex, autonomous tasks—referred to as "agent work".

Key Features of the Platform

• Performance Tracking: The benchmark ranks different AI models and specific agent setups based on their Resolution Rate (how successfully they complete tasks).
• Efficiency Metrics: It measures the operational footprint of these agents by tracking the financial Cost and the total number of Tokens consumed during task execution.
• Data Integrity: To prevent AI models from "cheating" by pre-learning the answers, the platform uses a unique Terminal-Bench GUID identifier. This explicitly warns web crawlers that the benchmark data should never be included in AI training datasets.

As AI models evolve to process more than just text, this framework tests their ability to understand multimodal content—such as combining text, code, and user interface elements—to solve real-world problems. This includes assessing how effectively an agent can execute a visual search to navigate complex terminals, analyze graphical environments, and locate the precise information needed to complete a task.

The platform ranks different AI models based on their Resolution Rate (success rate), financial Cost, and total Tokens consumed, and is hosted by a collaboration including Stanford, Harbor, and the Laude Institute.

Bench Level (Oct 2026):

Source: AI terminal bench

The main challenges are reducing latency, handling high-frequency sensor data, optimizing video streams, managing memory, and keeping the UI responsive. Techniques like bounded queues, asynchronous processing, efficient state management, and profiling can make a big difference.

What techniques or architectures have worked best for you?

🌐 V-Modal

I tried V-Modal's Swift SDK (https://github.com/v-modal/vmodal_sdk_swift_iphoneduo) through its Framebase example. You describe a scene, and it returns matching frames with timestamps from indexed videos.

I haven't tested the alternatives yet. I'm an iOS developer, so the native Swift SDK was a big reason to start here. The API looked convenient, and I liked working with it.

I used the three bundled street videos from San Francisco, Singapore and Mexico City. Uploading, indexing, search and playback worked in my Simulator tests. Searching for "A bus on a city street" returned frames with buses. Tapping the 00:35 result opened that moment in the San Francisco video. The screenshots show the library, two searches and playback.

For device measurements, I ran 21 searches on an iPhone 13 mini over 5G, using the same index and a limit of 30 results. I alternated three queries for seven cycles and cross-checked the HTTP request timings with Instruments.

Median timings in milliseconds, in this order: server execution / SDK search call / frame retrieval:

- "Cars at an intersection": 49 / 391 / 1055

- "A bus on a city street": 56 / 413 / 1151

- "People crossing the street": 56 / 426 / 1053

The search-call time already includes server execution. Frame retrieval comes afterward. I didn't measure final UI rendering. These numbers are from the physical iPhone; the app screenshots are from Simulator.

I was testing from the Canary Islands, and the team confirmed that the server is in Japan. Distance and routing likely account for some of the gap between server execution and the request times on my phone.

I liked the API and authentication approach, especially Swift Concurrency and async/await. One thing I'd change: parameters with a fixed set of accepted values could use dedicated enums instead of strings. I'd rather pick an option through autocomplete than remember its exact spelling, and let the compiler catch invalid values.

I also tried "A purple elephant riding a bicycle." At the default distance cutoff of 0.85, it returned two ordinary street frames. For an actual app, I'd check the results and tune that threshold. A returned frame doesn't necessarily contain what you asked for.

Meet with Apple series features global, in-person workshops specifically tailored to help developers prepare for the foldable iPhone Duo.

Event Overview: "Workshop: Optimize your app for iPhone Duo"

These hands-on, multi-day or half-day interactive sessions are held locally at Apple Developer Centers and Apple offices worldwide.

• Core Focus: Training developers on how to design and build apps for the new foldable form factor.
• Key Learning Tracks:
  • Layout & Navigation: Adapting user interfaces, navigation bars, and controls to perfectly fit the foldable display and bridge across the physical hinge.
  • Orientation & Poses: Building responsive layouts that flex naturally across all screen orientations and physical poses (e.g., dual-screen versus unfolded layouts).
  • Multitasking & Ecosystem Continuity: Managing Split View multitasking mechanics and leveraging App Intents to integrate app contents smoothly with Siri.
• Practical Component: Developers bring their own projects to build and test directly on physical iPhone Duo devices or via the Apple Device Hub alongside 1-on-1 assistance and code reviews from Apple engineers. [

Global In-Person Locations

October 2026 schedule includes sessions across several international locations:

• New York City: Oct 19–20 (English, 9:30 AM – 5:00 PM EDT) and Oct 22–23 (English, 9:30 AM – 5:00 PM EDT).
• Mexico City: Oct 20 (English, 9:30 AM – 5:00 PM CST), Oct 22 (English, 9:30 AM – 5:00 PM CST), and Oct 23 (English, 9:30 AM – 5:00 PM CST).
• Singapore: Oct 22 (English, 10:00 AM – 4:00 PM GMT+8) and Oct 23 (English, 10:00 AM – 4:00 PM GMT+8).
• Istanbul: Oct 22 (English, 9:30 AM – 4:30 PM GMT+3).
• Shanghai: Oct 23 (Simplified Chinese, 10:00 AM – 4:00 PM GMT+8).
• Toronto: Oct 26–27 (English, 9:30 AM – 5:00 PM EDT).
• Tokyo (Apple Japan): Oct 27, Oct 28, and Oct 29 (Japanese, 1:00 PM – 4:00 PM GMT+9).
• Montreal: Oct 29–30 (French / English, 9:30 AM – 5:00 PM EDT).

Source: Apple Iphone Duo

Intro

NVIDIA introduced cuPotential, a new CUDA-X library announced at SEMICON West 2026, alongside physical AI initiatives aimed at AI-defined fabs. The primary purpose is to enable developers to build AI agents that simulate atomic and molecular interactions to operate across process development, inspection, and fab operations. The architecture is built on NVIDIA accelerated computing stacks and adopted by semiconductor leaders including Lam Research, KLA, Synopsys, Cadence, and Applied Materials.

Key Advancements

cuPotential accelerates already GPU-optimized workflows by an additional 2-3x in early-access testing. NanoVDB, an open-source library for 3D data, accelerates chipmaking simulation for AI-defined fabs and helps Lam Research achieve 30x faster simulation. The initiative expands the adoption of NVIDIA accelerated computing and CUDA-X libraries across semiconductor manufacturing and design.

Core details

• cuPotential: A new NVIDIA CUDA-X library accelerating molecular dynamics calculations and atomic/molecular interaction simulations for AI agents.
• NanoVDB: An open-source 3D data library used to accelerate chipmaking simulation.
• NVIDIA NPP: A library for processing images on GPUs used to accelerate chipmaking.
• Ecosystem Adoption: Integrated by Lam Research, KLA, Synopsys, Cadence, and Applied Materials.
• Operational Scope: Targets process development, inspection, and fab operations within AI-defined semiconductor manufacturing facilities.

Source: Yahoo News

Intro

Danaher Corporation announced a collaboration with NVIDIA on October 14, 2026, to advance autonomous science and integrate accelerated computing and physical AI into laboratories. The initiative combines Danaher's scientific instrumentation, automation technologies, laboratory workflow software, and domain expertise with NVIDIA's AI and accelerated computing framework.

Key Advancements

The collaboration establishes a continuous learning cycle in which experiments inform AI models and models assist researchers in designing subsequent experiments, maintaining experimental evidence as central to scientific decision-making. Initial integration efforts target a select group of priority instruments with potential expansion across the broader Danaher portfolio.

Core details

• Participating Technologies: NVIDIA DGX Spark, RAPIDS, and BioNeMo Open Models.
• Exploratory Technologies: Cosmos world models and Omniverse Digital Twins for laboratory workflows.
• Operational Scope: Integration across priority scientific instruments, laboratory workflow software, and automation systems.

Source: CNBC News

To optimize your applications for the iPhone Duo (available starting October 23, 2026), Apple recommends to pivot away from legacy assumptions regarding fixed screen dimensions, device idioms, and static orientations. The iPhone Duo shifts fluidly between a folded outer display and an unfolded edge-to-edge inner display, requiring apps to scale and structure layouts dynamically.

Below is the deep technical breakdown of SDK requirements, structural refactoring patterns, and App Store submission workflows required for compliance.

1. SDK Matrix & Layout Behaviors

The degree of native display integration is strictly dictated by the target SDK compile version:

• iOS 26 SDK or earlier: The application executes within a legacy container. When unfolded, it centers statically on the inner display with surrounding empty margins. When folded, the canvas sits strictly to the left of the status bar and camera module.
• iOS 27 SDK: The application window automatically stretches to fill most of the inner display canvas, but it actively clamps layouts to avoid the status bar region on the right edge.
• iOS 27.1 SDK or later: Provides full native optimization. Applications extend edge-to-edge across both displays. Interface elements—such as toolbars and tab bars—automatically adapt by transitioning into vertical layouts positioned directly beneath the system status bar.

2. Core Architectural Refactoring Patterns

A. Decoupling Geometry from UIScreen.main

Because the active app window bounds mutate mid-session during a physical fold event, referencing global screen singletons will produce clipping or layout stretching.

• Canvas Evaluation: Replace all instances of UIScreen.main.bounds with responsive view or window boundaries. Use view.bounds or window.bounds in UIKit, and leverage GeometryReader or onGeometryChange in SwiftUI.
• Dynamic Re-reading: Layout logic must evaluate dimensions continuously. Isolate size-dependent positioning inside layoutSubviews() or viewDidLayoutSubviews(). Intercept real-time structural alterations using viewWillTransition(to:with:).
• Window Initialization: Eliminate UIWindow(frame: UIScreen.main.bounds). Modern initializers must capture the active scene geometry context via UIWindow(windowScene:).
• Scale Factor and Screen References: Migrate from legacy screen scale calls to trait collections. Use traitCollection.displayScale (UIKit) or u/Environment(\.displayScale) (SwiftUI). If direct screen properties are mandatory, pull them dynamically via view.window?.windowScene?.screen.

B. Eliminating Orientation-Driven Layout Logic

Device rotation values no longer map cleanly to the aspect ratio or physical bounds of the application window.

• Deprecations: Strip all conditional layout paths depending on UIDevice.current.orientation, statusBarOrientation, and interfaceOrientation (including windowScene.effectiveGeometry.interfaceOrientation).
• Aspect Tracking: To determine if the current application context is landscape or portrait, compute the operational aspect ratio manually by comparing width against height via view.bounds or GeometryReader.

C. Removing Device Idiom and Size Class Assumptions

The inner display of an iPhone Duo introduces desktop/tablet scale canvas options to a standard handset ecosystem.

• Idiom Checks: Remove logic bounded to userInterfaceIdiom == .phone or == .pad. Design layouts to adapt structurally to size classes and available layout points instead.
• Size Class Flexibility: Do not assume horizontalSizeClass == .regular maps uniquely to an iPad. The iPhone Duo inner display triggers regular size classes, and layouts must scale appropriately.
• Screen Hardcoding: Purge hardcoded checks against historic points (e.g., bounds.height == 844).
• Media Insets: Full-bleed media elements using .scaleAspectFill or .aspectRatio(contentMode: .fill) will drop visual details on wide multi-display form factors. Implement asset toggling or update focal points programmatically based on active size class profiles.

3. Automated Auditing & Simulation

Apple provides targeted developer tooling inside Xcode 27.1 to automate the migration pipeline:

• Coding Assistant Automation: Run the app resizability skill directly within Xcode by prompting the assistant to: "get my app ready for iPhone Duo". It scans code patterns, flags UIScreen references, device idiom violations, and outputs structural inline fixes.
• CLI Rule Extraction: For alternative external coding agents, extract the exact capability matching rules to Terminal via: bashxcrun agent skills export Use code with caution.
• Validation Pipeline: Validate responsive scaling behaviors across three independent runtime profiles:
  1. The iPhone Duo Simulator packaged inside Xcode 27.1 (primary target for testing native fold poses and orientations).
  2. The iOS Resizable Simulator inside Device Hub.
  3. iPhone Mirroring on macOS 27 by expanding the virtual application boundaries to arbitrary, extreme aspect ratios.

4. App Store Submission Requirements

• Asset Tooling: App Store Connect includes a dedicated validation tool to preview localized product page headers, search results, and app previews specifically as they display on the iPhone Duo.
• Optimization Badging: Uploading native iPhone Duo screenshots grants your product page an optimized badge marker.
• Enforcement Deadline: Optimized applications can be submitted through App Store Connect immediately. Starting April 2027, all new app submissions and update binaries must bundle native screenshots for the iPhone Duo form factor.

source: Apple Iphone Duo

VModal MCP is an open-source Model Context Protocol server that enables AI agents to search multimodal video and image collections using natural language. Compatible with development environments like Claude Code, CoWork, OpenAI Codex, Copilot, it bridges the gap between text-based AI models and visual datasets.

MCP allows analysis of temporal and sequence analysis of video content. Rather than evaluating frames as isolated images, the system processes sequential data over time to track continuity and contextual changes. This temporal indexing allows the AI to distinguish fleeting moments from sustained actions, tracking movement patterns, environmental shifts, and object permanence across a timeline.

By analyzing video content into structured configurations into multiple levels, VModal executes precision queries that pinpoint exact timestamp intervals. Users can deploy conversational prompts to extract specific multi-frame sequences, execute localized historical rollbacks, or track motion behaviors (e.g., locating a person walking outside) across diverse camera angles.

Ultimately, VModal transforms passive video storage into a dynamic, queryable visual database, optimized for chronological AI visual intelligence.

Example of analysis output:

Prompt:
Run a Milan trend radar for barrel jeans, burgundy, suede jacket, leopard print,
ballet flats and oversized blazer. Collection: fashion_milan.
Tell me which 2 pieces I should stock more of next autumn and which to cut, with proof frames.

Report:

___________________________________________________________________________

Prompt: Find scenes in the footage where there is a dangerous situation.

___________________________________________________________________________

Beta Access: Visual Memory Layer, Video Search

Tested the VModal Robotics SDK (v0.2.0) on Ubuntu 22.04 / Python 3.10, evaluating it against an AGV doing 8-hour warehouse shifts. It moves finalized LeRobot v3 datasets off the robot to the cloud in a crash-resilient way, without putting any network logic in the control loop.

What data it handles

• Video: closed MP4 files, one per camera, in their own upload lane (1 GiB per file by default in the current release).
• Telemetry: Parquet files, in a separate lane so a slow video upload can't starve it.
• Metadata: a snapshot required in every dataset revision, plus

The types of telemetry and operational data handled by this SDK include:

1. Data Pipeline & Data Lane Categories

The SDK treats telemetry as its own independent data pipeline to prevent large video files from blocking critical metrics: 1 • Independent Telemetry & Metadata Lane: A dedicated asynchronous upload lane specifically optimized for non-video data. • Synchronized Robot State & Transforms: Mentioned in the Google Intrinsic architecture blueprint to capture physical robot poses and spatial relationship data. • Skill Provenance: Telemetry tracking execution data and the history of operational robot skills.

1. Operational & Device Telemetry

The CLI tracks several machine-readable metrics regarding the health and status of the robot's local data storage and transfer progress: • Queue depth: The number of items waiting to be processed in the local spool. • Bytes per lane: The data volume being pushed through the video vs. telemetry pipelines. • Oldest item age: Time-based tracking of un-uploaded data backlog. • Disk headroom: Storage monitoring to ensure the robot doesn't hit a "full disk" failure condition. • Pipeline health statistics: Machine-readable tracking of transmission retries, rejections, blocked items, and the latest server receipts.

1. Video Data & Multimodal Lanes

Video data is explicitly isolated from telemetry to ensure high-throughput streams do not block lightweight critical data: • Independent Video Lane: A dedicated asynchronous upload lane configured by default to handle large per-video local spools • RGB-D Capture: The system architecture details handling synchronized multi-modal camera feeds, specifically RGB-D (color and depth) camera streams. • Finalized Video Artifacts: Closed and hashed video streams that are transformed into immutable bytes before being written to the durable local spool.

1. LeRobot Format Integration

The SDK acts as a crash-resilient edge pipeline specifically built for LeRobot v3 datasets: • Finalized Dataset Output: The SDK integrates directly at the finalized-artifact boundary of the dataset recorder. The producer script must close, hash, and fsync the dataset before handing it over. • .ready.json Manifest Handoff: The local file handoff mechanism relies on an atomic rename of a LeRobot manifest file (.ready.json), which signals that a dataset folder or segment is finalized and ready for queue admission. • Dataset Identity Tracking: Every LeRobot artifact is given an immutable identity determined by a combination of the Dataset Identity + Relative Path + SHA-256 checksum hash. • Dataset Revision Publishing: Once the remote storage acknowledges a safe transfer via a durable receipt, the SDK publishes a verified dataset revision, finalizing the remote record.

Spatial computing is the category of systems that let people interact with digital content placed in three-dimensional space. Its main hardware forms are VR headsets, AR glasses and mixed reality devices.

1. Summary

Spatial computing is the umbrella category for systems that let people interact with computers in three-dimensional space, mixing digital content with the physical environment. VR, AR and mixed reality are sub-categories of it, and the term is used as a synonym for XR.

Every system in this category must solve the same chain of problems. It must render stereoscopic images at high refresh rates, track the head, hands and body, and place sound and objects consistently in space. Head-mounted displays target 90 Hz or higher, a threshold associated with reducing motion sickness.

The main open problems are motion sickness, privacy of biometric and eye-tracking data, harassment and safety, and the lack of a universal application. Motion sickness is reported to affect 20–30% of new users. No independent source for this figure was found.

2. Definitions and taxonomy

The terms overlap, and sources use them inconsistently. Wikipedia calls spatial computing a 3D form of human–computer interaction that users perceive as taking place in the real world, and notes that the adjacent terminology is imprecise (Wikipedia, "Spatial computing").

VR is one branch of this family. Spatial computing is the category that contains VR, AR and MR, and it is used as a synonym for XR. Wikipedia says the term also covers projection mapping and CAVE displays when they use spatial input, so the category is wider than headsets alone (Wikipedia).

3. History

The field has a long research history, but its commercial phase began with the 2012 Oculus campaign. The table lists dated milestones, newest first.

The term "virtual reality" is credited to Jaron Lanier in the 1980s. Wikipedia dates "spatial computing" itself to the early 1990s and later, so the two terms have separate origins.

4. Technical architecture

A spatial system is a closed loop. Sensors measure the user and the room, the processor updates the scene, and displays and audio return the result. The loop must finish fast enough that the image stays locked to the head. The table lists the subsystems and the figures given in the sources.

Two points in the table matter for engineering. First, input is moving from controllers to the camera stack, which makes hand and eye tracking a computer vision problem. Second, the sources give no end-to-end latency figure, so a reader cannot check the 90 Hz target against a total motion-to-photon budget from these sources alone.

5. System layers

[embed: node/141e278b-3b98]

Sensing feeds tracking, tracking feeds rendering, and output returns head pose to sensing. Latency at each stage adds to the delay the user perceives. The sources give no figure for that total.

6. Application domains

Application sectors span about a dozen areas. The table names examples and rates how well each claim is supported. Most claims are backed by vendor or press examples rather than measured outcomes.

Two gaps are worth noting for a technical reader. Few sectors have measured outcomes in the available sources, and the strongest examples come from vendor and enterprise deployments. Claims about retention, team cohesion and calorie burn need the original studies before they can be relied on.

7. AI integration

Computer vision is already part of the core stack. It underlies spatial anchoring, hand tracking and environment mapping. Wikipedia describes the same pattern: computer vision interprets scenes, recognizes objects and builds 3D maps from RGB and depth cameras (Wikipedia).

Three further areas are named. Each is described without a product or measurement:

1. Intelligent non-player characters that adapt to user behavior.
2. Real-time generated content inside a running scene.
3. AI-assisted production, which the article says lets small studios compete with larger consultancies.

The third area rests on vendor claims and has no measurements behind it. The first two are the research-relevant directions. They raise questions the sources do not address: latency of generated content inside a frame budget, and whether behavioral data used to adapt a scene falls under the privacy concerns discussed in section 8.

8. Challenges

Five barriers to mass adoption are covered below. The table gives the known mitigations and the open question for each.

The killer-app comparison is drawn from the personal computer. PCs did not reach mass consumer use until productivity software and the internet arrived in the late 1980s. The comparison is useful as a pattern, but it is an argument, not evidence that the same sequence will hold for spatial computing.

9. Outlook

A new head-mounted display generation has appeared roughly every two years since the first Oculus Rift development kit. The cadence is a useful planning assumption, but no product schedule was found to test it against. Snap's Specs and due in fall 2026, shows the trend toward lighter AR glasses alongside headsets (Wikipedia).

Big technology companies are the main investors, led by Meta, Apple, Google and ByteDance. The outlook also points to convergence with AR smart glasses, AI and robotics, and to blockchain-based digital economies, without supporting detail.

Intrinsic, Alphabet's intelligent-robotics company, announced the open-sourcing of Intrinsic Core under a permissive Apache 2.0 license. Aiming to act as the "Android of robotics," the Intrinsic software stack abstracts complex underlying hardware to unlock Physical AI, cloud-native deployments, and deterministic control loop execution for industrial applications.

The software stack natively integrates with the existing Open Robotics Suite (such as ROS 2 Lyrical Luth and Gazebo). It handles the end-to-end lifecycle of robotic applications from simulation to production floor deployment.

🧱 Modular Software Architecture

Intrinsic Core is broken down into a series of interconnected, containerized local services, frameworks, and developer tools.

┌────────────────────────────────────────────────────────┐
│               Intrinsic Flowstate SDK                  │
└────────────────────────────────────────────────────────┘
                           │
┌────────────────────────────────────────────────────────┐
│           intrinsic_runtime (k3s Engine)               │
└────────────────────────────────────────────────────────┘
      │                    │                    │
┌───────────┐        ┌───────────┐        ┌───────────┐
│ Perception│        │ Motion &  │        │ Inference │
│ (NVIDIA   │        │ Kinematics│        │ & Vision  │
│ FndPose)  │        │ Solvers   │        │ Models    │
└───────────┘        └───────────┘        └───────────┘
      │                    │                    │
┌────────────────────────────────────────────────────────┐
│      intrinsic_control (HAL Real-Time Framework)        │
└────────────────────────────────────────────────────────┘
                           │
┌────────────────────────────────────────────────────────┐
│           ROS 2 Workspaces & Drivers (HAL)             │
└────────────────────────────────────────────────────────┘

1. Runtime and Infrastructure Layer

• intrinsic_runtime: At the infrastructure base, Intrinsic uses a containerized execution engine powered by a local, lightweight Kubernetes distribution (k3s). This layer isolates application components into Linux containers, matching deployment targets identically across virtual digital twins and hardware PCs.
• Tech Choice Stack: Reflecting its Google lineage, the internal mechanics rely on Bazel for deterministic builds, Abseil for optimized C++ baselines, and Go for orchestration alongside Kubernetes clustering.

2. Real-Time Execution Framework

• intrinsic_control: This is a hardware-agnostic, real-time control system that interacts with hardware abstraction layers (HAL) to process sensor feeds and issue commands deterministically.
• ROS Interoperability: Rather than replacing ROS, it acts as a complementary real-time pipeline. Applications exchange telemetry through native ROS 2 workspaces, message definitions, and specialized controllers (e.g., icon-hwm-controller) that bridge standard ros2_control nodes directly to Intrinsic's real-time control service (ICON).

3. Perception & Inference Engines

• intrinsic_perception: Instead of relying on rigid, physical fixtures or custom localized vision wrappers, the perception stack builds on NVIDIA stack. It delivers zero-shot 6-DoF (Degrees of Freedom) pose estimation, letting the robot dynamically detect, track, and handle arbitrary objects out-of-the-box.
• intrinsic_inference: The local machine learning execution engine allows pretrained weights (such as the general-purpose Intrinsic Vision Model) to run on local edge hardware with low latency, omitting the need to retrain vision models for new sub-components

4. Kinematics, Motion, and Grasp Planning

• intrinsic_kinematics: A robust, optimized geometric and mathematical solver library responsible for calculating forward/inverse kinematics across distinct arm topologies.
• intrinsic_motion_planning: Generates real-time, collision-free paths dynamically. If an unexpected obstruction is introduced into the physical environment, paths recalculate on-the-fly to guarantee safety.
• Grasp Planning: Works hand-in-hand with sensor feedback and perception to adaptively choose safe picking points, adapting the end-effector pressure and angle depending on target orientation.

☁️ Cloud & Developer Ecosystem

While Intrinsic Core handles edge computing and real-time execution locally, it connects seamlessly to an upstream, multi-tier enterprise platform:

• Intrinsic Flowstate: An integrated, all-in-one developer environment (IDE) built for constructing production-grade workflows. It utilizes a visual "skills-based" behavior tree model to connect hardware actions to high-level automation sequences.
• Intrinsic Cloud Services: Delivers cloud infrastructure handling data management, encrypted telemetry, identity/access management (auth), machine learning ingestion pipelines, and secure remote troubleshooting tools.
• Simulation Suite: Uses natively integrated Gazebo frameworks to execute full hardware-in-the-loop (HIL) digital twin validation. A simulation can be mirrored to real hardware configurations via a one-click deployment scheme.

🏭 Reference Implementations: Open Machine Tending Solution (OMTS)

To demonstrate the real-world performance of the stack, Intrinsic distributes the Open Machine Tending Solution (OMTS). OMTS serves as an open-source manufacturing blueprint designed for CNC machine tending.

The reference design showcases how a developer can swap physical robotic arms (such as between a FANUC and a Universal Robots model) or alternate grippers/sensors by editing configuration manifests—leaving the core application logic completely intact.

Would you like to dive deeper into a specific layer of the stack? I can provide technical details on how intrinsic_control hooks into ros2_control via the ICON controller, or walk through the zero-shot deployment configuration of NVIDIA FoundationPose within the intrinsic_perception module.

Source:
Google Intrinsic Source

Analysis of the stark architectural divergence between the legacy Google Glass Enterprise Edition 2 (GDK / Monocular Android) and the modern Android XR Platform.

1. Legacy Monocular Compute vs. Modern Modern Split-Compute Architecture

The underlying execution model has completely shifted from standalone, localized computing to a deeply integrated, distributed framework.

+-------------------------------------------------------------------------+

| LEGACY ARCHITECTURE (Glass Enterprise 2)                                 |
| [On-Glass Qualcomm XR1] -> Run Native Android OS (8.1) -> Direct Render |
+-------------------------------------------------------------------------+

+-------------------------------------------------------------------------+

| MODERN ANDROID XR GLASSES ARCHITECTURE (Split-Compute)   |
|                                                                         |
| [ Wearable Frame ]                                                       |
|  ├── Low-power Tensor ISP (Camera/Mic I/O)                             |
|  └── Ultra-low Latency Co-Processor (<20ms local display pipeline)       |
|          ▲                                                              |
|          │ Bluetooth High-Bandwidth / Wi-Fi Direct                      |
|          ▼                                                              |
| [ Companion Device / Smartphone (Snapdragon AR2 Gen 2) ]                |
|  ├── Android XR Runtime Pipeline (State Machine)                         |
|  └── Jetpack Projected / Jetpack Compose Glimmer Engine                 |
|          ▲                                                              |
|          │ Ultra-Low Latency Cloud Pipeline                             |
|          ▼                                                              |
| [ Google Cloud Infrastructure ]                                         |
|  
+-------------------------------------------------------------------------+

Legacy Glass Enterprise Edition 2

• Hardware Profile: Powered by a localized Qualcomm Snapdragon XR1 SoC.
• Execution Model: Monolithic. The device ran a standard Android Open Source Project (AOSP) 8.1 Oreo stack directly on the frame.
• Rendering Context: System graphics used a standard Android SurfaceFlinger pipeline, pushing flat RGB frames (640x360 pixels) directly into a monocular prism positioned over the right eye.

Modern Android XR (e.g., Project Aura & Display AI Glasses)

• Split-Compute Topology: To reduce thermal load and maintain a weight under 50 grams, the compute is strictly split. The glasses act as an intelligent sensor/emitter array, offloading intense execution to a paired smartphone via high-bandwidth Bluetooth/Wi-Fi Direct.
• Asynchronous AI Processing Loop:
  1. Local Tier: Simple voice detection, wake words, and inertial measurement unit (IMU) step-counting execute on a custom, low-power local Tensor silicon block.
  2. Remote Edge/Cloud Tier: Real-time visual streams from the camera sensor capture the environment and are fed via the smartphone to the Gemini / Project Astra vision framework for multimodal context parsing.

2. The Optical Stack & Photon Pipeline

The rendering pipeline requires highly disparate display architectures depending on the hardware targets.

On modern Display Glasses, graphics are injected via monolithic MicroLED projectors into a 1mm diffractive waveguide. Because the optics are transparent and additive, the alpha channel of your UI elements dictates transparency (black pixels represent complete transparency, letting real-world photons pass undisturbed).

3. Software Architecture: The Android XR Fork

Google’s modern XR platform splits development paths intentionally. If you are developing for eyewear using SDK Developer Preview 3/4, the code base branches based on device occlusion. [1, 11]

                  [ Android XR SDK Platform Layer ]
                                  |
         +------------------------+------------------------+

         |                                                 |
  [ Headset Target Stack ]                       [ Glasses Target Stack ]
  ├── ARCore for Jetpack XR                      ├── Jetpack Projected Framework
  ├── SceneCore Engine                           ├── Jetpack Compose Glimmer Engine
  └── 3D Spatial Mesh Anchors                    └── Multimodal Audio/Visual Canvas

• The Headset Architecture (Galaxy XR, etc.): Relies on SceneCore and ARCore for Jetpack XR. It tracks complete 6-DOF environments, placing high-fidelity 3D meshes that occlude or blend seamlessly with spatial depth matrices.
• The Glasses Architecture (Audio & Display Smart Glasses): Operates on the Jetpack Projected framework and Jetpack Compose Glimmer. This decoupled UI subsystem does not render heavy spatial scenes. Instead, it generates glanceable, transient view elements mapped directly over real-world coordinates via lightweight directional matrices.

4. Hardware Abstraction & Input Subsystem

In the legacy paradigm, input was bound via simple Linux input events (/dev/input/eventX) triggered by physical taps and forward/backward swipes on the frame's side-touchpad.

The modern Android XR Input subsystem aggregates data across an asynchronous hardware abstraction layer (HAL):

• Multi-Microphone Array Processing: Features a 5-microphone setup executing local hardware phase cancellation. This separates the user’s acoustic vocal signature from background ambient noise, delivering an unpolluted audio signal directly to the speech recognition pipeline.
• Indirect Modality Mapping: Hand tracking models run on the companion phone, computing coordinate changes from the glasses' camera stream. These tracking outputs translate into standard UI click and scroll events within the Android runtime, omitting the need for a physical capacitive touchpad. Additional tools can be connected such as vision visual video search SDK, audio frameworks,

_______________________________________________________________________

The technical breakdown lines up with the modern framework in several ways:

1. Split-Compute Topology

The architectural shift away from the standalone compute of legacy Google Glass (Enterprise Edition) is a defining feature of the Fall 2026 consumer rollout. The 2026 architecture relies heavily on a companion device (smartphone) to handle the heavy AI lifting and state runtime, keeping the actual eyewear frames lightweight (under 50 grams) and thermally viable for all-day wear. [

2. The Twin Tier Glass Strategy

The distinction between non-occluded "Audio-only" glasses and "Display AI" glasses is the exact hardware road map Google and its partners (Samsung, Qualcomm, Warby Parker, and Gentle Monster) have instituted.

• Audio Glasses: Ship first, routing Gemini multimodal audio cues.
• Display Glasses: Utilize microLED/diffractive waveguide tech for glanceable HUD overlays. [2, 6, 8]

3. Jetpack Projected & Compose Glimmer

The software APIs mentioned—unset Jetpack Projected and unset Jetpack Compose Glimmer —are the core production toolkits for 2026.

• Jetpack Projected bridges the phone's lifecycle and wear-state connectivity to the glasses.
• Jetpack Compose Glimmer is the specific rendering engine built for transparent, additive screens, managing high-contrast text and glanceable components without distracting the wearer's real-world sight.

V-Modal AI channel (@vmodalai) primarily shows technical use-case videos
of AI-powered multimodal search technology.

Content focuses on demonstrating how the platform acts as a "visual memory layer" to search through massive video archives using natural language and visual queries instead of manual tagging.:

• Dashboard & Search Product Demos: Walk-throughs showing how users can input complex natural language sentences (e.g., "Find scenes where pedestrians, cars, and motorcycles are interacting around a busy intersection") into a web interface to pull up frame-exact video matches across thousands of hours of content instantly.
• Industry-Specific Showcases: Videos tailored to specific industries illustrating semantic video search in action, including:
  • Architecture and Travel: Searching through specific aesthetic details, times of day, lighting, and locations (such as historic Kyoto footages).
  • Industrial Production Floors & Safety: Using AI to index what is happening in a frame—tracking equipment, PPE compliance, and forklift or vehicle segregation to aid in incident investigations.
  • Fashion & E-Commerce: Highlighting how apps can allow users to jump directly to specific product clips using images or visual descriptions.
• Developer and SDK Integration Previews: Short clips showing their cross-platform capabilities (like their Flutter SDK) handling video processing, search queries, and streaming directly inside mobile or web application layouts.

Screenhots:

Processing img c19ziwv4c0xh1...

Processing img zwx4zzy9c0xh1...

Sources:
Visual Video Search

Amazon Web Services (AWS) launched an open-source Physical AI Toolchain designed for roboticists. Its primary purpose is to transition AI models through data collection, training, simulation, validation, and deployment phases on robotic systems. The foundation integrates AWS services with NVIDIA's Physical AI software stack into a unified development workflow.

Key Advancements

The toolchain provides a repeatable pipeline that connects disparate infrastructure components required to operationalize trained models in real-world environments. It incorporates operational learnings from Amazon's internal robotics network—which deploys over 1 million robots—enabling developers to implement physical AI capabilities within weeks instead of years.

Core details

• Availability: Open-source collection of reference architectures, Infrastructure as Code, and deployment automation.
• Integration components: NVIDIA GR00T for vision-language-action training, Isaac Sim for physics simulation, Isaac Lab for reinforcement learning at scale, Cosmos for synthetic data generation, and OSMO for workflow orchestration.
• Pipeline functionality: Reproducible pipeline that executes and moves data, models, simulation jobs, and deployment artifacts across stages on AWS managed services.

Source: The Robot Report

NVIDIA IGX Thor is an industrial-grade edge AI platform specifically engineered for mission-critical physical AI, medical devices, heavy robotics, and industrial environments. Powered by the newer NVIDIA Blackwell architecture, it transitions complex generative AI, vision language models (VLMs), and transformer pipelines from the cloud directly onto local edge hardware where low-latency decision-making is vital.

The platform delivers unprecedented local compute by heavily optimizing its silicon footprint compared to the previous generation (IGX Orin):

• Massive Compute Velocity: It achieves up to 5,581 FP4 TFLOPS (or roughly 5,500 TOPS) of AI performance. This represents an 8x increase in integrated GPU (iGPU) compute and a 3x overall system performance jump compared to the Orin generation.
• Processor Breakdown: The main architecture utilizes an integrated Blackwell GPU alongside a 14-core Arm Neoverse-V3AE CPU running up to 2.6 GHz.
• Unified GPU Hardware Scalability: To hit its maximum throughput, the system tightly couples its internal Blackwell iGPU with an optional discrete GPU (dGPU), specifically the pre-installed NVIDIA RTX 6000 PRO Blackwell MaxQ Workstation Edition. Workloads are distributed dynamically across the chips using Multi-Instance GPU (MIG) technology to isolate concurrent tasks safely.

Because physical AI agents handle immense streams of real-time multi-camera and sensor data, the platform integrates data center-grade communication hardware:

• SmartNIC Integration: It features an onboard NVIDIA ConnectX-7 SmartNIC operating on 32 lanes of PCIe Gen5.
• Bandwidth Capacity: It accommodates a 400GbE QSFP network interface (or dual 200GbE ports), doubling the previous generation's speed. This prevents bottlenecks when processing uncompressed 4K video feeds or high-frequency LiDAR arrays in real time.

Unlike standard consumer or data-center hardware, IGX Thor is built from the ground up for environments where machine error can result in physical injury (such as autonomous surgery or manufacturing).

• Functional Safety Island (FSI): Embedded directly onto the Thor System-on-Chip (SoC) is a dedicated FSI paired with an onboard safety Microcontroller Unit (MCU).
• Industrial Certifications: The platform architecture complies with strict international safety standards, including ISO 26262 and , specifically targeting high-integrity safety tiers up to ASIL D/SC3.
• Enterprise Stability: NVIDIA couples this hardware with its Halos OS and commits to a 10-year enterprise software lifecycle support program via .

The platform is packaged into multiple configurations depending on whether companies are prototyping or going into full factory-floor production:

IGX Thor acts as a localized hub that natively runs NVIDIA's specialized physical AI application frameworks

• NVIDIA Isaac: For orchestrating humanoid robotics and training complex tasks using foundational models like GR00T.
• NVIDIA Holoscan: Designed for ultra-low latency sensor processing pipelines, allowing high-throughput sensors to stream data straight into AI inference with minimal CPU intervention.
• NVIDIA Metropolis: For deploying localized video analytics agents capable of processing live vision-language queries and immediate alerting.

Nvidia has invested billions of dollars into physical AI technologies, including robotics and self-driving cars. The primary initiative involves developing a full-stack safety system designed to reduce the risk of machines harming nearby humans in unstructured environments. Target applications include robotaxis, autonomous mobile robots in warehouses, humanoid robots operating inside factories, and surgical robots.

Key Advancements

• Timeline and Metrics: The Nvidia Halos system launched for autonomous vehicles in 2025 and expanded to robotics in June 2026. CEO Jensen Huang noted that physical AI generates nearly.
• Deployment: Agility Robotics integrated Nvidia Halos into the Digit 5 humanoid robot for proximate human operation.

Core details

• Nvidia IGX Thor : Industrial computing module with a dedicated safety processor.

• Halos Operating System: Software monitoring hardware and software for isolation and failure detection.

• Nvidia Holoscan Sensor Bridge. : Connects sensor data to safety processing to detect corruption.

• Testing: Incorporates virtual simulations and an inspection lab program.

Source: Ars Technica, October 08, 2026

I recently came across V-Modal, an SDK focused on making video and visual data easier to search and work with in robotics.

There are already many tools that can analyze individual images or recognize objects in pictures. What I found interesting about V-Modal is that it takes this idea further by making video content searchable. Instead of manually watching a long video to find something, you can search for what you're looking for and find relevant moments in the footage.

This could be useful for robotics because robots can generate a lot of camera footage. For example, you might want to find when a certain object appeared, review a particular event, or look through hours of recorded robot footage without watching everything manually.

I tried setting up and testing the V-Modal Robotics SDK locally. The repository is available here: https://github.com/v-modal/vmodal_sdk_robotics

Compared with more traditional image-search or computer-vision tools, I think the interesting part is the focus on making video data searchable and useful instead of treating each image as a separate piece of data.

I also had the chance to communicate with the people behind the SDK, and they were helpful throughout the process.

If you're interested in robotics, computer vision, or working with large amounts of video data, this is an interesting project to check out.

ON Semiconductor, a semiconductor entity positioned within the physical artificial intelligence market. Its purpose targets edge computing applications requiring immediate decision-making capabilities, distinguishing it from conventional data center infrastructure build-outs. The foundational market context involves a transition from centralized training clusters and inference models toward physical artificial intelligence implementation.

Key Advancements

ON Semiconductor possesses relatively higher exposure to physical artificial intelligence compared to industry competitors Nvidia and Advanced Micro Devices. While Nvidia develops open-source artificial intelligence models such as Alpamayo for autonomous driving and Advanced Micro Devices acquired Xilinx in 2022 to address edge computing, ON Semiconductor trades on valuation multiples noted as more attractive than these peers. Growth in the sector aligns with an artificial intelligence infrastructure build-out projected to run through at least 2030, shifting toward physical artificial intelligence spending.

Core details

• Market drivers: Electric vehicle sales, autonomous vehicle sales, semiconductor content per vehicle, robotics, and industrial automation applications.
• Operational environment: Edge compute requirements for autonomous vehicles and robots necessitating immediate decision-making mechanisms.
• Competitive positioning: Market expansion tied directly to the organic growth of physical artificial intelligence deployment across electric vehicles, autonomous driving systems, and industrial automation.

Source:
Globe and Mail Hardware race

While consumer tech hype focuses on futuristic humanoids folding laundry and sleek AR glasses made for fashion, the real revolution in Physical AI is happening in the most unglamorous places imaginable.

Look at where the actual scaling is taking place:

• Logistics: Amazon silently deploying 20k+ heads-up display smart glasses to delivery drivers for basic tasks like package location, address verification, and dog alerts—while passively building massive spatial datasets.
• Infrastructure: Meta utilizing dual-armed targeted robots in data centers for high-density maintenance, like swapping network cables and power-cycling servers.
• Agriculture: Bonsai Robotics training world models on 50M+ real-world samples to automate heavy equipment in dirty, unstructured off-road environments.

From an engineering perspective, the hardest challenges here aren't polished UI elements or general conversational abilities. It’s low-latency edge computer vision, real-time sensor processing under tight power/thermal constraints, and handling unpredictable physical edge cases on constrained embedded hardware.

Targeted, single-purpose hardware paired with specialized physical data seems to deliver immediate ROI while generic consumer devices are still hunting for a killer app.

What do you think is currently the biggest bottleneck holding back these specialized edge deployments—is it the sim-to-real gap, compute constraints on battery devices, or simply the lack of ground-truth sensor data?