One software stack for hardware, Unreal simulation, and digital twins
The platform is organized into four layers that run unchanged across all three realities. What you validate in Unreal Engine 5 is exactly what executes on the robot and what the twin mirrors back to you.
Navigation & Control
Deterministic real-time motion for legged, wheeled, and drive-by-wire platforms.
FAST-LIO2 / LIO-SAM tightly-coupled LiDAR-inertial odometry, GNSS-RTK fusion, loop closure with pose-graph optimization (GTSAM).
3D voxel + 2.5D elevation maps, semantic costmap layers, spatio-temporal obstacle decay, multi-session map merging.
Theta*, Hybrid A*, and lattice planners with kinodynamic feasibility checks for differential, omni, legged and Ackermann bases.
MPPI and MPC controllers at 200–250 Hz with dynamic obstacle prediction and Control Barrier Function safety constraints.
Perception & Sensor Fusion
Hardware-synchronized multi-modal pipelines from raw sensor frames to 3D semantic understanding.
Motion deskewing, ground segmentation, voxel downsampling, and GPU clustering at 1.4M pts/s.
Radiometric thermal calibration, stereo depth, thermal↔RGB↔LiDAR extrinsic projection, HDR exposure fusion.
Hotspot, corrosion, leak, gauge-reading and anomaly detectors trained on synthetic + field data (PyTorch → TensorRT).
3D multi-object Kalman tracking with radar-LiDAR association for autonomous driving.
Fleet Orchestration
Coordinates heterogeneous fleets across shared workspaces with open standards.
Market-based auctions with battery, payload, and deadline constraints.
Time-expanded graph reservations for deadlock-free corridors and intersections.
VDA5050 2.1, Open-RMF adapters, ROS 2 DDS / Zenoh bridging to cloud.
Authenticated SSE/WebRTC streams with p99 < 20 ms for fleet-wide dashboards and twins.
Agentic Behavior Planning
Foundation-model reasoning connected to deterministic, verifiable robot execution.
3D scene graphs fusing detections, map topology, and mission context for LLM/VLM reasoning.
Agents emit BT.CPP subtrees from a constrained node library; schema-validated before execution.
Every synthesized action is bounded by Control Barrier Functions, velocity envelopes, and geofences at 1 kHz.
Field anomalies are captured as episodes, replayed in Unreal, and used to improve planners.
Deterministic at the edge, cognitive in the cloud
ROS 2 C++20 real-time graph: SLAM, planning, control, perception and the BehaviorTree.CPP executor on an edge GPU. Runs identically on hardware and inside Unreal Engine.
Python FastAPI + Pixi compute engine for agentic behavior synthesis, twin telemetry ingestion, AST-guarded analytics queries, and JWKS-authenticated streaming.
Next.js 15 console with a split-stage layout: navigable telemetry snapshots, 3D twins, fleet health, and a Copilot dock streaming from the engine.
See the stack on a real platform
Browse the robot catalog to see sensors, specs, and twin channels per platform.