Cloud-Edge Hybrid Inference
Run AI at the edge for speed and privacy, fall back to the cloud for complexity. Offline-first operation, local context caching, and native SDKs for mobile and IoT.
What is Forge Edge?
Forge Edge brings AI inference to the device. Instead of sending every query to a cloud API and waiting for a response, Edge classifies each query by complexity and routes simple tasks to a local model running on the device. Complex tasks still go to the cloud. The user gets faster responses, lower costs, and the ability to work offline.
The system is built around a lightweight ONNX complexity classifier that runs in under 5 milliseconds. It evaluates each query and decides: can a local model handle this, or does it need cloud-grade reasoning? The split is automatic and adaptive, adjusting based on network conditions, device capabilities, and model confidence scores.
For developers building mobile apps, IoT products, or latency-sensitive applications, Forge Edge provides native SDKs for React Native, Flutter, and embedded platforms. The SDK handles model loading, inference routing, offline queuing, and cloud synchronization out of the box.
Core Capabilities
Everything needed to run AI inference at the edge while maintaining seamless cloud integration for complex tasks.
Hybrid Inference Routing
Every query is evaluated before execution. Simple tasks -- classification, extraction, FAQ lookups -- run locally on edge models. Complex tasks -- multi-step reasoning, code generation, creative writing -- route to cloud LLMs. The split happens automatically based on the complexity classifier.
Edge Complexity Classifier
A lightweight ONNX model running locally in under 5ms classifies each incoming query by complexity. The classifier considers token count, reasoning depth, domain specificity, and required capabilities to decide whether edge or cloud inference is appropriate.
Offline-First Operation
When network connectivity is unavailable, edge agents continue functioning using local models. Requests and responses are queued locally. When connectivity returns, the system syncs state with the cloud, replays missed events, and resolves any conflicts.
Edge Context Cache
Frequently accessed context -- user preferences, recent conversation history, domain-specific knowledge -- is cached locally on the edge device. The cache is synchronized with cloud memory on a configurable interval, keeping local context fresh without constant network calls.
Model Compression Pipeline
Production cloud models are compressed for edge deployment through quantization (INT8/INT4), knowledge distillation, and pruning. The pipeline produces models that are 4-10x smaller while retaining 85-95% of the original quality for their target task domains.
Edge SDK
Native SDKs for React Native, Flutter, and IoT platforms. The SDK handles local model loading, complexity classification, cloud fallback, offline queuing, and context cache management. Integrate edge AI into mobile and embedded applications with a few lines of code.
Intelligent Routing
The routing layer decides where each query runs. It optimizes for speed, cost, and quality simultaneously, adapting in real time to network and device conditions.
Sub-5ms Classification
The ONNX complexity classifier runs in under 5 milliseconds on mobile hardware. There is no perceptible delay before routing begins. The classifier output includes a confidence score -- low-confidence classifications default to cloud routing for safety.
Adaptive Thresholds
Routing thresholds adjust based on conditions. When the device has strong connectivity, the threshold shifts toward cloud routing for higher quality. When connectivity is weak or metered, the threshold shifts toward edge inference to minimize data transfer.
Graceful Fallback
If an edge model produces a low-confidence result, the system automatically falls back to cloud inference and returns the cloud response instead. The edge result is logged for evaluation so the edge model can be improved over time.
Cost Reduction
By handling simple queries locally, edge routing eliminates cloud API costs for a significant portion of traffic. Typical deployments see 40-60% of queries handled entirely on-device, with proportional cost savings and lower latency.
Model Compression Pipeline
Production models are too large for edge devices. The compression pipeline produces optimized variants that retain task-specific quality at a fraction of the size.
Quantization
Reduce model precision from FP32 to INT8 or INT4. This shrinks model size by 4-8x and speeds up inference on edge hardware. Dynamic quantization preserves accuracy on critical layers while aggressively compressing less sensitive ones.
Knowledge Distillation
Train a smaller student model to replicate the behavior of a larger teacher model on your specific task domain. The student model captures the teacher's decision patterns at a fraction of the parameter count.
Structured Pruning
Remove entire attention heads and feed-forward layers that contribute least to output quality for your target tasks. Pruned models are smaller and faster while maintaining accuracy on the domains they were pruned for.
Continuous Improvement
Edge model performance is tracked against cloud model baselines. When quality drift is detected, the compression pipeline re-runs with updated training data and the new edge model is pushed to devices over-the-air. This keeps edge models aligned with cloud quality over time.
Edge SDK
Native SDKs that handle model loading, inference routing, offline queuing, and cloud sync. Integrate edge AI into your application with minimal code.
React Native
Drop-in integration for React Native applications. The SDK provides hooks for edge inference, offline state management, and cloud sync. Compatible with Expo and bare React Native workflows.
Flutter
Native Dart package for Flutter applications. Platform channels handle model loading and inference on iOS and Android. The SDK exposes a unified API for edge routing, caching, and sync across both platforms.
IoT & Embedded
Lightweight C/Rust SDK for embedded devices and IoT gateways. Runs ONNX models on ARM Cortex processors and RISC-V boards. Designed for constrained environments with minimal memory footprint and no OS dependencies.
MCP Modules
Bring AI to the Edge
Learn how to set up edge inference routing, compress models for your target devices, and integrate the Edge SDK into your mobile or IoT application.