What is an Edge AI?

The motivation is a mix of latency, privacy, and connectivity constraints: a self-driving car can't wait for a round trip to a data center to decide whether to brake, and a hospital device may not be allowed to send patient data off-premises at all. Edge AI requires models small and efficient enough to run on constrained hardware, which is why techniques like quantization (reducing numeric precision), pruning (removing redundant weights), and distillation (training a small model to mimic a large one) are central to the field. The trade-off is capability: edge-sized models are generally far less capable than the largest cloud-hosted frontier models, so edge AI tends to be used for narrower, well-defined tasks — wake-word detection, object detection, on-device autocomplete — rather than open-ended reasoning. A growing pattern is hybrid deployment: a small edge model handles the common case instantly and locally, escalating to a larger cloud model only when the task exceeds the edge model's confidence or capability.