Video System Architecture in the Edge-First Era

In the context of high-load video surveillance systems, I believe the greatest potential is unlocked precisely through synergy: the performance and autonomy of Edge-First solutions combined with the global availability of the cloud (SaaS) and centralized device fleet management.

A striking confirmation of this synergy's success in the market is AWS IoT Greengrass. Amazon effectively split the system into two parts: it provided the local edge runtime as a free open-source project while retaining a paid cloud subscription for centralized control and fleet management. The pioneer of cloud solutions became the leading pioneer and ambassador of the Edge-First approach.

But Greengrass is not an isolated example. Azure IoT Edge, Eclipse ioFog, EdgeX Foundry, and even Home Assistant all build edge node autonomy on the same proven combination: a local MQTT broker + SQLite for event buffering. These platforms are where you can study battle-tested patterns for communication, autonomy, and context synchronization after connectivity loss — but more on that in the articles on this site.

For a clearer illustration of these ideas, take a look at the block diagram of an On-Edge video system:

Consider a telling scenario — a critical loss of connectivity, which in a Cloud-Only architecture leads to complete system degradation. Under the classic cloud model, archive recording stops and video analytics become unavailable. In an Edge-First architecture, the peripheral node retains full autonomy: the video stream continues to be recorded to the local ring buffer archive, the local AI engine keeps running inference as normal, and alert events are cached in a local queue. Once connectivity is restored, the node automatically synchronizes accumulated metadata and critical video clips with the cloud platform. The system operates without downtime and guarantees processing of 100% of events.

I believe this model will become the architectural standard for high-load distributed systems where autonomy and privacy (Privacy-First) are paramount. And it is in video surveillance that this approach will reach its fullest potential.

I see Rust and Media over QUIC (MoQ) as the technological foundation. Rust's speed, memory control, and concurrency allow you to squeeze maximum performance out of local edge hardware (including onboard NPUs). MoQ was designed with the key limitations of WebRTC and SRT in mind and solves the real-time streaming challenge with minimal latency, combining a relatively simple API with easy scalability. This stack is an excellent fit for Edge-First — it makes on-edge video data processing fast and resilient.

Why not WebRTC, not SRT, not plain RTSP? WebRTC is complex, tailored for P2P calls, and scales poorly through relays. SRT is great for broadcast and point-to-point but lacks native multiplexing. RTSP is a legacy of the LAN era: it does not work through NAT without hacks and cannot encrypt data on its own.

MoQ is built on top of QUIC and solves these problems architecturally: it provides 0-RTT reconnection (no extra handshakes), stream multiplexing within a single connection, and — critically for a hybrid Edge-First + Cloud architecture — a relay-native approach where the cloud relay simply forwards MoQ packets without transcoding the video.

This is exactly what is needed: the edge streams, the relay scales, and the viewer gets sub-100 ms latency within the Edge/Relay region of presence without the need to spin up heavy media servers for transcoding. Although the MoQ specification is currently (2026) in its final stages at the IETF (with active participation from Meta, Cisco, and Akamai), the protocol is already de facto recognized as the future of ultra-low-latency streaming.

A common question arises: "How does an Edge node differ from a standard NVR?" The difference is fundamental. A traditional NVR is a passive storage device: its functions are limited to cyclic recording of the video stream to disk and serving it on demand. An Edge node is a full-fledged compute platform. It performs local real-time AI inference, programmatically filters false alarms, streams video over MoQ with sub-100 ms latency, and synchronizes context with the cloud ecosystem. An NVR is just a local archive. An Edge node is a peripheral decision-making center.

Local Intelligence: AI as an Architectural Necessity

But if the Edge node is the center of decision-making, a natural question arises: who exactly makes these decisions and on what basis? A traditional NVR simply detects motion and writes a buffer to disk. An Edge node of the new generation must do something fundamentally different—turn a continuous, semantically "empty" video stream into a structured event stream, where each event carries weight, context, and priority. This is where local AI inference enters the scene not as a marketing add-on, but as an architectural necessity, without which the very idea of Edge-First loses half its meaning.

Consider this: a single camera generates 86,400 seconds of video per day. On average, 90–95% of that is static background, swaying trees, and changing lighting. In a Cloud-Only model, you pay to transfer and analyze all these seconds in the cloud, burning GPU resources on processing emptiness. In the Edge-First paradigm, a local AI engine—the same YOLOv8/v11 running via ONNX Runtime on your Mini-PC’s NPU—acts as an intelligent first-level filter. In real time, it separates signal from noise: detects objects, tracks trajectories, classifies intrusion zones, and what lands on the ring-buffer disk is no longer a raw stream, but an indexed sequence of events with metadata, bounding boxes, and confidence scores. The cloud receives not terabytes of "silence," but compact event segments—and this changes the unit economics of the entire system.

For a long time, split inference was considered an unavoidable architectural compromise—where primary detection was executed on the periphery, while "heavy" secondary analysis was delegated to the data center. However, with the emergence of ultra-efficient next-generation architectures (such as YOLOv26 and YOLOEv26), optimized for hardware execution on modern NPUs and edge accelerators, split inference is losing its relevance and turning into an anti-pattern due to its complexity. Interestingly, the release of YOLOv26 was framed under the slogan: Ultralytics YOLO26: The new standard for edge-first vision AI.

The Edge node no longer needs cloud "crutches": it is capable of performing a full On-Edge visual analysis cycle with latency under 100 ms—from high-precision tracking, segmentation, and LPR (license plate recognition) to spatial classification of complex behavioral patterns. The entire compute load is fully contained within the local Data Plane. What goes to the cloud is neither raw frames nor intermediate tensors, but exclusively finished semantic events and aggregated metadata. The cloud Control Plane is completely freed from video inference and focuses on global coordination, Fleet Management, and long-term cross-site analytics.

This refusal to send frames to the cloud makes the principle of privacy by default (Privacy by Architecture) an absolute reality rather than a compromise. In the context of regulatory requirements like 152-FZ (or GDPR in Europe), the classic model of transmitting video streams with faces and license plates to third-party data centers carries colossal regulatory risks. In a fully local Edge-First architecture, all analysis and de-identification tasks (face blurring, silhouetting, plate anonymization) run on site, before a single byte leaves the physical perimeter of the facility. The position for regulators is crystal clear: all personal and biometric data is processed in isolation on a trusted local node. The system architecture guarantees that confidential information never leaves the protected internal perimeter.

If we look slightly beyond the horizon—and in the video surveillance industry, the horizon is now shrinking to two or three years—the next qualitative leap will occur at the moment of transition from detection to understanding. Today, edge AI draws bounding boxes and labels them: "person", "car", "dog". Tomorrow, edge nodes with sufficiently powerful NPUs will host compact VLM models (such as YOLOEv26) capable of describing a scene in coherent text: "a courier in a yellow jacket left a package at the side entrance and walked toward the parking lot." Combined with local generation of vector embeddings (CLIP, SigLIP) and their synchronization via MoQ into a cloud vector database, this opens up the possibility of semantic search across the video archive. An operator will no longer spend hours scrubbing a timeline in search of an incident—they will ask the system in natural language: "show all moments from the past week when someone approached the perimeter after midnight," and receive an answer in seconds. At that point, VSaaS will cease to be a "remote-access archive" and become a situational awareness system—and this will happen on the edge, not in the cloud, because only a local node has direct access to the full video stream with sub-second latency and zero bandwidth constraints.

Finally, there is one more argument in favor of laying the AI foundation right now, rather than "when a clear business case emerges." I am not claiming that tomorrow every warehouse will feature autonomous AI agents controlling turnstiles and PTZ cameras without human intervention—video surveillance, with its conservative adoption cycles and strict reliability requirements, still has years to go before that. I am talking about the direction of evolution, which is already visible clearly enough to be accounted for in architectural decisions today.

Today, typical AI in VSaaS is a model that answers the question "what is happening in the frame?" and returns a bounding box with a confidence score. This is a model-as-a-tool: a detector, a classifier, a counter. The next logical step is systems that do not merely record an event, but track its evolution over time and respond: detected an unauthorized intrusion into the perimeter—panned the PTZ camera toward the intruder, triggered the siren, generated an incident report, and escalated to the on-duty operator with priority. This is no longer a single inference call, but a closed-loop decision-making cycle (Closed-Loop Decision Making)—even if initially under human supervision and with simple logic, it is architecturally a fundamentally different pattern. And if your Data Plane is designed as a one-way pipeline "frame → model → metadata → storage," embedding such a feedback loop a couple of years down the line could prove difficult. By understanding the direction of technology evolution, we can immediately lay down an extensible event bus, a pluggable enrichment pipeline (semantic enrichment), and an abstraction over actuators (PTZ drives, turnstiles, I/O relays, sirens, etc.), so that closed decision loops fit in natively as just another consumer in an existing event stream.

That is why I am convinced: an Edge-First video system you design today must be ready not only for AI inference, but also for closed loops of autonomous response at the edge (the MAPER approach - Extending MAPE-K with LLM-Based Reasoning)—even if in the first version of the product you simply write an archive and send alerts. Architectural debt in distributed systems is paid with interest, and the earlier you lay down the right abstractions, the cheaper the evolution will be.