Edge or cloud? Start with the operating conditions.

“Should this run at the edge?” is often asked before the operating requirement has been written down. The answer then becomes a debate about platforms rather than the system’s needs.

Edge processing means running some computation close to where the data is produced or used. Cloud processing moves some of that work to remote infrastructure. A hybrid design divides the work. Each choice changes the dependencies and the responsibilities of the operating team.

Follow the result to its use

Begin with the action the output supports. If a person uses a result later in the day, a brief loss of connectivity may be acceptable. If the result is needed within a tightly constrained process, delayed delivery could make it useless.

Measure the full time from input to usable result. For vision systems, this can include capture, decoding, preprocessing, inference, post-processing and delivery to the receiving application. A fast model does not necessarily create a fast system.

Write the requirement in terms of the task. “An operator needs a reliable alert before the item reaches the inspection point” is more useful than choosing a latency target with no connection to the process.

Decide what happens without a connection

Connectivity should be treated as an operating condition. What happens when the connection slows, drops or returns after an interruption?

An on-site system may continue processing but lose central reporting. A cloud-dependent system may need to queue work, stop a function or fall back to a local process. A hybrid system still needs a clear rule for where decisions are made during disruption.

Test recovery as well as outage. Queued data, repeated events and conflicting state can create work for the people managing the service.

Measure sustained performance

On embedded hardware, results depend on the workload and operating configuration. NVIDIA’s Jetson documentation describes power modes, monitoring and thermal throttling; those details matter when interpreting a benchmark. NVIDIA Jetson power and performance guide.

Record the relevant settings and test long enough to observe the intended workload. Include supporting tasks such as decoding several streams, storing events or updating a local interface.

A development board on an open desk and a deployed unit inside an enclosure are different test conditions. The comparison should reflect the proposed product, including cooling and power constraints.

Trace the information, not the label

Keeping inference on site may reduce the information sent elsewhere. It does not by itself establish that the entire system is private. Remote administration, analytics dashboards, logs and model updates can introduce other data paths.

For a cloud option, inspect the actual service, configuration and contract. For an edge option, inspect the surrounding application and support arrangements. In either case, draw the data flows rather than relying on the architecture label.

Include maintenance in the design

An estate of local devices needs an update, recovery and support process. A central service needs capacity management, monitoring and a response to provider changes. Both require someone to own the operating work.

Ask how faults will be identified, how a failed unit or service will be restored and how an update will be tested before wider release. These answers can change the preferred architecture even when the first prototype performs similarly.

Compare a small set of complete options

Choose two or three plausible designs and assess each against the same requirements: task performance, response time, outage behaviour, data flows, support and total operating cost.

Identify the assumption most likely to change the decision. It might be sustained throughput, the reliability of a site connection or the staff time needed to support devices. Test that assumption first.

The aim is to choose an architecture whose limitations are understood and whose responsibilities the organisation can support.

Next step: Use our AI evaluation plan to turn the main uncertainty into a test. Beeston Labs can help with architecture review and embedded-system evaluation.