Quick answer
Choose the AI deployment boundary from the operating constraint, not from a preference for cloud or local servers. Fully offline suits restricted or disconnected environments but makes updates and support harder. On-premises fits customer-controlled infrastructure. Private cloud supports distributed operations inside the customer's tenancy. Edge fits low-latency or high-volume sensing. Many production systems use a governed hybrid of these patterns.
Key takeaways
- Offline describes connectivity; it does not automatically provide security or maintainability.
- Model and software licences must permit the intended local or edge use.
- Identity, logs, backup, updates and incident handling are required in every mode.
- Customer ownership of deliverables is separate from third-party model ownership.
Start with the data-flow diagram
Before choosing hardware or a model, draw the sources, processing steps, storage, users, outputs and external connections. Mark which information may leave the plant, region or company account. Include support access, telemetry, backups and updates; these paths are often missed when a system is described as private.
Four deployment patterns
Fully offline; air-gapped only if designed that way
For the agreed offline workflow, the model, application, retrieval and required supporting services have no external dependency. Air-gapping additionally requires a deliberately isolated physical and network design; offline operation alone does not establish that isolation. This can fit restricted sites and sensitive engineering knowledge. The design still needs a controlled process for software packages, model updates, vulnerability fixes, knowledge refresh, logs and recovery.
On-premises private network
The system runs in customer-controlled plant or data-centre infrastructure. It can integrate with internal identity and systems while restricting external access. Capacity planning, segmentation, observability and support responsibilities must be explicit.
Private cloud or VPC
The system runs inside accounts, regions, networks and storage controlled by the customer. This can help distributed teams and managed operations. Review every managed service, model endpoint, egress path, region and contractual data term.
Edge and hybrid
Inference runs near the equipment while governed events, summaries or approved data synchronize elsewhere. Edge suits vision and low-latency workflows, but introduces device fleet, buffering, rollout, monitoring and physical-security responsibilities.
Privacy is more than model location
A private architecture should answer who can use the system, which documents each role may retrieve, where prompts and outputs are logged, how data is deleted, who can diagnose incidents and how a compromised component is isolated. Operational technology has availability and safety constraints that differ from ordinary office IT.
Ownership and licence rights
Contracts can transfer customer-specific code, configuration, prompts, retrieval setup and evaluation assets. That does not transfer ownership of a third-party foundation model, connector, library or cloud service. Record each dependency, licence, version, update route and replacement option so the customer has an executable exit path.
Design the update path before go-live
An offline system that cannot receive tested security fixes or approved knowledge updates becomes fragile. Define package signing, scanning, staging, rollback and evidence retention. Also define what happens when the approved model reaches end of support or a licence changes.
The practical selection rule
Select the least complex architecture that meets data, safety, latency, resilience and support requirements. A hybrid design is often more honest than forcing every component into one label.
Decision table
| Option | Use when | Evidence needed | Watch for |
|---|---|---|---|
| Fully offline | External connectivity is prohibited or unreliable. | Approved update, support, backup and recovery process. | Assuming isolation removes patching and identity needs. |
| On-premises | Internal infrastructure and operations can support the workload. | Capacity, segmentation, identity and support ownership. | Hidden dependencies on external APIs or licences. |
| Private cloud | Distributed access and managed infrastructure are permitted. | Account, region, egress, key and service control map. | Treating every managed service as inside the same boundary. |
| Edge or hybrid | Latency, bandwidth or local continuity matters. | Device lifecycle, buffering, rollout and monitoring plan. | An unmanaged fleet of production devices. |
Frequently asked questions
Can generative AI run fully offline?
Yes, when the selected model, runtime, hardware and licences support local operation. The useful question is whether the complete application including retrieval, identity, logs, updates and support can operate within the same boundary.
Does on-premises mean the customer owns the model?
No. Deployment location and intellectual-property rights are separate. The contract and each third-party licence determine ownership and permitted use.
Can an edge AI system keep raw images inside the plant?
Yes. Inference can occur locally and only approved events or metadata can leave, if the architecture is designed that way. Retention, review, monitoring and device updates still need explicit rules.
Limitations of this guide
- The correct design depends on the selected model, software licences, hardware and customer security architecture.
- Air-gapping can reduce exposure paths but does not guarantee confidentiality, integrity, safety or availability.
- This guide is not a cybersecurity assessment or architecture approval.
Primary sources used in this guide
- Guide to Operational Technology Security, SP 800-82 Rev. 3
U.S. National Institute of Standards and Technology
Primary guidance on OT threats, vulnerabilities, architectures and security controls.
- OPC Unified Architecture Overview and Concepts
OPC Foundation
Primary specification context for interoperable information exchange across industrial systems.
- AI Risk Management Framework
U.S. National Institute of Standards and Technology
Primary framework for governing, mapping, measuring and managing AI risk across the lifecycle.