Deployment model comparison

    Cloud LLM APIs vs On-Premises Open-Weight Models for Health Systems: Cost and Control Compared

    Neither deployment model wins on both axes. Cloud LLM APIs shift infrastructure to a provider that becomes a business associate under a BAA, and cost follows usage. Self-hosted open-weight models give the health system direct custody of weights and infrastructure, and cost follows capacity and staffing. Under both, the health system keeps its own risk analysis and HIPAA technical safeguard duties, and hosting location does not determine what the AI is allowed to do or what evidence exists of what it did.

    Evaluation criteria for cloud, on-premises or hybrid

    • Will the provider sign a BAA covering every service that touches ePHI, and what does it say about data use and retention?
    • Has the risk analysis and risk management plan been updated for the specific configuration (public, private, hybrid)?
    • For self-hosted models, who protects weights, code and training data, and how is that evidenced?
    • Which transparency artifacts will be required: SBOMs, SLAs, SSAE reports?
    • Where do prompts, outputs and tool-call records reside, and for how long?
    • Can the same policy be enforced, and the same evidence produced, for every model in the estate?

    Treat cost and control as separate axes

    Health system decisions about LLM deployment often collapse into one question: which is cheaper per token, or which keeps data closer. Both framings are incomplete. Cost describes how spend is generated and what drives it. Control describes who operates the infrastructure, who is accountable under HIPAA, and whether the organization can constrain and evidence what the model does.

    An open-weight model is one whose weights are publicly available. That makes it possible to host it yourself, fine-tune it without training from scratch, and move it between hosting providers. A cloud LLM API is operated by a provider that processes your requests on its own infrastructure. These are different operating models, but they raise the same governance questions about PHI handling, access, change management and evidence.

    The comparison below keeps the axes apart so that a gain on one is not mistaken for a gain on the other.

    Cloud LLM API vs self-hosted open-weight model

    Each row names a dimension an architect can assess. The descriptions state where the deployment model changes the answer and where it does not.

    • Cost driver: Cloud API: spend scales with usage volume. Self-hosted: spend scales with provisioned capacity, utilization and the staff who run it.
    • HIPAA status of the host: Cloud API: a provider handling ePHI is a business associate and a BAA is required. Self-hosted: the health system operates the environment, and any third-party host or infrastructure vendor handling ePHI needs the same business associate analysis.
    • Custody of model assets: Cloud API: the provider holds weights and code. Self-hosted: the health system must protect the integrity, availability and confidentiality of weights, code and any training data.
    • Residency, retention, version control: Cloud API: set by provider terms and configuration, so they must be confirmed contractually. Self-hosted: set by your own environment and change process.
    • Vendor due diligence: Cloud API: procurement review is the normal path. Self-hosted: a model can be adopted without procurement, so review must be deliberately required.
    • Policy on interactions and tool calls: Neither model provides this by default. It is enforced separately from where the model runs.

    What drives total cost in each model

    In a cloud API model, the visible cost is inference pricing, and it moves with usage. The less visible costs sit around it: engineering to build retrieval, prompt handling and integration, contract and security review, and the compliance work needed to document the configuration. Variable usage is easy to start with, but spend is harder to predict once a use case spreads across departments.

    In a self-hosted model, cost is driven by capacity. GPU infrastructure must be sized for peak demand, and utilization determines how much of that capacity does useful work. Inference and fine-tuning are resource-intensive and depend heavily on hardware, so sizing decisions carry real weight. Staffing is the other large component: MLOps for serving, scaling and upgrades, and security engineering to protect the environment, the weights and the access paths. Availability is a design requirement too, so redundancy adds to the footprint.

    A fair total cost comparison models both paths with your own volumes, your own utilization assumptions and fully loaded staffing. Comparing a per-token price to a GPU invoice leaves out the work around each. Compliance overhead applies in both cases and should not be treated as a differentiator.

    HIPAA obligations that apply regardless of where the model runs

    As of November 2026, HHS guidance states that a cloud service provider that creates, receives, maintains or transmits ePHI for a covered entity is a business associate, and a BAA is required. This holds even if the provider cannot view the ePHI because it is encrypted and the provider does not hold the key. The BAA is the mechanism for satisfactory assurances that the provider will safeguard the information.

    A BAA does not transfer the compliance burden. HHS says a covered entity may use cloud services of any configuration, public, hybrid or private, if it has a BAA, but the configuration can affect its risk analysis, the risk management plans of all parties and the BAA provisions. The covered entity should understand the provider's environment well enough to conduct its own risk analysis and set its own risk management policies. Business associates are directly liable for certain provisions, so liability is shared rather than handed to the vendor.

    The technical safeguards in 45 CFR 164.312 apply to a covered entity or business associate, including audit controls under 164.312(b). That applies whether the model runs on a provider's infrastructure or your own. NIST adds that existing risk controls, such as procurement due diligence, SBOM requests, SLAs and SSAE reports, apply to both proprietary and open-source generative AI. Self-hosting an open-weight model does not exempt it from third-party risk practices.

    Runtime policy enforcement and evidence in either architecture

    Hosting location answers who operates the infrastructure. It does not answer what a model, agent or tool call is allowed to do with PHI in a given interaction, or whether you can show an auditor what happened. A cloud API with a signed BAA can still receive more PHI than a use case needs. A self-hosted model can still call a tool it should not. Both need policy applied at the point of execution and evidence that outlasts the interaction.

    The problem is sharper in mixed deployments. Open-weight portability means a model may move between environments over time. If policy and evidence controls are tied to one host, they fragment as soon as workloads move or when a cloud API and a self-hosted model serve the same application. Enforcement and evidence are more durable when they are independent of the model's location.

    This is the layer Trussed addresses. Trussed enforces policy in the execution path across models, agents, tools and MCP servers, on individual interactions and tool calls: allow, redact, block, rate limit, or escalate for human approval. Policy is enforced at runtime, not reviewed after the fact, and the same policy plane covers cloud-hosted and self-hosted models, with 100+ models under one policy plane in production. Enforcement produces continuous audit-ready evidence mapped to HIPAA, including the audit controls in 164.312(b). Trussed is self-managed in the customer cloud, with configurable retention and data residency, and runs in production with healthcare and pharma organizations. It does not replace the cost, BAA or risk analysis work described above. It addresses what the model is permitted to do and what evidence exists.

    Two axes, evaluated separately

    Cost structure

    Usage-driven in a cloud API; capacity and staffing-driven when self-hosted.

    Control over infrastructure

    Provider-operated in the cloud; health-system-operated on premises.

    Obligations that do not move

    Risk analysis, risk management and technical safeguards stay with the health system.

    Behavior and evidence

    Policy on prompts, outputs and tool calls must be enforced and evidenced in either model.

    Apply one policy and one evidence standard across every model

    Review how runtime enforcement and audit-ready evidence work across cloud-hosted and self-hosted models in a regulated enterprise.

    Talk to an Expert