Routing an Existing OpenAI SDK Application Through a Governance Proxy by Changing the Base URL
To route an existing OpenAI SDK application through a governance proxy, point the SDK client at the proxy's OpenAI-compatible endpoint using the client's base URL setting or its environment variable equivalent, and give the application a proxy-issued credential in place of the provider key. This needs little or no application code change. It does not by itself prevent direct calls, hardcoded endpoints, or unmanaged keys, so network egress controls and credential isolation must accompany it.
What a base URL change does and does not do
Changing the base URL sends the application's OpenAI SDK traffic to the governance proxy's OpenAI-compatible endpoint. Pairing it with a proxy-issued credential in place of the provider key lets the proxy identify the caller. Existing code keeps using the OpenAI SDK, and the change needs little or no application code change.
Request path with a governance proxy
- ApplicationExisting code using the OpenAI SDK, with only the base URL and credential changed.
- Governance proxyAuthenticates the caller, applies policy, attributes the request, and records it.
- Upstream model APIReceives the request using a provider credential held by the proxy.
- Audit recordCaptures who called what, which policy applied, and what was returned or blocked.
Configuration methods in the Python and Node SDKs
In both the Python and Node SDKs there are two ways to make the change, and both leave the rest of the application code as it is.
Client base URL setting
Set the client's base URL to the proxy's OpenAI-compatible endpoint, and supply the proxy-issued credential in place of the provider key.
Environment variable equivalent
Use the client's environment variable equivalent to supply the same endpoint, so the change can be made in deployment configuration rather than in code.
Compatibility requirements for the proxy
For the SDK to behave as it would against the upstream API, the proxy must meet the following requirements.
| Area | Requirement |
|---|---|
| API surfaces | Every endpoint the application uses is supported, for example chat completions, responses, and embeddings, with matching paths and payload shapes. |
| Streaming | Streamed responses are passed through incrementally and not buffered, and policy checks do not break the stream format the SDK expects. |
| Tool and function calls | Tool definitions and tool call results pass through intact, so the SDK parses them as it would from the upstream API. |
| Errors | Upstream status codes and error bodies are returned in a form the SDK can interpret, and proxy-originated denials use a clear, documented error. |
| Retries and timeouts | Proxy timeouts exceed or align with SDK timeouts, and retry behavior does not duplicate requests or double-count usage. |
| Headers and attribution | Required headers are forwarded, and the proxy can identify the calling application, team, or user from its credential. |
Validation and rollout sequence
- Confirm the proxy meets each compatibility requirement above for every endpoint the application uses.
- Confirm requests appear in the audit record with the calling application, team, or user, the policy that applied, and what was returned or blocked.
- Confirm that direct calls, hardcoded endpoints, and unmanaged keys are closed off by network egress controls and credential isolation.
Closing the bypass paths
A base URL change only governs traffic that follows it. Three paths remain open until they are addressed separately:
- Direct calls: use network egress controls so the application cannot reach the provider except through the proxy.
- Hardcoded endpoints: code that sets its own endpoint is not affected by the configured base URL, so egress controls matter here as well.
- Unmanaged keys: use credential isolation so provider keys are held by the proxy and the application uses a proxy-issued credential.
Governance at the proxy layer versus the application
With a proxy in the path, the application changes only its base URL and credential. Authentication, policy, attribution, and recording happen at the proxy, so they apply consistently without each application implementing them.
The proxy holds the provider credential and sends requests upstream, while the audit record captures who called what, which policy applied, and what was returned or blocked.
Plan your proxy rollout
Talk through your SDK inventory, enforcement requirements, and egress design with a runtime governance specialist.
Talk to an Expert