feat(cli): add --env flag to sandbox create/exec and fix env var passthrough

[russellb]

Summary

Add --env KEY=VALUE (repeatable) to openshell sandbox create and openshell sandbox exec, and fix a bug where user-specified environment variables were silently dropped inside sandbox SSH sessions.

Related Issue

N/A — discovered during investigation of env var injection support for sandboxes.

Changes

CLI (new feature):

• Add --env KEY=VALUE repeatable flag to sandbox create and sandbox exec subcommands
• Reuse existing parse_key_value_pairs() helper for consistent KEY=VALUE parsing
• Wire environment maps into SandboxSpec.environment and ExecSandboxRequest.environment protobuf fields (which were previously only accessible via the SDK/gRPC API)

Sandbox env passthrough (bug fix):

• The SSH handler's apply_child_env() calls env_clear() to strip supervisor secrets before spawning user shells, but this also wiped user-specified env vars set on the container by compute drivers. Introduced in 5fd4885 (2026-02-26, "feat(sandbox): VS Code Remote-SSH support with platform detection fix").
• Add OPENSHELL_USER_ENVIRONMENT sidecar env var: compute drivers serialize SandboxSpec.environment as JSON, the supervisor deserializes it at startup, and the SSH handler re-injects the values after env_clear()
• All four compute drivers updated (Docker, Podman, Kubernetes, VM)
• User-specified vars applied at lowest priority so proxy, TLS, and provider credentials still take precedence

Testing

• cargo test -p openshell-cli — 18 tests pass (including 2 new: env passthrough + invalid format rejection)
• cargo test across all modified crates — 1,252 tests pass
• cargo clippy clean across all modified crates
• End-to-end verified: rebuilt supervisor image, restarted Podman gateway, confirmed --env FOO=BAR appears in sandbox env output

Checklist

• Follows Conventional Commits
• No secrets or credentials committed

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

Label test:e2e applied, but pull-request/1730 is at {"messa while the PR head is fada35e. A maintainer needs to comment /ok to test fada35ee1f3aaddd6b57ee3a7e7afcddb337b638 to refresh the mirror. Once the mirror catches up, re-run Branch E2E Checks from the Actions tab.

[johntmyers]

gator-agent

/ok to test fada35e

[johntmyers]

gator-agent

PR Review Status

Validation: this PR is project-valid as a concentrated CLI/sandbox feature and bug fix for environment injection and passthrough. No exact duplicate was found.
Head SHA: fada35ee1f3aaddd6b57ee3a7e7afcddb337b638

Review findings:

• Blocking: crates/openshell-driver-vm/src/driver.rs builds required supervisor environment first, then extends with user env. That lets a create-time env value override supervisor-critical values such as OPENSHELL_ENDPOINT, sandbox identity, SSH socket path, TLS paths, or PATH. Docker/Kubernetes/Podman reapply required values after user env; VM should preserve the same invariant and add override-protection coverage.
• Blocking: OPENSHELL_USER_ENVIRONMENT is user-clobberable in Docker, Podman, and VM because the sidecar is inserted before environment.extend(user_env). A user-supplied --env OPENSHELL_USER_ENVIRONMENT=... can replace the sidecar that the sandbox supervisor later parses. Kubernetes avoids this by upserting the sidecar after user env. Please reserve this key or insert the sidecar after user env consistently, with tests.
• Needs resolution: sandbox exec --env is rendered as leading shell assignments by the gateway, so command-level env can override proxy/TLS/provider env for that command. That conflicts with the PR statement that proxy, TLS, and provider credentials still take precedence. Please either enforce reserved-key behavior for exec env or document/confirm the intended precedence.
• Needs resolution: Podman currently keeps template-over-spec environment precedence, while Docker, Kubernetes, and VM use spec-over-template. Please make precedence consistent or explain why Podman should differ, with a conflict test.
• Existing maintainer question remains open on whether user env should override the baseline child env block (HOME, USER, LOGNAME, SHELL, PATH, TERM).

Docs: missing for direct UX changes. The new --env flags and sandbox env behavior should be documented in the Fern docs under docs/; the existing sandbox docs page appears sufficient, so docs/index.yml likely does not need navigation changes. A maintainer-authored no-docs rationale would also satisfy this gate.

Tests/E2E: test:e2e is required because this touches sandbox lifecycle, supervisor/gateway exec behavior, and compute driver environment behavior. I applied test:e2e and posted /ok to test for the current head SHA.

Next state: gator:in-review

[johntmyers]

/ok to test fada35e

[johntmyers]

gator-agent

CI Follow-Up

Branch Checks failed on both Rust jobs at rust:format:check. The failure is PR-caused: the new USER_ENVIRONMENT read in crates/openshell-sandbox/src/lib.rs needs rustfmt.

Branch E2E is still running, Helm Lint passed, and the PR remains in gator:in-review because author changes are required for the review findings, docs, and formatting failure.

[johntmyers]

gator-agent

E2E Result

Branch E2E completed with one failing job: e2e / E2E (rust-podman-rootless). The failed test was websocket_text_placeholder_is_rewritten_in_sandbox; sandbox creation failed during SSH with RuntimeError("websocket upgrade failed"). Docker, regular Podman, Python, and Kubernetes E2E jobs passed.

This does not look directly caused by the env-var changes from the current evidence, but the required E2E status is red. The PR remains in gator:in-review; author changes are still needed for the review findings, docs, and rustfmt failure before pipeline monitoring can advance.

[russellb]

AI coding agent response — addressing review feedback from @mrunalp and gator-agent.

Fixes pushed in f190fa3

Blocking: VM driver override-protection

Restructured build_guest_environment in the VM driver to apply user env before required driver vars (OPENSHELL_ENDPOINT, SANDBOX_ID, PATH, etc.), matching the pattern already used by Docker, Podman, and Kubernetes. User-specified values can no longer override security-critical driver vars.

Blocking: OPENSHELL_USER_ENVIRONMENT clobberable

Three-layer defense:

1. CLI: new parse_env_pairs() rejects any --env key starting with OPENSHELL_ with a clear error message.
2. Drivers: OPENSHELL_USER_ENVIRONMENT sidecar is now inserted after environment.extend(user_env) in all four drivers, so even if a key slipped through, the sidecar overwrites it.
3. SSH handler: apply_child_env() filters out OPENSHELL_* keys from user_environment before injecting into child processes.

Needs resolution: sandbox exec --env precedence

Exec env vars are rendered as leading shell assignments by the gateway — this is intentional per-command override behavior, consistent with standard Unix semantics. The user already has shell access and could run export http_proxy=... manually. Updated the PR description to document the actual precedence rather than claiming proxy/TLS/provider always wins on exec. Documented in the new docs section.

Needs resolution: Podman template/spec env precedence

Fixed. Template env is now applied first, then spec env overwrites — matching Docker, Kubernetes, and VM behavior (spec is user-specified and should win over image defaults).

Inline comment (@mrunalp): user env overriding baseline vars

Decision: user env can override baseline child env vars (HOME, USER, SHELL, PATH, TERM) but OPENSHELL_* keys are always filtered out. This gives users flexibility (e.g., custom PATH, different HOME) while protecting internal sandbox machinery.

CI: rustfmt failure

Fixed formatting in lib.rs, container.rs, and driver.rs.

Docs

Added "Set Environment Variables" section to docs/sandboxes/manage-sandboxes.mdx covering --env for both sandbox create and sandbox exec, per-command override semantics, and the OPENSHELL_* reservation. Also added --env to the exec flags table.

Tests

Added sandbox_create_env_rejects_reserved_prefix integration test verifying the OPENSHELL_* key rejection.

[johntmyers]

/ok to test f190fa3

[johntmyers]

gator-agent

PR Review Status

Validation: this PR remains project-valid as a concentrated CLI/sandbox feature and bug fix for environment injection and passthrough.
Head SHA: f190fa3d1b2294ead2d9b5d1fbbca38ed20ed2db

Review findings:

• Blocking: SandboxSpec.environment and template environment can still carry reserved OPENSHELL_* keys through SDK/gRPC paths because only the CLI rejects them. In Docker, user env is applied before the final required driver env block, but it is still extended into the runtime environment before the supervisor starts, so a non-CLI caller can affect supervisor-visible OPENSHELL_* values before crates/openshell-sandbox/src/ssh.rs filters child env. Please enforce the reserved prefix at the API/model or driver boundary and add driver/API-level coverage, especially for Docker.
• Warning: parse_key_value_pairs still accepts an empty key such as --env =value; please reject empty env keys in parse_env_pairs.
• Prior feedback mostly resolved: VM required env precedence, OPENSHELL_USER_ENVIRONMENT sidecar clobbering, Podman template/spec precedence, and docs coverage look addressed in the updated diff.

Docs: updated under docs/sandboxes/manage-sandboxes.mdx; no navigation update appears necessary.

Checks/E2E: test:e2e remains required and has passed at this head. Branch Checks, Core E2E, Helm Lint, and the non-required GPU E2E gate are all green.

Process blocker: GitHub currently reports mergeable_state: dirty, so the branch appears to need a merge-conflict update before it can proceed.

Next state: gator:blocked

[johntmyers]

/ok to test cf1baf9

[johntmyers]

gator-agent

PR Review Status

Validation: this PR remains project-valid as a concentrated CLI/sandbox feature and bug fix for environment injection and passthrough.
Head SHA: cf1baf98ef940037ed01b9cc474931b80fe219a7

Review findings:

• Needs resolution: spec.environment and spec.template.environment are now size-checked and protected against reserved OPENSHELL_* keys, but they still do not use the same environment-name and control-character validation as exec env. Create-time env now reaches driver runtime env and is replayed into sandbox child processes, so invalid names or control characters should fail fast at the gRPC boundary instead of surfacing as driver-specific pod/container/spawn failures. Please validate create-time env keys with the same ^[A-Za-z_][A-Za-z0-9_]*$ rule, reject control characters in values, and add coverage for both spec.environment and spec.template.environment. CLI coverage for invalid env names such as --env 1BAD=value / BAD-NAME=value would also close the user-facing path.
• Suggestion: docs/sandboxes/manage-sandboxes.mdx says keys starting with OPENSHELL_ are rejected by the CLI. Since the gateway now rejects SDK/gRPC callers too, consider wording this as "OpenShell rejects..." so the docs match the current boundary.
• Prior blocking feedback appears resolved: server-side reserved-prefix validation covers create/template env and unary/interactive exec; driver-required env precedence is preserved; OPENSHELL_USER_ENVIRONMENT sidecar clobbering is addressed; Podman template/spec precedence is consistent; docs were added under docs/.

Docs: updated under docs/sandboxes/manage-sandboxes.mdx; no navigation update appears necessary.

Checks/E2E: test:e2e remains required. Current required checks are waiting for a copy-pr mirror refresh, so I am posting /ok to test for this head SHA.

Next state: gator:in-review

[johntmyers]

gator-agent

CI Follow-Up

Required E2E completed for head cf1baf98ef940037ed01b9cc474931b80fe219a7 and OpenShell / E2E is red. Branch Checks and Helm Lint are green; Docker, Podman, Podman rootless, and Kubernetes E2E jobs passed.

The failing job is e2e / E2E (python). The failure looks PR-caused: many Python E2E tests fail during sandbox execution with StatusCode.INVALID_ARGUMENT because the new server-side reserved-prefix validation rejects OPENSHELL_PYFUNC_B64 in ExecSandboxRequest.environment.

Please adjust the reserved-env validation so user-supplied OPENSHELL_* keys remain blocked without breaking OpenShell-owned/internal exec environment used by the Python sandbox helper, and add coverage for that path. The earlier review feedback about create-time env name/control-character validation also remains open.

Next state: gator:in-review

[johntmyers]

@gator-comment-pr1730.md

[russellb]

@gator-comment-pr1730.md

@johntmyers looks like a gator bug :)

[johntmyers]

/ok to test 2f226e8

[johntmyers]

gator-agent

Re-check After Author Update

I re-evaluated latest head 2f226e86594b2acd30a904c35ed4389c3b907a2e after @russellb's 2026-06-08 comment about the stray @gator-comment-pr1730.md post and the latest pushed changes.

Disposition: partially resolved. The stray filename-only comment was a gator posting mistake; no author action was required for that comment.

Remaining items:

• Blocking: ExecSandboxRequest.environment still rejects every OPENSHELL_* key in crates/openshell-server/src/grpc/validation.rs, so the Python SDK's exec_python() path still fails when it sends the OpenShell-owned OPENSHELL_PYFUNC_B64 helper value from python/openshell/sandbox.py. Please keep create-time env strict, but split exec validation so it allows only the known internal exec helper key while still rejecting user-clobberable runtime keys such as OPENSHELL_SANDBOX_ID, and add coverage for both cases.
• Warning: create-time env validation now rejects invalid names and control characters for both spec.environment and spec.template.environment, but the tests only cover control characters for spec.environment. Please add the symmetric spec.template.environment control-character test requested in the prior review.
• Consideration: the serialized OPENSHELL_USER_ENVIRONMENT sidecar can aggregate the full env map into one runtime env var. A total serialized-size limit at create-time validation would avoid driver/runtime-specific startup failures for very large maps.

Resolved from prior feedback: VM required-env precedence, OPENSHELL_USER_ENVIRONMENT sidecar clobbering, Podman template/spec precedence, create-time env name validation, spec.environment control-character validation, and docs coverage look addressed. The docs update under docs/sandboxes/manage-sandboxes.mdx is sufficient; no navigation update appears necessary.

Checks/E2E: test:e2e remains required. Current required checks are waiting on the copy-pr mirror for this head, so I posted /ok to test 2f226e86594b2acd30a904c35ed4389c3b907a2e.

Next state: gator:in-review

[johntmyers]

gator-agent

CI Follow-Up

Required E2E completed for head 2f226e86594b2acd30a904c35ed4389c3b907a2e and OpenShell / E2E is red. Branch Checks, Helm Lint, Docker E2E, Podman E2E, Podman rootless E2E, and Kubernetes smoke E2E passed.

The failing job is e2e / E2E (python), with the failure in the Run tests step. I could not retrieve the full failed log because the Actions log storage URL returned Forbidden, but the failed job matches the unresolved Python SDK/internal exec-env blocker already called out in the latest review feedback.

The PR remains in gator:in-review; author changes are still needed for the unresolved exec environment validation feedback and the red required E2E gate.

[johntmyers]

gator-agent

CI Follow-Up Correction

I re-checked the failed E2E logs for head 2f226e86594b2acd30a904c35ed4389c3b907a2e; the Actions log is now retrievable.

Correction to my previous CI note: I do not see the earlier INVALID_ARGUMENT / OPENSHELL_PYFUNC_B64 signature in this run's failed log. The required OpenShell / E2E gate is still red because e2e / E2E (python) failed broadly, with repeated supervisor-session stream errors reported as h2 protocol error: error reading a body from connection while sandbox exec requests were running.

The PR remains in gator:in-review. Author follow-up is still needed for the red required E2E gate and the unresolved review feedback from the latest review comment:

• Split exec-env validation so OpenShell-owned internal exec helper keys do not break SDK paths while user-clobberable OPENSHELL_* keys remain blocked.
• Add the symmetric spec.template.environment control-character validation test.

[johntmyers]

gator-agent

PR Review Status

Validation: this PR remains project-valid as a concentrated CLI/sandbox feature and bug fix for environment injection and passthrough.
Head SHA: 2f226e86594b2acd30a904c35ed4389c3b907a2e

Independent review result: confirmed the current head still has blocking feedback.

Review findings:

• Blocking: ExecSandboxRequest.environment still rejects every OPENSHELL_* key in crates/openshell-server/src/grpc/validation.rs, while the Python SDK exec_python() path in python/openshell/sandbox.py still sends OPENSHELL_PYFUNC_B64. This breaks that SDK path and matches the red required Python E2E gate. Prefer moving the Python callable payload out of env, for example through stdin, rather than creating a broad reserved-prefix exception. If an exception is used instead, it should be narrowly scoped and covered against user-clobberable runtime keys such as OPENSHELL_SANDBOX_ID.
• Warning: create-time env validation now applies to both spec.environment and spec.template.environment, but the newline/control-character regression test only covers spec.environment. Please add the symmetric spec.template.environment test.
• Warning: the merged OPENSHELL_USER_ENVIRONMENT sidecar can aggregate up to 128 create-time values of 8192 bytes each into one runtime env var. A create-time serialized-size bound would avoid later driver/runtime startup failures for very large maps.
• Suggestion: the docs example uses API_KEY=sk-test for --env; consider using a non-secret-looking example or noting that sensitive values should use provider/credential flows where possible.

Docs: updated under docs/sandboxes/manage-sandboxes.mdx; no navigation update appears necessary.

Checks/E2E: test:e2e remains required. Current required OpenShell / E2E is red on e2e / E2E (python), while Branch Checks, Helm Lint, Docker E2E, Podman E2E, Podman rootless E2E, and Kubernetes smoke E2E are green.

Next state: gator:in-review

[johntmyers]

gator-agent

Re-check After Author Update

I re-evaluated latest head 69c18c796130cdf5d56548f59a3cd4cc019a3faa after @russellb pushed the exec helper validation and environment size-limit update on 2026-06-08.

Disposition: resolved for the blocking review feedback. The Python SDK exec_python() helper key is now allowed narrowly, runtime-clobberable OPENSHELL_* keys remain rejected, the template control-character coverage was added, and a create-time environment size bound was added.

Remaining items:

• No blocking review findings remain.
• Non-blocking warning: the new size limit is computed from raw key/value bytes per map, while drivers merge template plus spec env and serialize the merged map as JSON into one OPENSHELL_USER_ENVIRONMENT value. A follow-up could validate the merged serialized payload directly to avoid driver/runtime-specific startup failures for edge-case large maps.
• Non-blocking docs suggestion: the docs example uses API_KEY=sk-test; a less secret-looking example or a short pointer to provider/credential flows would be clearer.

Docs: updated under docs/sandboxes/manage-sandboxes.mdx; no navigation update appears necessary.

Checks/E2E: test:e2e remains required. Current required checks are waiting on the copy-pr mirror for this head, so I am posting /ok to test 69c18c796130cdf5d56548f59a3cd4cc019a3faa.

Next state: gator:watch-pipeline

[johntmyers]

/ok to test 69c18c7

[johntmyers]

gator-agent

Maintainer Approval Needed

Gator validation and PR monitoring are complete.

Validation: this PR is project-valid as a concentrated CLI/sandbox feature and bug fix for sandbox environment injection and passthrough.
Review: no blocking review findings remain at head 69c18c796130cdf5d56548f59a3cd4cc019a3faa. Prior feedback on reserved OPENSHELL_* handling, Python SDK internal exec env, create/template env validation, sidecar clobbering, driver precedence, and docs coverage has been resolved. Remaining notes are non-blocking follow-ups.
Docs: updated under docs/sandboxes/manage-sandboxes.mdx; no navigation update appears necessary.
Checks: OpenShell / Branch Checks, OpenShell / Helm Lint, and OpenShell / E2E are green for this head.
E2E: test:e2e is applied and the required core E2E gate passed. GPU E2E is not required for this change.

Human maintainer approval or merge decision is now required.