Skip to content

feat: extract openshell-policy-schema as a thin crate with no proto dependency#1775

Open
feloy wants to merge 17 commits into
NVIDIA:mainfrom
feloy:crate-policy-schema-2
Open

feat: extract openshell-policy-schema as a thin crate with no proto dependency#1775
feloy wants to merge 17 commits into
NVIDIA:mainfrom
feloy:crate-policy-schema-2

Conversation

@feloy

@feloy feloy commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Summary

openshell-policy is consumed by external projects that only need YAML parsing and serialization, but it transitively pulls in tonic, tonic-build, and protobuf-src (which compiles protobuf from C++ source). This makes it unusable on Windows without MSYS2.

This PR extracts the YAML serde types and all pure-Rust logic into a new crates/openshell-policy-schema crate whose only dependencies are serde, serde_yml, serde_json, and miette — no proto, no gRPC. openshell-policy is updated to delegate all YAML work to the new crate and add only the proto conversion layer on top. Its public API and all existing behavior are unchanged.

Related Issue

Fixes #1608

Changes

Each commit is intentionally minimal and independently reviewable.

New crate — additive only, no changes to existing code:

Commit What to look for
feat(policy-schema): create openshell-policy-schema crate skeleton Just Cargo.toml + empty src/lib.rs. Verify deps: serde, serde_yml, serde_json, miette — nothing else.
feat(policy-schema): add YAML serde types (verbatim, names unchanged) Verbatim copy of the private serde structs/enums from openshell-policy/src/lib.rs lines 37–229. Only change: structpub struct, enumpub enum. No logic.
feat(policy-schema): add constants and utility functions Verbatim copy from openshell-policy/src/lib.rs. MAX_FILESYSTEM_PATHS, MAX_PATH_LENGTH, and truncate_for_display promoted from private to pub(crate).
feat(policy-schema): add PolicyViolation enum and Display impl Verbatim copy from openshell-policy/src/lib.rs lines 681–735. No changes.
feat(policy-schema): add public API functions Function bodies adapted from openshell-policy/src/lib.rs: same logic, operating on schema types instead of proto types (policy.filesystem_policy instead of policy.filesystem, ProcessDef instead of ProcessPolicy, etc.). Default added to ProcessDef — necessary because proto-generated types auto-derive it, but our serde struct does not.
feat(policy-schema): add tests Ported from openshell-policy/src/lib.rs test suite. See commit message for the full list of adaptations (enum matching instead of proto field access, no port normalization, etc.).
style(policy-schema): apply rustfmt formatting Pure formatting — no logic change.

Wiring openshell-policy to the schema crate:

Commit What to look for
feat(policy): add openshell-policy-schema dependency One line added to openshell-policy/Cargo.toml.
refactor(policy): import YAML serde types from openshell-policy-schema Removes ~200 lines from openshell-policy/src/lib.rs (the private type definitions) and replaces them with a single use openshell_policy_schema::{...} import. to_proto, from_proto, and all public functions are unchanged.
refactor(policy): delegate YAML functions to openshell-policy-schema parse_sandbox_policy, serialize_sandbox_policy, sandbox_policy_to_json_value, serialize_sandbox_policy_json, load_sandbox_policy, and restrictive_default_policy now call the schema crate and apply to_proto / from_proto. No behaviour change.
refactor(policy): delegate normalize_path to openshell-policy-schema One-liner delegation. validate_sandbox_policy and its helpers (truncate_for_display, MAX_FILESYSTEM_PATHS, MAX_PATH_LENGTH) intentionally stay: from_proto elides the process section when both run_as_user and run_as_group are empty strings (serialization optimization), so delegating via from_proto would silently pass policies that should be rejected.
refactor(policy): re-export PolicyViolation from schema, drop serde/serde_yml deps Removes the local PolicyViolation definition and re-exports it from openshell-policy-schema. Callers importing openshell_policy::PolicyViolation are unaffected. serde and serde_yml are removed from openshell-policy's direct dependencies.
refactor(policy): re-export path constants from openshell-policy-schema CONTAINER_POLICY_PATH and LEGACY_CONTAINER_POLICY_PATH are re-exported from the schema crate instead of being defined locally.
fix(policy): move HashMap import into test module HashMap is only used in tests; the module-level import was unused in non-test builds.
docs(policy): update module doc comment to reflect schema crate split The doc comment no longer claims the serde types live here.
docs(agents): add openshell-policy-schema to architecture table New crate registered in the AGENTS.md component table.

Testing

  • mise run pre-commit passes
  • Unit tests added/updated
  • E2E tests added/updated (if applicable)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

@feloy feloy requested review from a team, derekwaynecarr, maxamillion and mrunalp as code owners June 5, 2026 11:45
@copy-pr-bot

copy-pr-bot Bot commented Jun 5, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@feloy feloy force-pushed the crate-policy-schema-2 branch from bb47e36 to 5c67177 Compare June 5, 2026 11:46
@pimlock

pimlock commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator

/ok to test 5c67177

@pimlock pimlock added the test:e2e Requires end-to-end coverage label Jun 5, 2026
@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

Label test:e2e applied for 5c67177. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

Comment on lines +471 to +563
pub fn validate_policy(policy: &PolicyFile) -> std::result::Result<(), Vec<PolicyViolation>> {
let mut violations = Vec::new();

// Check process identity — must be "sandbox".
// `ensure_sandbox_process_identity` should be called before this to
// fill in defaults; anything other than "sandbox" is rejected.
if let Some(ref process) = policy.process {
if process.run_as_user != "sandbox" {
violations.push(PolicyViolation::InvalidProcessIdentity {
field: "run_as_user",
value: process.run_as_user.clone(),
});
}
if process.run_as_group != "sandbox" {
violations.push(PolicyViolation::InvalidProcessIdentity {
field: "run_as_group",
value: process.run_as_group.clone(),
});
}
}

// Check filesystem paths
if let Some(ref fs) = policy.filesystem_policy {
let total_paths = fs.read_only.len() + fs.read_write.len();
if total_paths > MAX_FILESYSTEM_PATHS {
violations.push(PolicyViolation::TooManyPaths { count: total_paths });
}

for path_str in fs.read_only.iter().chain(fs.read_write.iter()) {
if path_str.len() > MAX_PATH_LENGTH {
violations.push(PolicyViolation::FieldTooLong {
path: truncate_for_display(path_str),
length: path_str.len(),
});
continue;
}

let path = Path::new(path_str);

if !path.has_root() {
violations.push(PolicyViolation::RelativePath {
path: path_str.clone(),
});
}

if path
.components()
.any(|c| matches!(c, std::path::Component::ParentDir))
{
violations.push(PolicyViolation::PathTraversal {
path: path_str.clone(),
});
}
}

// Only reject "/" as read-write (overly broad)
for path_str in &fs.read_write {
let normalized = path_str.trim_end_matches('/');
if normalized.is_empty() {
// Path is "/" or "///" etc.
violations.push(PolicyViolation::OverlyBroadPath {
path: path_str.clone(),
});
}
}
}

// Check network policy endpoint hosts for TLD wildcards.
for (key, rule) in &policy.network_policies {
let name = if rule.name.is_empty() {
key.clone()
} else {
rule.name.clone()
};
for ep in &rule.endpoints {
if ep.host.contains('*') && (ep.host.starts_with("*.") || ep.host.starts_with("**.")) {
let label_count = ep.host.split('.').count();
if label_count <= 2 {
violations.push(PolicyViolation::TldWildcard {
policy_name: name.clone(),
host: ep.host.clone(),
});
}
}
}
}

if violations.is_empty() {
Ok(())
} else {
Err(violations)
}
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This validation logic is now duplicated between here and what remained in openshell-policy: https://github.com/feloy/OpenShell/blob/5c6717756dc344691f820d2a70df961d25624dea/crates/openshell-policy/src/lib.rs#L439-L533

This could lead to drift when one function is updated and not the other one.

@feloy feloy Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added a commit to remove the duplicated logic

@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

PR Review Status

Validation: This PR is project-valid because it implements linked issue #1608 by splitting dependency-light policy YAML schema handling out of the proto-backed openshell-policy adapter.
Head SHA: 5c6717756dc344691f820d2a70df961d25624dea

Review findings:

  • crates/openshell-policy-schema/src/lib.rs and crates/openshell-policy/src/lib.rs: truncate_for_display slices by byte offset (&s[..77]), so an overlong non-ASCII path can panic validation instead of returning PolicyViolation::FieldTooLong. Please truncate on chars() or another UTF-8-safe boundary.
  • crates/openshell-policy-schema/src/lib.rs: the new public API exposes implementation names such as PolicyFile, FilesystemDef, and NetworkEndpointDef, while issue feat: extract openshell-policy-schema as a thin crate with no proto dependency #1608 describes consumer-facing names such as SandboxPolicy, FilesystemPolicy, and NetworkEndpoint. Please add the consumer-facing aliases or otherwise align the public contract.
  • architecture/security-policy.md: the architecture boundary is not updated. Please document that openshell-policy-schema owns YAML serde types and pure parsing, while openshell-policy owns proto conversion, compose, and merge.

Docs: Fern docs are not required for a direct user-facing workflow change in this PR, but the architecture crate-boundary note above is still needed.
Checks: DCO, Branch Checks, Helm Lint, and E2E are passing for the current head.
E2E: test:e2e is applied and green.

Next state: gator:in-review

@johntmyers johntmyers added the gator:in-review Gator is reviewing or awaiting PR review feedback label Jun 7, 2026
feloy added 17 commits June 8, 2026 09:15
Signed-off-by: Philippe Martin <phmartin@redhat.com>
Signed-off-by: Philippe Martin <phmartin@redhat.com>
Verbatim copy from crates/openshell-policy/src/lib.rs.
Only visibility changes: MAX_FILESYSTEM_PATHS, MAX_PATH_LENGTH, and
truncate_for_display promoted to pub(crate).

Signed-off-by: Philippe Martin <phmartin@redhat.com>
Verbatim copy from crates/openshell-policy/src/lib.rs lines 681-735.

Signed-off-by: Philippe Martin <phmartin@redhat.com>
Verbatim copy of function bodies from crates/openshell-policy/src/lib.rs,
adapted to operate on schema types (PolicyFile, FilesystemDef, ProcessDef,
etc.) instead of proto types, and using policy.filesystem_policy instead
of policy.filesystem.

Added Default derive to ProcessDef: the original ensure_sandbox_process_identity
uses ProcessPolicy::default(), which proto-generated types provide automatically.
ProcessDef needs an explicit derive to achieve the same.

Signed-off-by: Philippe Martin <phmartin@redhat.com>
Ported from crates/openshell-policy/src/lib.rs tests section.

Adaptations required vs the original:
- parse_sandbox_policy/serialize_sandbox_policy -> parse_policy/serialize_policy
- policy.filesystem -> policy.filesystem_policy
- FilesystemPolicy/ProcessPolicy -> FilesystemDef/ProcessDef
- SandboxPolicy{..., network_policies: HashMap::new()} -> PolicyFile{..., BTreeMap::new()}
- rule.allow.as_ref().unwrap() -> rule.allow (L7RuleDef.allow is not an Option)
- .query[k].glob/.any proto fields -> QueryMatcherDef::Glob/Any enum matching
- NetworkPolicyRule/NetworkEndpoint ..Default::default() construction ->
  parse_policy() helper (those types have no Default derive)
- parse_ports_array/parse_single_port: no port normalization in schema crate
  (normalization happens in to_proto in openshell-policy)

Signed-off-by: Philippe Martin <phmartin@redhat.com>
Signed-off-by: Philippe Martin <phmartin@redhat.com>
Signed-off-by: Philippe Martin <phmartin@redhat.com>
Remove the private serde type definitions (PolicyFile, FilesystemDef,
LandlockDef, ProcessDef, NetworkPolicyRuleDef, NetworkEndpointDef,
GraphqlOperationDef, L7RuleDef, L7AllowDef, QueryMatcherDef, QueryAnyDef,
L7DenyRuleDef, NetworkBinaryDef, is_zero, is_zero_u32) and import them
from openshell-policy-schema instead.

to_proto, from_proto, and all public functions are unchanged — they
reference the same type names, now resolved from the schema crate.

Signed-off-by: Philippe Martin <phmartin@redhat.com>
parse_sandbox_policy, serialize_sandbox_policy, sandbox_policy_to_json_value,
serialize_sandbox_policy_json, load_sandbox_policy, and restrictive_default_policy
now delegate to the schema crate and apply to_proto / from_proto for the
proto conversion layer. No behaviour change.

Signed-off-by: Philippe Martin <phmartin@redhat.com>
truncate_for_display stays in openshell-policy as a private helper for
validate_sandbox_policy, since the schema crate exposes it as pub(crate) only.

Signed-off-by: Philippe Martin <phmartin@redhat.com>
…erde_yml deps

Remove the local PolicyViolation definition and re-export it from
openshell-policy-schema. Callers importing openshell_policy::PolicyViolation
are unaffected — the re-export preserves the public path.

serde and serde_yml are no longer direct dependencies of openshell-policy
since all YAML logic is now handled by openshell-policy-schema.

Signed-off-by: Philippe Martin <phmartin@redhat.com>
Signed-off-by: Philippe Martin <phmartin@redhat.com>
Signed-off-by: Philippe Martin <phmartin@redhat.com>
Signed-off-by: Philippe Martin <phmartin@redhat.com>
Signed-off-by: Philippe Martin <phmartin@redhat.com>
Replace the duplicated validation logic in validate_sandbox_policy with
a one-liner that delegates to openshell_policy_schema::validate_policy
via from_proto. Remove the dead constants, truncate_for_display helper,
and Path import that were only used by the now-removed logic.

Replace 15 duplicate validation tests with 3 smoke tests (one per field
category: process, filesystem, network) that confirm violations survive
the from_proto roundtrip.
@feloy feloy force-pushed the crate-policy-schema-2 branch from 5c67177 to a02621d Compare June 8, 2026 08:14
@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

Re-check After Author Update

I re-evaluated latest head a02621d8b90d6f5ac093b4996b2d45fc2e9b8396 after @feloy's 2026-06-08 08:14 UTC inline reply that the duplicated validation logic was removed.

Disposition: partially resolved. The duplicate validation concern is structurally addressed, but the update introduced a validation regression and two prior gator findings remain unresolved.

Remaining items:

  • crates/openshell-policy/src/lib.rs:199 and crates/openshell-policy/src/lib.rs:421: validate_sandbox_policy now validates from_proto(policy), but from_proto drops Some(ProcessPolicy { run_as_user: "", run_as_group: "" }) to None. openshell_policy_schema::validate_policy accepts process: None, so explicitly empty proto process identity values now pass validation where the previous validator rejected them. Please preserve explicit proto process values for validation, or use a validation-specific adapter, and restore a test for empty proto process fields.
  • crates/openshell-policy-schema/src/lib.rs:334: truncate_for_display still slices with &s[..77], so an overlong non-ASCII path can panic validation instead of returning PolicyViolation::FieldTooLong. Please truncate on a UTF-8-safe boundary and add a regression test with an overlong UTF-8 path.
  • crates/openshell-policy-schema/src/lib.rs:27: the public schema API still exposes implementation names such as PolicyFile, FilesystemDef, and NetworkEndpointDef. Please align the public contract with the consumer-facing names from issue feat: extract openshell-policy-schema as a thin crate with no proto dependency #1608, such as SandboxPolicy, FilesystemPolicy, and NetworkEndpoint, or otherwise document the intended API shape.
  • architecture/security-policy.md: the architecture boundary still needs to mention that openshell-policy-schema owns YAML serde types and pure parsing, while openshell-policy owns proto conversion, compose, and merge.

Docs: Fern docs are not required for this PR, but the architecture boundary note above is still needed.
Checks: Current required gates are pending for the new head and waiting on the copy-pr mirror; review feedback remains blocking first.
E2E: test:e2e remains applied.

Next state: gator:in-review

@johntmyers

Copy link
Copy Markdown
Collaborator

/ok to test a02621d

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gator:in-review Gator is reviewing or awaiting PR review feedback test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: extract openshell-policy-schema as a thin crate with no proto dependency

3 participants