Skip to content

fix(cli): make required_version advisory, warn on eval failure instead of prompting/exiting#1396

Draft
christso wants to merge 1 commit into
mainfrom
fix/1395-required-version-advisory
Draft

fix(cli): make required_version advisory, warn on eval failure instead of prompting/exiting#1396
christso wants to merge 1 commit into
mainfrom
fix/1395-required-version-advisory

Conversation

@christso

@christso christso commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Make required_version mismatches advisory by default: warn once, never prompt, self-update, or exit unless --strict is explicitly used.
  • Surface a failure-time eval summary note when results have quality failures or execution errors and the installed version does not satisfy required_version.
  • Update serve/Dashboard comments, self-update header docs, public docs, and tests for the new contract.

Verification

  • bun run verify

Manual UAT

Temp project used .agentv/config.yaml with required_version: ">=999.0.0", a mock target, and deterministic contains assertions.

Red (origin/main):

  • failing eval exited 1 from threshold failure and printed RESULT: FAIL, but no summary note about the version mismatch.
  • passing eval exited 0; mismatch warning was stderr-only.
  • TTY path hung after the mismatch warning until killed, demonstrating the old prompt path.

Green (this branch):

  • failing eval exited 1 from threshold failure and printed the version mismatch note in the summary.
  • passing eval exited 0 with no summary note, only one low-noise stderr warning.
  • TTY passing eval completed with no prompt, self-update, or version-based exit.

Failure-time note observed:

note: agentv 4.38.0 does not satisfy this project's required_version >=999.0.0 - this may be the cause. Run `agentv self update`.

Closes #1395

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: dd86f45
Status: ✅  Deploy successful!
Preview URL: https://64f7e17d.agentv.pages.dev
Branch Preview URL: https://fix-1395-required-version-ad.agentv.pages.dev

View logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make required_version advisory: warn (esp. on eval failure), don't enforce/prompt/self-update

1 participant