Trim verbose asides from docs and website

CONTRIBUTING.md

@@ -1,25 +1,19 @@
 # Submitting a model run
 
 The benchmark gives every model the **same $20 API budget** and asks: how many distinct
-reduction-rule bugs (counterexamples) can it find? This document describes the end-to-end
-submission pipeline.
+reduction-rule bugs can it find?
 
 ```
   make run ─▶ submission.json ─▶ python -m benchmark.submit  ──▶  private store (R2)
-                                 (certificate + trajectory = answer key, never in git)
                                           │
                     maintainer's scorer re-verifies every certificate with pred
-                    (zero trust) + a provenance check, OFF the public repo
                                           │
-              only the AGGREGATE (counts / cost / tokens — no rules, no certs)
-              is published ─▶ PR to site/results.json ─▶ GitHub Pages (deploy)
+              only the aggregate is published ─▶ PR ─▶ GitHub Pages
 ```
 
-Your submission carries the certificate + trajectory — the answer key. On a fixed public
-library commit a `pred`-confirmed certificate counts regardless of who produced it, so it
-must stay private: the CLI uploads to a private store, and only the aggregate becomes
-public. Self-reported counts are never trusted — the score is recomputed from `pred`
-re-verification.
+Your submission carries the certificate + trajectory, so it uploads to a private store;
+only the aggregate is published. Self-reported counts are never trusted — the score is
+recomputed by `pred`.
 
 ## 1. Produce a `submission.json` (dockerized runner)
 
@@ -122,14 +116,12 @@ make preflight        # docker run --env-file submission.env <image> --preflight
 ```
 
 It checks the `pred` binary + version, that the library rules are present, and makes one
-minimal model call through the exact batch code path (validating credentials, endpoint,
-`model_kwargs`, and that pricing computes). It exits non-zero on any failure. (The runner's
-no-API wiring is covered by the pytest suite, not a separate command.)
+minimal model call through the exact batch code path. It exits non-zero on any failure.
 
 ## 2. Submit it (CLI upload)
 
-Submission is a **CLI upload** — there's no web form and the file never enters git (it
-carries the answer key). Get the endpoint URL + a token from the maintainer, then:
+Submission is a **CLI upload** — no web form, and the file never enters git. Get the
+endpoint URL + a token from the maintainer, then:
 
 ```bash
 export PRB_SUBMIT_URL=<intake endpoint>   # from the maintainer
@@ -140,10 +132,9 @@ python -m benchmark.submit --predictions out/submission.json
 #   --test      scored + stored privately, but excluded from the public leaderboard
 ```
 
-The CLI validates the file against `submission.schema.json`, then uploads it over HTTPS to
-a private store (Cloudflare R2). The maintainer's scorer re-verifies every certificate with
-`pred` off-repo (see §3) and opens a PR that updates only the aggregate `site/results.json`;
-merging deploys the site to **GitHub Pages**. Your self-reported counts are never trusted.
+The CLI validates the file against `submission.schema.json`, then uploads it to a private
+store (Cloudflare R2). The maintainer's scorer re-verifies it with `pred` (see §3) and opens
+a PR that updates the aggregate `site/results.json`; merging deploys to **GitHub Pages**.
 See `intake/cloudflare-worker/README.md` for the intake setup.
 
 ## 3. Backend verification (automatic, zero-trust)

README.md

@@ -2,7 +2,7 @@
 
 A benchmark that measures how efficiently AI models find bugs in reduction rules from the [problem-reductions](https://github.com/CodingThrust/problem-reductions) library (290+ rules).
 
-The leaderboard is a static site (`site/`) published to **GitHub Pages**. Submitting uses a CLI (`python -m benchmark.submit`) that uploads your run to a **private store** — the certificate + trajectory are the answer key, so they never enter the public repo. The maintainer re-verifies every certificate with `pred` off-repo, and only the **aggregate** (counts, no rules/certs) becomes public. See [CONTRIBUTING.md](CONTRIBUTING.md) to run and submit.
+The leaderboard is a static site (`site/`) published to **GitHub Pages**. Submitting uses a CLI (`python -m benchmark.submit`) that uploads your run to a private store; the maintainer re-verifies it with `pred` and publishes only the aggregate. See [CONTRIBUTING.md](CONTRIBUTING.md) to run and submit.
 
 ## What this measures
 
@@ -46,7 +46,7 @@ class MyRunner(AgentRunner):
 
 Then pass it to `Scheduler` in `benchmark/scheduler.py`. See `MiniSweRunner` for a full example.
 
-A run is packaged as a `submission.json` (envelope around the per-rule rows, see `benchmark/submission.schema.json`) and uploaded with `python -m benchmark.submit` to a private store; the maintainer re-verifies every certificate with `pred` off-repo and publishes only the aggregate. See [CONTRIBUTING.md](CONTRIBUTING.md).
+A run is packaged as a `submission.json` (see `benchmark/submission.schema.json`) and uploaded with `python -m benchmark.submit`. See [CONTRIBUTING.md](CONTRIBUTING.md).
 
 ## How to run locally
 

site/README.md

@@ -1,16 +1,13 @@
 # Leaderboard site
 
-Static leaderboard (one `index.html`, no app server, instant first paint), published to
-**GitHub Pages** by `.github/workflows/publish-on-merge.yml`. Same **$20** budget for every
-model — who finds the most bugs in the problem-reductions reduction rules? Every bug is
-independently re-verified by `pred` (one rule = one bug).
+Static leaderboard (`index.html`, no app server) published to **GitHub Pages** by
+`.github/workflows/publish-on-merge.yml`. Every model gets the same **$20** budget; every
+bug is re-verified by `pred`.
 
 `index.html` reads two data files served alongside it:
 
-- `results.json` — the scored aggregate leaderboard (refreshed by `score-from-r2.yml`,
-  which re-verifies submissions off-repo and opens a PR; merging deploys the site)
+- `results.json` — the aggregate leaderboard, refreshed by `score-from-r2.yml`
 - `tasks.json` — the rule set shown on the Tasks tab
 
-Preview locally with `make serve` (the data files are loaded via `fetch`, so open it over
-HTTP, not `file://`). `results.json` holds only aggregate counts — never certificates or
-buggy-rule identities.
+Preview locally with `make serve` (data files load via `fetch`, so serve over HTTP, not
+`file://`). `results.json` holds only aggregate counts.

site/index.html

@@ -297,15 +297,15 @@ <h1 id="page-title">Leaderboard</h1>
       <h2>From your model to the leaderboard</h2>
       <p>Give a model the same <b>$20</b> and see how many bugs it can find in the library's problem
          reductions. You run it on your own model and key; we independently re-check every bug before
-         it scores — so the ranking is earned, not claimed.</p>
+         it scores.</p>
     </div>
 
     <div class="lane you">You run it</div>
     <div class="flow">
       <div class="fnode"><div class="fdot">1</div><div class="fcard"><h4>Get the runner</h4><p>One Docker image: the solver (<code>pred</code>) plus the rule library, pinned to the benchmark version.</p></div></div>
       <div class="fnode"><div class="fdot">2</div><div class="fcard"><h4>Plug in any model</h4><p>OpenAI, Anthropic, DeepSeek, a self-hosted endpoint — you bring the API key and the price.</p></div></div>
       <div class="fnode"><div class="fdot">3</div><div class="fcard"><h4>Hunt for bugs</h4><p>Your model probes each rule for a counterexample until the $20 runs out; each find is checked by the solver on the spot.</p><span class="eq">a bug = solve(x) ≠ solve(reduce(x))</span></div></div>
-      <div class="fnode"><div class="fdot">4</div><div class="fcard"><h4>Submit from the command line</h4><p>One <code>hf upload</code> of <code>submission.json</code> to the dataset, or open a pull request — no web form.</p></div></div>
+      <div class="fnode"><div class="fdot">4</div><div class="fcard"><h4>Submit from the command line</h4><p>Upload <code>submission.json</code> with <code>benchmark.submit</code>. No web form.</p></div></div>
     </div>
 
     <div class="trust"><div class="ln"></div></div>
@@ -319,42 +319,41 @@ <h2>From your model to the leaderboard</h2>
     <div class="defs">
       <div class="def">
         <h4>What counts as a bug</h4>
-        <p>Each reduction turns problem <i>A</i> into an equivalent problem <i>B</i>. It's buggy on an input when solving that input directly gives a different answer than solving it through the reduction — a fact anyone can re-check, so a wrong claim simply fails to reproduce.</p>
+        <p>Each reduction turns problem <i>A</i> into an equivalent problem <i>B</i>. It's buggy on an input when solving that input directly gives a different answer than solving it through the reduction.</p>
       </div>
       <div class="def">
         <h4>Why the $20 is fair</h4>
-        <p>Every model gets the same budget, measured from real token usage at the price you declare — not a gateway's dollar guess. Ranking is by <b>confirmed distinct-rule bugs</b>, with <b>bugs per 1K tokens</b> as the efficiency tie-break.</p>
+        <p>Every model gets the same budget, measured from real token usage at the price you declare. Ranking is by <b>confirmed distinct-rule bugs</b>, with <b>bugs per 1K tokens</b> as the efficiency tie-break.</p>
       </div>
     </div>
   </section>
 
   <section class="tab" id="tab-submit">
     <div class="prose">
       <h2>Submit a run</h2>
-      <p>Run the dockerized runner at the fixed <b>$20</b> budget against problem-reductions <code>v0.6.0</code> — any provider, all config in one <code>submission.env</code> — then add the <code>submission.json</code> it produces to the submissions dataset. Every certificate is re-verified by <code>pred</code> on the backend before it counts.</p>
+      <p>Run the dockerized runner at the fixed <b>$20</b> budget against problem-reductions <code>v0.6.0</code>, then upload the <code>submission.json</code> it produces. Every certificate is re-verified by <code>pred</code> before it counts.</p>
     </div>
     <div class="steps" style="margin-top:18px">
       <div class="step"><div class="num">1</div><div>
         <h4>Produce <code>submission.json</code></h4>
-        <p>Configure once (model · key · price), validate with one tiny real call, then run.</p>
-        <pre class="cmd"><span class="c"># configure once (model · key · price — any provider), then:</span>
-cp submission.env.example submission.env
+        <p>Configure once, validate with one tiny real call, then run.</p>
+        <pre class="cmd">cp submission.env.example submission.env   <span class="c"># model · key · price</span>
 make preflight   <span class="c"># validate config with one tiny real call</span>
 make run         <span class="c"># → ./out/submission.json</span></pre>
       </div></div>
       <div class="step"><div class="num">2</div><div>
         <h4>Upload it</h4>
-        <p>It lands as <code>PENDING</code>; the backend picks it up and re-verifies every certificate.</p>
-        <pre class="cmd">hf upload isPANN/problem-reductions-submissions submission.json \
-  submissions/&lt;your-handle&gt;/&lt;model&gt;.json --repo-type dataset</pre>
+        <p>The backend picks it up and re-verifies every certificate.</p>
+        <pre class="cmd">export PRB_SUBMIT_URL=… PRB_API_KEY=…
+python -m benchmark.submit --predictions out/submission.json</pre>
       </div></div>
     </div>
-    <p class="note" style="margin-top:14px">…or open a pull request adding it to the GitHub repo. Self-reported counts are advisory — only confirmed, distinct-rule bugs are ranked.</p>
+    <p class="note" style="margin-top:14px">Self-reported counts are advisory; only confirmed distinct-rule bugs are ranked.</p>
   </section>
 
   <section class="tab" id="tab-about">
     <h3 class="about-h">About the benchmark</h3>
-    <p class="about-p">An open measurement of how many bugs a model can find in the <b>problem-reductions</b> library — each reduction is meant to translate one computational problem into an equivalent one. Every model gets the same <b>$20</b>, and every claimed bug is independently re-verified server-side, so the ranking reflects confirmed findings, not self-reported counts.</p>
+    <p class="about-p">An open measurement of how many bugs a model can find in the <b>problem-reductions</b> library. Every model gets the same <b>$20</b>, and every claimed bug is independently re-verified server-side.</p>
     <div class="panel" style="margin-top:26px">
       <h3>How to cite</h3>
       <pre class="cmd" style="margin-top:12px">@misc{TODO_citation_key,
@@ -541,7 +540,7 @@ <h3>How to cite</h3>
     <div><dt>Bugs / Ktok</dt><dd>${row.effNum.toFixed(4)}</dd></div>
     <div><dt>Bugs / $</dt><dd>${row.spentEff.toFixed(3)}</dd></div>
   </dl>
-  <p class="note" style="margin-top:14px">Which rules each model found bugs in is not published — a confirmed certificate on this fixed commit counts regardless of provenance, so the buggy-rule identities are kept private to protect the benchmark.</p>`;
+  <p class="note" style="margin-top:14px">Buggy-rule identities are not published.</p>`;
   openDrawer();
 }
 

site/results/README.md

@@ -1,10 +1,8 @@
 # Per-submission leaderboard entries
 
 Each file here is **one submission's** public leaderboard entry:
-`<model-slug>--<UTC-time>--<short-id>.json` (aggregate only — counts, cost, tokens,
-efficiency; never certificates or buggy-rule identities).
+`<model-slug>--<UTC-time>--<short-id>.json`. Aggregate only; no certificates or rule identities.
 
-- Written by `score-from-r2.yml`, one **PR per submission**, so each is reviewed, merged,
-  or reverted independently.
+- Written by `score-from-r2.yml`, one **PR per submission**.
 - On merge, `publish-on-merge.yml` aggregates them (best run per model) into the deployed
-  `site/results.json` (generated, not committed) and publishes to GitHub Pages.
+  `site/results.json` and publishes to GitHub Pages.

submissions/README.md

@@ -1,17 +1,12 @@
 # Submissions
 
-**Submissions are never committed to this public repo.** A submission carries the
-certificate + trajectory — the answer key. On a fixed public library commit a
-`pred`-confirmed certificate counts regardless of who produced it, so publishing one would
-be a free answer key. Everything under `submissions/*.json` is `.gitignore`d.
-
-This directory is only a **local scratch space** for the self-run scoring path.
+**Submissions are never committed to this public repo** — they carry the answer key
+(certificate + trajectory) and are `.gitignore`d. This directory is only local scratch for
+the self-run scoring path.
 
 ## How to submit (external)
 
-Use the CLI intake — it uploads over HTTPS to a private store (Cloudflare R2); only the
-maintainer ever reads it back, and only the aggregate (counts, no rules/certs) becomes
-public.
+Use the CLI intake — it uploads to a private store; only the aggregate becomes public.
 
 ```bash
 export PRB_SUBMIT_URL=<intake endpoint>   # from the maintainer
@@ -26,8 +21,8 @@ the public leaderboard. See `intake/cloudflare-worker/README.md`.
 ## Self-run scoring (maintainer / local)
 
 Drop scored submission files into this directory and run `make publish-local`: it scores
-them with `pred`, rebuilds the aggregate, and writes `site/results.json` (guarded so no
-answer-key field leaks). The files here stay local — they never enter git.
+them with `pred`, rebuilds the aggregate, and writes `site/results.json`. The files here
+stay local; they never enter git.
 
 ## Notes