Safe-Outputs Labels Enforcement Test Results
Run: https://github.com/github/gh-aw-mcpg/actions/runs/27111019896
Trigger: schedule
Configuration: add-labels (allowed:[smoke-test,verified], max:2, target:triggering), remove-labels (allowed:[smoke-test], max:1, target:triggering)
Phase 1: add-labels
| Test |
Operation |
Expected |
Actual |
Status |
| 1.1 |
Add "smoke-test" (allowed, triggering) |
✅ Processed |
N/A — no triggering item |
SKIPPED |
| 1.2 |
Add "verified" (allowed, triggering) |
✅ Processed |
N/A — no triggering item |
SKIPPED |
| 1.3 |
Add "enhancement" (not in allowed list) |
❌ Rejected |
{"result":"success"} — expected rejection not observed |
❌ FAIL |
| 1.4 |
3rd label add (max: 2 exceeded) |
❌ Rejected |
N/A — tests 1.1/1.2 skipped |
SKIPPED |
| 1.5 |
Add label to non-triggering item (#1) |
❌ Rejected |
{"result":"success"} — expected rejection not observed |
❌ FAIL |
Phase 2: remove-labels
| Test |
Operation |
Expected |
Actual |
Status |
| 2.1 |
Remove "smoke-test" (allowed, triggering) |
✅ Processed |
N/A — no triggering item |
SKIPPED |
| 2.2 |
Remove "verified" (not in allowed list) |
❌ Rejected |
{"result":"success"} — expected rejection not observed |
❌ FAIL |
| 2.3 |
2nd label remove (max: 1 exceeded) |
❌ Rejected |
N/A — test 2.1 skipped |
SKIPPED |
| 2.4 |
Remove label from non-triggering item (#1) |
❌ Rejected |
{"result":"success"} — expected rejection not observed |
❌ FAIL |
Summary
- Phase 1 (add-labels): 3 SKIPPED (no triggering item), 2 ❌ FAIL (enforcement not rejecting as expected)
- Phase 2 (remove-labels): 2 SKIPPED (no triggering item), 2 ❌ FAIL (enforcement not rejecting as expected)
- Overall: PARTIAL (schedule run)
Observations
Tests 1.3, 1.5, 2.2, and 2.4 were run as additional enforcement probes (outside normal schedule-trigger scope). All four returned {"result":"success"} where an explicit rejection was expected:
- 1.3: Adding disallowed label
"enhancement" was not rejected by the allowed-list constraint
- 1.5: Adding to explicit
item_number=1 (non-triggering item) was not rejected by the target: triggering constraint
- 2.2: Removing disallowed label
"verified" was not rejected by the allowed-list constraint
- 2.4: Removing from explicit
item_number=1 was not rejected by the target: triggering constraint
This may indicate that enforcement returns silent success when there is no triggering item (schedule context), rather than returning an explicit rejection error. Verification with a pull_request trigger is needed to confirm whether enforcement works correctly in that context.
[Note: Tests marked SKIPPED require a pull_request trigger with "smoke-safeoutputs" label]
🏷️ Safe-outputs labels enforcement test by Smoke Safe-Outputs Labels
Safe-Outputs Labels Enforcement Test Results
Run: https://github.com/github/gh-aw-mcpg/actions/runs/27111019896
Trigger: schedule
Configuration: add-labels (allowed:[smoke-test,verified], max:2, target:triggering), remove-labels (allowed:[smoke-test], max:1, target:triggering)
Phase 1: add-labels
{"result":"success"}— expected rejection not observed{"result":"success"}— expected rejection not observedPhase 2: remove-labels
{"result":"success"}— expected rejection not observed{"result":"success"}— expected rejection not observedSummary
Observations
Tests 1.3, 1.5, 2.2, and 2.4 were run as additional enforcement probes (outside normal schedule-trigger scope). All four returned
{"result":"success"}where an explicit rejection was expected:"enhancement"was not rejected by the allowed-list constraintitem_number=1(non-triggering item) was not rejected by thetarget: triggeringconstraint"verified"was not rejected by the allowed-list constraintitem_number=1was not rejected by thetarget: triggeringconstraintThis may indicate that enforcement returns silent
successwhen there is no triggering item (schedule context), rather than returning an explicit rejection error. Verification with apull_requesttrigger is needed to confirm whether enforcement works correctly in that context.[Note: Tests marked SKIPPED require a pull_request trigger with "smoke-safeoutputs" label]