Add is_first_party label to CLI command metrics#7758
Open
SimonSiju wants to merge 2 commits into
Open
Conversation
Adds an is_first_party label to cli_commands_total (and the duration / wall-clock-elapsed metrics) so first-party (1P) Shopify developer usage can be distinguished from third-party usage in observability dashboards. The label is keyed off the 1P dev path (SHOPIFY_CLI_1P_DEV) via firstPartyDev(), the same signal that sets the X-Shopify-Cli-Employee header. This unblocks the 1P CLI guardrail panels on the [Dev Platform] 1P Orgs Migration dashboard, which currently blend 1P + 3P traffic. Closes: shop/issues-develop#22933
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WHY are these changes introduced?
closes: https://github.com/shop/issues-develop/issues/22933
Part of the 1P org migration observability work (GSD #50881). "1P CI/CD pipelines function without breakage after cutover" is a named guardrail on the [Dev Platform] 1P Orgs Migration dashboard (Observe
si2kjvt), but today it can't be measured:cli_commands_total(and the companion duration/elapsed metrics) carry onlycli_version,exit, andjoblabels — no org / 1P dimension — so the CLI guardrail panels blend 1P + 3P + all external developers.WHAT is this pull request doing?
Adds an
is_first_partylabel to the CLI command metrics emitted fromcli-kit:cli_commands_totalcli_commands_duration_mscli_commands_wall_clock_elapsed_msThe label is keyed off the 1P dev path via
firstPartyDev()(theSHOPIFY_CLI_1P_DEVenv var), which is the same signal that sets theX-Shopify-Cli-Employeeheader. Values are the strings"true"/"false", keeping cardinality low.This lets the dashboard's CLI guardrail panels filter to 1P, so we get a 1P-scoped success/failure signal for commands like
app dev,app dev clean, andgenerate extensionafter cutover.Files changed:
packages/cli-kit/src/private/node/otel-metrics.ts— addedis_first_partyto theLabelstype and populated it fromfirstPartyDev()inrecordMetrics; importedfirstPartyDevfromcontext/local.packages/cli-kit/src/private/node/otel-metrics.test.ts— added a test asserting all recorded metrics carryis_first_party: "true"whenSHOPIFY_CLI_1P_DEVis set; addedafterEachenv cleanup.packages/cli-kit/src/private/node/__snapshots__/otel-metrics.test.ts.snap— regenerated to include the new label ("false"when the 1P dev path is off)..changeset/first-party-cli-metrics-label.md—@shopify/cli-kitpatch changeset.How to test your changes?
labels metrics as first-party when the 1P dev path is enabled.cli_commands_totalrecordsis_first_party: "false".SHOPIFY_CLI_1P_DEV=1 shopify app dev) and confirm the metric recordsis_first_party: "true".Checklist
patchfor bug fixes ·minorfor new features ·majorfor breaking changes) and added a changeset withpnpm changeset add