Skip to content

[WIP] feat/labs migration#1968

Draft
alanpeixinho wants to merge 3 commits into
kernelci:mainfrom
profusion:feat/labs-migration
Draft

[WIP] feat/labs migration#1968
alanpeixinho wants to merge 3 commits into
kernelci:mainfrom
profusion:feat/labs-migration

Conversation

@alanpeixinho

@alanpeixinho alanpeixinho commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Migrates all read paths for lab information from JSONB misc fields (builds.misc->>'lab',
tests.misc->>'runtime') to the new labs table via lab_id foreign key, with JSONB fallback for rows not yet backfilled.
Every SQL query and Python helper that previously extracted lab names by parsing JSONB now uses COALESCE(labs.name, misc_fallback) through a LEFT JOIN on the labs table. This ensures:

  • New data (with lab_id populated by the ingester) reads from the structured FK
  • Historical data (without lab_id) still works via the JSONB fallback
  • After a full backfill, the JSONB fallbacks can be removed (all marked with TODO comments)

How to test

legacy database

  • Use a database where lab_id is NULL on existing rows
  • Verify all pages display lab names as before:
    • Tree details page (boots/tests tabs) — lab column in test history
    • Tree commits history page — build lab and test lab in filters and rows
    • Hardware details page — lab in test/build summaries, history, and filters
    • Build details page — lab column in test list
    • Issue details page — lab column in test list
    • Notifications/metrics endpoint — lab summary counts

Migrated database (test in a local database)

  • Run the migrations available
  • Run the ingester (which already fills lab_id)
    • Verify all pages display lab names as before:
    • Tree details page (boots/tests tabs) — lab column in test history
    • Tree commits history page — build lab and test lab in filters and rows
    • Hardware details page — lab in test/build summaries, history, and filters
    • Build details page — lab column in test list
    • Issue details page — lab column in test list
    • Notifications/metrics endpoint — lab summary counts

Part of kernelci#1948

Signed-off-by: Alan Peixinho <alan.peixinho@profusion.mobi>
Part of kernelci#1948

Signed-off-by: Alan Peixinho <alan.peixinho@profusion.mobi>
    * For analysis queries we are going for lab column information
      first, and coalescing to json misc information.
    *  All COALESCE expressions are marked with TODO comments for removal
      after the lab_id backfill is complete.

    Closes kernelci#1948

Signed-off-by: Alan Peixinho <alan.peixinho@profusion.mobi>
@alanpeixinho alanpeixinho marked this pull request as draft July 1, 2026 20:59
@alanpeixinho alanpeixinho changed the title feat/labs migration {feat/labs migration Jul 1, 2026
@alanpeixinho alanpeixinho changed the title {feat/labs migration [WIP] feat/labs migration Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant