Feature/654 add and use general matrix by ArBridgeman · Pull Request #860 · exasol/python-toolbox

ArBridgeman · 2026-06-09T12:32:48Z

relates to #654

Checklist

Note: If any of the items in the checklist are not relevant to your PR, just check the box.

For any Pull Request

Is the following correct:

the title of the Pull Request?
the title of the corresponding issue?
there are no other open Pull Requests for the same update/change?
that the issue which this Pull Request fixes ("Fixes...") is mentioned?

When Changes Were Made

Did you:

update the changelog?
update the cookiecutter-template?
update the implementation?
check coverage and add tests: unit tests and, if relevant, integration tests?
update the User Guide & other documentation?
resolve any failing CI criteria (incl. Sonar quality gate)?

When Preparing a Release

Have you:

thought about version number (major, minor, patch)?
checked Exasol packages for updates and resolved open vulnerabilities, if easily possible?

ArBridgeman · 2026-06-09T12:33:29Z

  workflow_call:
+    inputs:
+      matrix_keys:
+        description: "Space-separated BaseConfig keys to include in the generated matrix output"


Should check if space separated is secure, reasonable

Why not a JSON string (which then must be an array)?

ArBridgeman · 2026-06-09T12:35:27Z

+
+@pytest.fixture
+def config(tmp_path) -> BaseConfig:
+    class Config(BaseConfig):


@tomuben and @tkilias
We should discuss if this is an ok pattern for the ITDE.

So the two options I see are:

As built in, one can extend the BaseConfig with additional fields. These should be typed & pydantic can validate the input before other operations are executed. I think that's a nice feature.

We allow BaseConfig to have extra fields which are not checked by pydantic. The benefit would be that you do not have to inherit and extend the BaseConfig.

ArBridgeman · 2026-06-09T12:36:46Z

@@ -1,40 +0,0 @@
-(( workflow_header ))


Similar to how we have released slow-checks.yml from PTB control, I though we could remove these template ymls. If a project cannot immediately migrate, they can retain these files. We will keep the nox sessions until 2026-09-15, so there isn't a major change. Does that sound alright or do we need to not do (or a major release)?

ckunki · 2026-06-12T07:11:54Z

+    """
+    Return the config keys that are valid for matrix generation.

+    Includes both declared fields and computed fields.


I don't understand the difference and intention of distinguishing "declared" vs. "computed" fields.
Could we (if this distinction is important) maybe add some explanation and/or example to the documentation?

Thanks, I added more context in the docstring.

ckunki · 2026-06-12T07:19:37Z

+    return parser.parse_args(session.posargs).keys
+
+
+def _dump_matrix(config: BaseConfig, keys: Iterable[str]) -> dict[str, Any]:


I am a bit confused by the variety of functions and names.
There is _dump...(), _print_...(), and generate_...().
Could we maybe

Reduce this to a single term, e.g. generate (which is my favorite)

If some of these functions are somewhat deprecated, old, outdated, or to be removed in near future, then maybe add a prefix or suffix like old_ or _deprecated, etc.?

Additionally, it seems, the file is (still) called _ci.yml but all functions in it only seem to deal with matrixes. Maybe _build_matrix.py could be a better name?

Hence, we could maybe also simplify the names of the functions inside it my removing redundant pre- or suffixes which could also give us more freedom in other naming issues, e.g. when distinguishing current/new/future variants from old/deprecated/outdated ones?

Good point with the name. To match the other ones, I've shifted it to _matrix.py. At this time, I wouldn't do other refactorings on this higher file level, as I feel they're out of scope with this issue.

Ok, I've move deprecated to be at the front of the function, but it was in the _print_deprecated_matrix and tried to align the names.

ckunki · 2026-06-12T07:24:57Z

-    matrix = _python_matrix(PROJECT_CONFIG)
-    matrix.update(_exasol_matrix(PROJECT_CONFIG))
-    print(json.dumps(matrix))
+    _print_deprecated_matrix(


Could we maybe in one place add an explanation about the semantics and meaning of the key_map attribute?

Currently, it maps strings (which a dash character) to other strings (with an underscore).
Why?

Yes, I will add an explanation. The main purpose is to include in the deprecated method the shift you need to switch to the newer matrix generation.

I'm ok to remove that if you think it's too much.

sonarqubecloud · 2026-06-12T09:02:56Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
97.7% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

ArBridgeman added 3 commits June 9, 2026 14:08

Add matrix:generate and deprecate older endpoints

0877d12

Add matrix.yml and remove older ones

ec7f6f9

Update PTB's workflows

233eba4

ArBridgeman marked this pull request as draft June 9, 2026 12:32

ArBridgeman commented Jun 9, 2026

View reviewed changes

Fix missed workflow to matrix.yml

b0012bd

ArBridgeman temporarily deployed to manual-approval June 9, 2026 12:38 — with GitHub Actions Inactive

ArBridgeman mentioned this pull request Jun 10, 2026

Refactoring/242 update to ptb 8.2.0 exasol/mcp-server#244

Merged

ArBridgeman added 2 commits June 10, 2026 14:05

Switch poetry.urls to project.urls

c552e5e

Switch to a JSON array

814f2a0

ArBridgeman temporarily deployed to manual-approval June 10, 2026 12:10 — with GitHub Actions Inactive

ArBridgeman added 2 commits June 10, 2026 14:12

Alter so runs when workflow file is modified to reduce delayed errors

3fc086b

Switch to jq and bash

15a8d40

ArBridgeman temporarily deployed to manual-approval June 11, 2026 04:56 — with GitHub Actions Inactive

ArBridgeman added 4 commits June 11, 2026 07:14

Put behind slow approval for PR too

ed980e2

Merge branch 'main' into feature/654_add_and_use_general_matrix

fdd7433

Add summary description

b9bf47b

Add documentation

9eca9d6

ArBridgeman temporarily deployed to manual-approval June 11, 2026 11:46 — with GitHub Actions Inactive

ArBridgeman marked this pull request as ready for review June 11, 2026 11:46

ckunki requested changes Jun 12, 2026

View reviewed changes

ArBridgeman added 4 commits June 12, 2026 09:46

Update .gitattributes for matrix.yml

0be697e

Apply reviewer corrections to unreleased.md

e4b9a78

Apply review improvement to create_and_update.rst

9b08dcf

Apply review improvements to workflow_variables.rst

a4257a7

ArBridgeman requested a deployment to manual-approval June 12, 2026 07:55 — with GitHub Actions Waiting

ArBridgeman added 5 commits June 12, 2026 10:00

Fix indentation

6edd52e

Improve docstring to give the pydantic context

8fe1025

Switch to generate instead of dump

185f956

Switch _ci.py to _matrix.py

54a47ff

Switch to deprecated_matrix_output and add docstring

c62cdcb

ArBridgeman requested a deployment to manual-approval June 12, 2026 08:14 — with GitHub Actions Waiting

Align names once more

bfd8217

ArBridgeman requested a deployment to manual-approval June 12, 2026 08:18 — with GitHub Actions Waiting

Use format:fix

bd7b518

ArBridgeman requested a deployment to manual-approval June 12, 2026 08:19 — with GitHub Actions Waiting

Remove duplicate key check

3cb1e7e

ArBridgeman requested a deployment to manual-approval June 12, 2026 08:37 — with GitHub Actions Waiting

Remove unneeded test

8d2c065

ArBridgeman requested a deployment to manual-approval June 12, 2026 08:51 — with GitHub Actions Waiting

Remove unneeded call

b9f9681

ArBridgeman requested a deployment to manual-approval June 12, 2026 08:59 — with GitHub Actions Waiting

ArBridgeman requested review from ckunki and tomuben June 12, 2026 10:07

		return parser.parse_args(session.posargs).keys


		def _dump_matrix(config: BaseConfig, keys: Iterable[str]) -> dict[str, Any]:

Conversation

ArBridgeman commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

For any Pull Request

When Changes Were Made

When Preparing a Release

Uh oh!

ArBridgeman Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ArBridgeman Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ArBridgeman Jun 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud Bot commented Jun 12, 2026

Quality Gate passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ArBridgeman commented Jun 9, 2026 •

edited

Loading

ArBridgeman Jun 9, 2026 •

edited

Loading

ArBridgeman Jun 9, 2026 •

edited

Loading

ArBridgeman Jun 12, 2026 •

edited

Loading