feat: use common resources in swarm#406
Merged
jason-lynch merged 9 commits intoJun 16, 2026
Merged
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✅ Files skipped from review due to trivial changes (1)
📝 WalkthroughWalkthroughSix swarm-local resource types (EtcdCreds, PatroniCluster, PatroniMember, PgBackRestConfig, PgBackRestStanza, PostgresCerts) are deleted and moved to the ChangesResource consolidation from swarm to common and state migration v1.2.0
E2e heartbeat helper for add-node test
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Up to standards ✅🟢 Issues
|
| Category | Results |
|---|---|
| Complexity | 6 medium |
🟢 Metrics 72 complexity · 64 duplication
Metric Results Complexity 72 Duplication 64
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🧹 Nitpick comments (1)
e2e/heartbeat_test.go (1)
1-2: ⚡ Quick winE2E test build tag is correct, but
-debugflag support is missing.The
//go:build e2e_testbuild tag follows the coding guideline. However, as per coding guidelines, E2E tests should also support debug mode with a-debugflag to preserve failed test environments for inspection.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@e2e/heartbeat_test.go` around lines 1 - 2, Add support for a `-debug` flag to the E2E test file heartbeat_test.go to preserve failed test environments for inspection. Define a package-level boolean flag variable for debug mode using the flag package, then implement logic in the test functions to skip cleanup or preserve resources when the debug flag is enabled. This allows engineers to inspect the test environment after failures for debugging purposes.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@e2e/heartbeat_test.go`:
- Around line 63-68: The Stop() method in the HeartBeat type closes the done
channel and waits for the goroutine to finish, but it never cancels the context
by calling l.cancel(). This can cause the goroutine to hang indefinitely if any
context-aware operation (such as database calls or the WithConnection call) is
blocking at the time Stop() is called. Add a call to l.cancel() in the Stop()
method before or after closing the done channel to ensure the context is
properly cancelled and the goroutine can exit cleanly.
- Around line 44-61: The sync.WaitGroup type does not have a Go method, only
Add, Done, and Wait. In the Start method where l.wg.Go is called on line 50, you
need to either replace l.wg with an errgroup.Group (from
golang.org/x/sync/errgroup) which provides the Go method, or manually manage the
WaitGroup by calling l.wg.Add(1) before launching the goroutine and l.wg.Done()
at the end of the goroutine function (replacing the current defer l.cancel()
with both defer l.wg.Done() and defer l.cancel() or combining them
appropriately).
In `@server/internal/resource/migrations/1_2_0.go`:
- Line 269: The error message in the fmt.Errorf call at line 269 contains a
copy/paste error where it says "patroni member resource" but should say
"pgBackRest config" to correctly reflect the actual resource type being migrated
in this section. Update the error message string to replace "patroni member"
with "pgBackRest config" while keeping the rest of the error message structure
intact.
- Line 234: The error message in the fmt.Errorf call contains a copy/paste error
where it refers to "patroni member resource" but should refer to "pgBackRest
config" to accurately describe the actual resource type being unmarshaled in
this migration. Update the error message string to use the correct resource type
name.
- Line 477: In the migration file at line 477, the OwnerUID field is incorrectly
being assigned from old.OwnerGID instead of old.OwnerUID. Fix this by changing
the assignment from OwnerUID: old.OwnerGID to OwnerUID: old.OwnerUID to prevent
file ownership metadata corruption during the migration.
In
`@server/internal/resource/migrations/golden_test/TestVersion_1_2_0/simple_instance.json`:
- Line 2: The golden fixture file currently shows version "1.1.0" (the
pre-migration state) when it should reflect the post-migration state with
version "1.2.0". Update the test code that sets up the state before calling
golden.Run to explicitly set state.Version to the target migration version
(1.2.0), matching the production behavior in service.go line 166 where
state.Version is set after migration.Run succeeds. This ensures the golden
fixture captures the correct final state after migration.
In `@server/internal/resource/migrations/schemas/v1_2_0/swarm.go`:
- Around line 53-58: The nested fields `Name`, `ID`, `Subnet`, and `Gateway`
within the `BridgeNetworkInfo` struct are missing JSON tags, causing them to
serialize with uppercase names instead of the lowercase convention used
throughout the schema. Add lowercase JSON tags to each of these four fields in
the nested struct (e.g., `json:"name"`, `json:"id"`, `json:"subnet"`,
`json:"gateway"`) to maintain consistency with the established naming
convention.
---
Nitpick comments:
In `@e2e/heartbeat_test.go`:
- Around line 1-2: Add support for a `-debug` flag to the E2E test file
heartbeat_test.go to preserve failed test environments for inspection. Define a
package-level boolean flag variable for debug mode using the flag package, then
implement logic in the test functions to skip cleanup or preserve resources when
the debug flag is enabled. This allows engineers to inspect the test environment
after failures for debugging purposes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 0f056246-5621-4d87-82e5-a14fe275f461
📒 Files selected for processing (43)
e2e/db_update_add_node_test.goe2e/heartbeat_test.goserver/internal/database/instance_resource.goserver/internal/orchestrator/common/patroni_config.goserver/internal/orchestrator/common/patroni_config_generator.goserver/internal/orchestrator/swarm/etcd_creds.goserver/internal/orchestrator/swarm/mcp_config_resource.goserver/internal/orchestrator/swarm/orchestrator.goserver/internal/orchestrator/swarm/patroni_cluster.goserver/internal/orchestrator/swarm/patroni_config.goserver/internal/orchestrator/swarm/patroni_config_golden_test.goserver/internal/orchestrator/swarm/patroni_member.goserver/internal/orchestrator/swarm/pgbackrest_config.goserver/internal/orchestrator/swarm/pgbackrest_restore.goserver/internal/orchestrator/swarm/pgbackrest_stanza.goserver/internal/orchestrator/swarm/postgres_certs.goserver/internal/orchestrator/swarm/postgres_service_spec.goserver/internal/orchestrator/swarm/postgrest_config_resource.goserver/internal/orchestrator/swarm/rag_config_resource.goserver/internal/orchestrator/swarm/resources.goserver/internal/resource/migrations/1_0_0.goserver/internal/resource/migrations/1_0_0_test.goserver/internal/resource/migrations/1_1_0.goserver/internal/resource/migrations/1_1_0_test.goserver/internal/resource/migrations/1_2_0.goserver/internal/resource/migrations/1_2_0_test.goserver/internal/resource/migrations/golden_test/TestVersion_1_2_0/simple_instance.jsonserver/internal/resource/migrations/golden_test/TestVersion_1_2_0/with_backup_and_restore_configs.jsonserver/internal/resource/migrations/golden_test/TestVersion_1_2_0/with_backup_config.jsonserver/internal/resource/migrations/golden_test/TestVersion_1_2_0/with_in-place_restore_config.jsonserver/internal/resource/migrations/golden_test/TestVersion_1_2_0/with_restore_config.jsonserver/internal/resource/migrations/main_test.goserver/internal/resource/migrations/provide.goserver/internal/resource/migrations/schemas/v1_1_0/database.goserver/internal/resource/migrations/schemas/v1_1_0/filesystem.goserver/internal/resource/migrations/schemas/v1_1_0/swarm.goserver/internal/resource/migrations/schemas/v1_2_0/common.goserver/internal/resource/migrations/schemas/v1_2_0/database.goserver/internal/resource/migrations/schemas/v1_2_0/filesystem.goserver/internal/resource/migrations/schemas/v1_2_0/swarm.goserver/internal/resource/service.goserver/internal/resource/state.goserver/internal/utils/utils.go
💤 Files with no reviewable changes (9)
- server/internal/orchestrator/swarm/resources.go
- server/internal/orchestrator/swarm/pgbackrest_config.go
- server/internal/orchestrator/swarm/pgbackrest_stanza.go
- server/internal/orchestrator/swarm/patroni_config_golden_test.go
- server/internal/orchestrator/swarm/patroni_cluster.go
- server/internal/database/instance_resource.go
- server/internal/orchestrator/swarm/patroni_member.go
- server/internal/orchestrator/swarm/postgres_certs.go
- server/internal/orchestrator/swarm/etcd_creds.go
jason-lynch
force-pushed
the
fix/PLAT-610/update-global-patroni-params-swarm
branch
from
26a3bd1 to
4c84cb3
Compare
Removes duplicated resources from the `swarm` package and adopts the newer resources from the `common` package. This deduplicates our code and enables us to take advantage of recently-added improvements in the `common` package, such as global parameter updates for Patroni. PLAT-610
Adds the pgBackRest backup and restore config resources to the common Patroni config dependencies. The pgBackRest configs are already being created before the Patroni config, but only because of happenstance. This change codifies that dependency to ensure these resources exist before we create the Patroni config. PLAT-610
Passing the database ID in as an argument is more convenient than extracting the ID from one of the resources in the state. PLAT-610
Refactors the migration tests so that tests can share an instance of the golden test helper. PLAT-610
Adds a migration for state v1.2.0 to migrate old swarm resources to the new common resources. PLAT-610
We're consistently setting failsafe mode in the patroni config resources now, so we no longer need to set it in the instance resource. PLAT-610
Adds a helper to provide a consistent volume of activity on a database to avoid the replication stalls and waits that can happen in an idle database. This became necessary for the add node tests after the swarm common resource refactor because they shifted the timing so that these stalls became much more likely. PLAT-610
jason-lynch
force-pushed
the
fix/PLAT-610/update-global-patroni-params-swarm
branch
from
4c84cb3 to
6ba8698
Compare
tsivaprasad
approved these changes
Jun 16, 2026
tsivaprasad
left a comment
tsivaprasad
left a comment
Contributor
There was a problem hiding this comment.
Awesome...!
control-plane git:(main) cp-init
checking if host-1 is initialized
checking if host-2 is initialized
checking if host-3 is initialized
checking if host-4 is initialized
checking if host-5 is initialized
checking if host-6 is initialized
initializing cluster from host-1
joining host-2 to the cluster
joining host-3 to the cluster
joining host-4 to the cluster
joining host-5 to the cluster
joining host-6 to the cluster
➜ control-plane git:(main) rm -rf /tmp/PLAT-610 && mkdir -p /tmp/PLAT-610
➜ control-plane git:(main) cp1-req create-database <<EOF | cp-follow-task
{
"id": "storefront",
"spec": {
"database_name": "storefront",
"database_users": [
{
"username": "admin",
"password": "password",
"db_owner": true,
"attributes": ["SUPERUSER", "LOGIN"]
}
],
"orchestrator_opts": {
"swarm": {
"extra_volumes": [
{
"host_path": "/tmp/PLAT-610",
"destination_path": "/backups"
}
]
}
},
"backup_config": {
"repositories": [
{
"id": "original",
"type": "posix",
"base_path": "/backups"
}
]
},
"nodes": [
{ "name": "n1", "host_ids": ["host-1"] },
{ "name": "n2", "host_ids": ["host-2"] }
]
}
}
EOF
{
"message": "validation error for node n1, host host-1: volumes: failed to start container: failed to ensure docker image \"ghcr.io/pgedge/pgedge-postgres:18.4-spock5.0.9-standard-1\": failed to pull image \"ghcr.io/pgedge/pgedge-postgres:18.4-spock5.0.9-standard-1\": Error response from daemon: Get \"https://ghcr.io/v2/\": dialing ghcr.io:443 container via direct connection because Docker Desktop has no HTTPS proxy: connecting to ghcr.io:443: dial tcp: lookup ghcr.io: no such host
validation error for node n2, host host-2: volumes: failed to start container: failed to ensure docker image \"ghcr.io/pgedge/pgedge-postgres:18.4-spock5.0.9-standard-1\": failed to pull image \"ghcr.io/pgedge/pgedge-postgres:18.4-spock5.0.9-standard-1\": Error response from daemon: Get \"https://ghcr.io/v2/\": dialing ghcr.io:443 container via direct connection because Docker Desktop has no HTTPS proxy: connecting to ghcr.io:443: dial tcp: lookup ghcr.io: no such host",
"name": "invalid_input"
}
no task object found on stdin
➜ control-plane git:(main) cp1-req create-database <<EOF | cp-follow-task
{
"id": "storefront",
"spec": {
"database_name": "storefront",
"postgres_version": "17.9",
"spock_version": "5",
"database_users": [
{
"username": "admin",
"password": "password",
"db_owner": true,
"attributes": ["SUPERUSER", "LOGIN"]
}
],
"orchestrator_opts": {
"swarm": {
"extra_volumes": [
{
"host_path": "/tmp/PLAT-610",
"destination_path": "/backups"
}
]
}
},
"backup_config": {
"repositories": [
{
"id": "original",
"type": "posix",
"base_path": "/backups"
}
]
},
"nodes": [
{ "name": "n1", "host_ids": ["host-1"] },
{ "name": "n2", "host_ids": ["host-2"] }
]
}
}
EOF
{
"database": {
"created_at": "2026-06-16T17:37:44Z",
"id": "storefront",
"spec": {
"backup_config": {
"repositories": [
{
"base_path": "/backups",
"id": "original",
"type": "posix"
}
]
},
"database_name": "storefront",
"database_users": [
{
"attributes": [
"SUPERUSER",
"LOGIN"
],
"db_owner": true,
"username": "admin"
}
],
"nodes": [
{
"host_ids": [
"host-1"
],
"name": "n1"
},
{
"host_ids": [
"host-2"
],
"name": "n2"
}
],
"orchestrator_opts": {
"swarm": {
"extra_volumes": [
{
"destination_path": "/backups",
"host_path": "/tmp/PLAT-610"
}
]
}
},
"postgres_version": "17.9",
"spock_version": "5"
},
"state": "creating",
"updated_at": "2026-06-16T17:37:44Z"
},
"task": {
"created_at": "2026-06-16T17:37:44Z",
"database_id": "storefront",
"entity_id": "storefront",
"scope": "database",
"status": "pending",
"task_id": "019ed182-b273-7f09-b9e9-3600ad15b629",
"type": "create"
}
}
[2026-06-16T17:37:45Z] refreshing current state
[2026-06-16T17:37:45Z] finished refreshing current state (took 202.786126ms)
[2026-06-16T17:37:47Z] creating resource swarm.patroni_cluster::n2
[2026-06-16T17:37:47Z] finished creating resource swarm.patroni_cluster::n2 (took 38.042µs)
[2026-06-16T17:37:47Z] creating resource swarm.patroni_cluster::n1
[2026-06-16T17:37:47Z] finished creating resource swarm.patroni_cluster::n1 (took 9.416µs)
[2026-06-16T17:37:47Z] creating resource swarm.network::storefront-database
[2026-06-16T17:37:47Z] finished creating resource swarm.network::storefront-database (took 6.33ms)
[2026-06-16T17:37:47Z] creating resource filesystem.dir::storefront-n2-9ptayhma-instance
[2026-06-16T17:37:47Z] finished creating resource filesystem.dir::storefront-n2-9ptayhma-instance (took 521.583µs)
[2026-06-16T17:37:47Z] creating resource filesystem.dir::storefront-n1-689qacsi-instance
[2026-06-16T17:37:47Z] finished creating resource filesystem.dir::storefront-n1-689qacsi-instance (took 626.791µs)
[2026-06-16T17:37:47Z] creating resource swarm.patroni_member::storefront-n2-9ptayhma
[2026-06-16T17:37:47Z] finished creating resource swarm.patroni_member::storefront-n2-9ptayhma (took 22.791µs)
[2026-06-16T17:37:47Z] creating resource swarm.patroni_member::storefront-n1-689qacsi
[2026-06-16T17:37:47Z] finished creating resource swarm.patroni_member::storefront-n1-689qacsi (took 6µs)
[2026-06-16T17:37:47Z] creating resource filesystem.dir::storefront-n2-9ptayhma-configs
[2026-06-16T17:37:47Z] finished creating resource filesystem.dir::storefront-n2-9ptayhma-configs (took 529.125µs)
[2026-06-16T17:37:47Z] creating resource filesystem.dir::storefront-n2-9ptayhma-data
[2026-06-16T17:37:47Z] finished creating resource filesystem.dir::storefront-n2-9ptayhma-data (took 570.917µs)
[2026-06-16T17:37:47Z] creating resource filesystem.dir::storefront-n2-9ptayhma-certificates
[2026-06-16T17:37:47Z] finished creating resource filesystem.dir::storefront-n2-9ptayhma-certificates (took 502.792µs)
[2026-06-16T17:37:47Z] creating resource filesystem.dir::storefront-n1-689qacsi-certificates
[2026-06-16T17:37:47Z] finished creating resource filesystem.dir::storefront-n1-689qacsi-certificates (took 623.667µs)
[2026-06-16T17:37:47Z] creating resource filesystem.dir::storefront-n1-689qacsi-data
[2026-06-16T17:37:47Z] finished creating resource filesystem.dir::storefront-n1-689qacsi-data (took 780.083µs)
[2026-06-16T17:37:47Z] creating resource filesystem.dir::storefront-n1-689qacsi-configs
[2026-06-16T17:37:47Z] finished creating resource filesystem.dir::storefront-n1-689qacsi-configs (took 326.709µs)
[2026-06-16T17:37:48Z] creating resource swarm.pgbackrest_config::storefront-n2-9ptayhma-backup
[2026-06-16T17:37:48Z] finished creating resource swarm.pgbackrest_config::storefront-n2-9ptayhma-backup (took 1.094833ms)
[2026-06-16T17:37:48Z] creating resource swarm.etcd_creds::storefront-n2-9ptayhma
[2026-06-16T17:37:48Z] creating resource swarm.postgres_certs::storefront-n1-689qacsi
[2026-06-16T17:37:48Z] creating resource swarm.etcd_creds::storefront-n1-689qacsi
[2026-06-16T17:37:48Z] finished creating resource swarm.postgres_certs::storefront-n1-689qacsi (took 10.605375ms)
[2026-06-16T17:37:48Z] finished creating resource swarm.etcd_creds::storefront-n2-9ptayhma (took 77.757333ms)
[2026-06-16T17:37:48Z] finished creating resource swarm.etcd_creds::storefront-n1-689qacsi (took 66.198917ms)
[2026-06-16T17:37:48Z] creating resource swarm.postgres_certs::storefront-n2-9ptayhma
[2026-06-16T17:37:48Z] finished creating resource swarm.postgres_certs::storefront-n2-9ptayhma (took 6.652292ms)
[2026-06-16T17:37:48Z] creating resource swarm.pgbackrest_config::storefront-n1-689qacsi-backup
[2026-06-16T17:37:48Z] finished creating resource swarm.pgbackrest_config::storefront-n1-689qacsi-backup (took 550.167µs)
[2026-06-16T17:37:48Z] creating resource swarm.patroni_config::storefront-n2-9ptayhma
[2026-06-16T17:37:48Z] finished creating resource swarm.patroni_config::storefront-n2-9ptayhma (took 2.413959ms)
[2026-06-16T17:37:48Z] creating resource swarm.patroni_config::storefront-n1-689qacsi
[2026-06-16T17:37:48Z] finished creating resource swarm.patroni_config::storefront-n1-689qacsi (took 4.0715ms)
[2026-06-16T17:37:49Z] creating resource swarm.postgres_service_spec::storefront-n2-9ptayhma
[2026-06-16T17:37:49Z] finished creating resource swarm.postgres_service_spec::storefront-n2-9ptayhma (took 163.542µs)
[2026-06-16T17:37:49Z] creating resource swarm.postgres_service_spec::storefront-n1-689qacsi
[2026-06-16T17:37:49Z] finished creating resource swarm.postgres_service_spec::storefront-n1-689qacsi (took 156.5µs)
[2026-06-16T17:37:49Z] creating resource swarm.check_will_restart::storefront-n2-9ptayhma
[2026-06-16T17:37:49Z] finished creating resource swarm.check_will_restart::storefront-n2-9ptayhma (took 2.011083ms)
[2026-06-16T17:37:49Z] creating resource swarm.check_will_restart::storefront-n1-689qacsi
[2026-06-16T17:37:49Z] finished creating resource swarm.check_will_restart::storefront-n1-689qacsi (took 1.995458ms)
[2026-06-16T17:37:49Z] creating resource swarm.switchover::storefront-n2-9ptayhma
[2026-06-16T17:37:49Z] finished creating resource swarm.switchover::storefront-n2-9ptayhma (took 18.667µs)
[2026-06-16T17:37:49Z] creating resource swarm.switchover::storefront-n1-689qacsi
[2026-06-16T17:37:49Z] finished creating resource swarm.switchover::storefront-n1-689qacsi (took 14.375µs)
[2026-06-16T17:37:50Z] creating resource swarm.postgres_service::storefront-n2-9ptayhma
[2026-06-16T17:37:50Z] creating resource swarm.postgres_service::storefront-n1-689qacsi
[2026-06-16T17:38:10Z] finished creating resource swarm.postgres_service::storefront-n2-9ptayhma (took 20.01505705s)
[2026-06-16T17:38:10Z] finished creating resource swarm.postgres_service::storefront-n1-689qacsi (took 20.01015455s)
[2026-06-16T17:38:10Z] creating resource database.instance::storefront-n2-9ptayhma
[2026-06-16T17:38:10Z] creating resource database.instance::storefront-n1-689qacsi
[2026-06-16T17:38:20Z] finished creating resource database.instance::storefront-n2-9ptayhma (took 10.083895005s)
[2026-06-16T17:38:20Z] finished creating resource database.instance::storefront-n1-689qacsi (took 10.075291838s)
[2026-06-16T17:38:20Z] creating resource database.node::n1
[2026-06-16T17:38:20Z] finished creating resource database.node::n1 (took 71.083µs)
[2026-06-16T17:38:20Z] creating resource database.node::n2
[2026-06-16T17:38:20Z] finished creating resource database.node::n2 (took 47µs)
[2026-06-16T17:38:21Z] creating resource swarm.pgbackrest_stanza::n2
[2026-06-16T17:38:21Z] creating resource swarm.pgbackrest_stanza::n1
[2026-06-16T17:38:21Z] finished creating resource swarm.pgbackrest_stanza::n1 (took 396.027042ms)
[2026-06-16T17:38:21Z] finished creating resource swarm.pgbackrest_stanza::n2 (took 425.849416ms)
[2026-06-16T17:38:21Z] creating resource monitor.instance::storefront-n2-9ptayhma
[2026-06-16T17:38:21Z] finished creating resource monitor.instance::storefront-n2-9ptayhma (took 15.692083ms)
[2026-06-16T17:38:21Z] creating resource monitor.instance::storefront-n1-689qacsi
[2026-06-16T17:38:21Z] finished creating resource monitor.instance::storefront-n1-689qacsi (took 11.961959ms)
[2026-06-16T17:38:22Z] creating resource database.postgres_database::n2:storefront
[2026-06-16T17:38:22Z] creating resource database.postgres_database::n1:storefront
[2026-06-16T17:38:22Z] finished creating resource database.postgres_database::n2:storefront (took 313.239625ms)
[2026-06-16T17:38:22Z] finished creating resource database.postgres_database::n1:storefront (took 301.758292ms)
[2026-06-16T17:38:22Z] creating resource database.replication_slot::n2:n1:storefront
[2026-06-16T17:38:22Z] finished creating resource database.replication_slot::n2:n1:storefront (took 40.084µs)
[2026-06-16T17:38:22Z] creating resource database.replication_slot::n1:n2:storefront
[2026-06-16T17:38:22Z] finished creating resource database.replication_slot::n1:n2:storefront (took 10.333µs)
[2026-06-16T17:38:23Z] creating resource database.subscription::n1:n2:storefront
[2026-06-16T17:38:23Z] creating resource database.subscription::n2:n1:storefront
[2026-06-16T17:38:23Z] finished creating resource database.subscription::n1:n2:storefront (took 39.227125ms)
[2026-06-16T17:38:23Z] finished creating resource database.subscription::n2:n1:storefront (took 20.928417ms)
database entity storefront task 019ed182-b273-7f09-b9e9-3600ad15b629 completed
➜ control-plane git:(main)
➜ control-plane git:(main)
➜ control-plane git:(main)
➜ control-plane git:(main) cp1-req backup-database-node storefront n1 '{ "type": "full" }' | cp-follow-task
{
"task": {
"created_at": "2026-06-16T17:41:30Z",
"database_id": "storefront",
"entity_id": "storefront",
"scope": "database",
"status": "pending",
"task_id": "019ed186-2693-759a-a1bc-60bb5628e8a5",
"type": "node_backup"
}
}
[2026-06-16T17:41:36Z] P00 INFO: backup command begin 2.58.0: --config=/opt/pgedge/configs/pgbackrest.backup.conf --exec-id=512-6e85dfdc --lock-path=/tmp/pgbackrest/storefront-n1-689qacsi --log-level-console=info --no-log-timestamp --pg1-path=/opt/pgedge/data/pgdata --pg1-user=pgedge --repo1-cipher-type=none --repo1-path=/backups/databases/storefront/original/n1 --repo1-retention-full=7 --repo1-retention-full-type=time --repo1-type=posix --stanza=db --start-fast --type=full
[2026-06-16T17:41:36Z] P00 INFO: execute backup start: backup begins after the requested immediate checkpoint completes
[2026-06-16T17:41:36Z] P00 INFO: backup start archive = 000000010000000000000003, lsn = 0/3000028
[2026-06-16T17:41:36Z] P00 INFO: check archive for prior segment 000000010000000000000002
[2026-06-16T17:41:37Z] P00 INFO: execute backup stop and wait for all WAL segments to archive
[2026-06-16T17:41:38Z] P00 INFO: backup stop archive = 000000010000000000000003, lsn = 0/3000120
[2026-06-16T17:41:38Z] P00 INFO: check archive for segment(s) 000000010000000000000003:000000010000000000000003
[2026-06-16T17:41:38Z] P00 INFO: new backup label = 20260616-174136F
[2026-06-16T17:41:38Z] P00 INFO: full backup size = 30.3MB, file total = 1334
[2026-06-16T17:41:38Z] P00 INFO: backup command end: completed successfully
[2026-06-16T17:41:38Z] P00 INFO: expire command begin 2.58.0: --config=/opt/pgedge/configs/pgbackrest.backup.conf --exec-id=512-6e85dfdc --lock-path=/tmp/pgbackrest/storefront-n1-689qacsi --log-level-console=info --no-log-timestamp --repo1-cipher-type=none --repo1-path=/backups/databases/storefront/original/n1 --repo1-retention-full=7 --repo1-retention-full-type=time --repo1-type=posix --stanza=db
[2026-06-16T17:41:38Z] P00 INFO: repo1: time-based archive retention not met - archive logs will not be expired
[2026-06-16T17:41:38Z] P00 INFO: expire command end: completed successfully
database entity storefront task 019ed186-2693-759a-a1bc-60bb5628e8a5 completed
➜ control-plane git:(main)
➜ control-plane git:(fix/PLAT-610/update-global-patroni-params-swarm) cp1-req backup-database-node storefront n1 '{ "type": "full" }' | cp-follow-task
{
"task": {
"created_at": "2026-06-16T17:43:24Z",
"database_id": "storefront",
"entity_id": "storefront",
"scope": "database",
"status": "pending",
"task_id": "019ed187-e42d-7544-8941-1060dbce11e1",
"type": "node_backup"
}
}
[2026-06-16T17:43:30Z] P00 INFO: backup command begin 2.58.0: --config=/opt/pgedge/configs/pgbackrest.backup.conf --exec-id=755-aa0a6924 --lock-path=/tmp/pgbackrest/storefront-n1-689qacsi --log-level-console=info --no-log-timestamp --pg1-path=/opt/pgedge/data/pgdata --pg1-user=pgedge --repo1-cipher-type=none --repo1-path=/backups/databases/storefront/original/n1 --repo1-retention-full=7 --repo1-retention-full-type=time --repo1-type=posix --stanza=db --start-fast --type=full
[2026-06-16T17:43:30Z] P00 INFO: execute backup start: backup begins after the requested immediate checkpoint completes
[2026-06-16T17:43:30Z] P00 INFO: backup start archive = 000000010000000000000005, lsn = 0/5000028
[2026-06-16T17:43:30Z] P00 INFO: check archive for prior segment 000000010000000000000004
[2026-06-16T17:43:32Z] P00 INFO: execute backup stop and wait for all WAL segments to archive
[2026-06-16T17:43:32Z] P00 INFO: backup stop archive = 000000010000000000000005, lsn = 0/5000120
[2026-06-16T17:43:32Z] P00 INFO: check archive for segment(s) 000000010000000000000005:000000010000000000000005
[2026-06-16T17:43:32Z] P00 INFO: new backup label = 20260616-174330F
[2026-06-16T17:43:32Z] P00 INFO: full backup size = 30.3MB, file total = 1334
[2026-06-16T17:43:32Z] P00 INFO: backup command end: completed successfully
[2026-06-16T17:43:32Z] P00 INFO: expire command begin 2.58.0: --config=/opt/pgedge/configs/pgbackrest.backup.conf --exec-id=755-aa0a6924 --lock-path=/tmp/pgbackrest/storefront-n1-689qacsi --log-level-console=info --no-log-timestamp --repo1-cipher-type=none --repo1-path=/backups/databases/storefront/original/n1 --repo1-retention-full=7 --repo1-retention-full-type=time --repo1-type=posix --stanza=db
[2026-06-16T17:43:32Z] P00 INFO: repo1: time-based archive retention not met - archive logs will not be expired
[2026-06-16T17:43:32Z] P00 INFO: expire command end: completed successfully
database entity storefront task 019ed187-e42d-7544-8941-1060dbce11e1 completed
➜ control-plane git:(fix/PLAT-610/update-global-patroni-params-swarm) cp1-req get-database storefront \
| jq '{spec}' \
| cp1-req update-database storefront \
| cp-follow-task
{
"database": {
"created_at": "2026-06-16T17:37:44Z",
"id": "storefront",
"instances": [
{
"created_at": "2026-06-16T17:37:47Z",
"host_id": "host-1",
"id": "storefront-n1-689qacsi",
"node_name": "n1",
"postgres": {
"patroni_state": "running",
"role": "primary",
"version": "17.9"
},
"spock": {
"read_only": "off",
"subscriptions": [
{
"name": "sub_n2_n1",
"provider_node": "n2",
"status": "replicating"
}
],
"version": "5.0.6"
},
"state": "available",
"status_updated_at": "2026-06-16T17:44:25Z",
"updated_at": "2026-06-16T17:43:32Z"
},
{
"created_at": "2026-06-16T17:37:47Z",
"host_id": "host-2",
"id": "storefront-n2-9ptayhma",
"node_name": "n2",
"postgres": {
"patroni_state": "running",
"role": "primary",
"version": "17.9"
},
"spock": {
"read_only": "off",
"subscriptions": [
{
"name": "sub_n1_n2",
"provider_node": "n1",
"status": "replicating"
}
],
"version": "5.0.6"
},
"state": "available",
"status_updated_at": "2026-06-16T17:44:25Z",
"updated_at": "2026-06-16T17:42:15Z"
}
],
"spec": {
"backup_config": {
"repositories": [
{
"base_path": "/backups",
"id": "original",
"type": "posix"
}
]
},
"database_name": "storefront",
"database_users": [
{
"attributes": [
"SUPERUSER",
"LOGIN"
],
"db_owner": true,
"username": "admin"
}
],
"nodes": [
{
"host_ids": [
"host-1"
],
"name": "n1"
},
{
"host_ids": [
"host-2"
],
"name": "n2"
}
],
"orchestrator_opts": {
"swarm": {
"extra_volumes": [
{
"destination_path": "/backups",
"host_path": "/tmp/PLAT-610"
}
]
}
},
"postgres_version": "17.9",
"spock_version": "5"
},
"state": "modifying",
"updated_at": "2026-06-16T17:44:27Z"
},
"task": {
"created_at": "2026-06-16T17:44:27Z",
"database_id": "storefront",
"entity_id": "storefront",
"scope": "database",
"status": "pending",
"task_id": "019ed188-da66-7e75-8d18-2312d20346c9",
"type": "update"
}
}
[2026-06-16T17:44:29Z] refreshing current state
[2026-06-16T17:44:38Z] finished refreshing current state (took 9.594426547s)
[2026-06-16T17:44:40Z] updating resource common.pgbackrest_config::storefront-n1-689qacsi-backup
[2026-06-16T17:44:40Z] finished updating resource common.pgbackrest_config::storefront-n1-689qacsi-backup (took 760.542µs)
[2026-06-16T17:44:41Z] updating resource swarm.patroni_config::storefront-n1-689qacsi
[2026-06-16T17:44:50Z] finished updating resource swarm.patroni_config::storefront-n1-689qacsi (took 8.862926254s)
[2026-06-16T17:44:50Z] updating resource swarm.postgres_service_spec::storefront-n1-689qacsi
[2026-06-16T17:44:50Z] finished updating resource swarm.postgres_service_spec::storefront-n1-689qacsi (took 157.208µs)
[2026-06-16T17:44:50Z] updating resource swarm.check_will_restart::storefront-n1-689qacsi
[2026-06-16T17:44:50Z] finished updating resource swarm.check_will_restart::storefront-n1-689qacsi (took 4.541625ms)
[2026-06-16T17:44:50Z] updating resource swarm.switchover::storefront-n1-689qacsi
[2026-06-16T17:44:50Z] finished updating resource swarm.switchover::storefront-n1-689qacsi (took 139.25µs)
[2026-06-16T17:44:51Z] updating resource swarm.postgres_service::storefront-n1-689qacsi
[2026-06-16T17:44:56Z] finished updating resource swarm.postgres_service::storefront-n1-689qacsi (took 5.017397169s)
[2026-06-16T17:44:56Z] updating resource database.instance::storefront-n1-689qacsi
[2026-06-16T17:45:12Z] finished updating resource database.instance::storefront-n1-689qacsi (took 16.341339632s)
[2026-06-16T17:45:13Z] updating resource database.node::n1
[2026-06-16T17:45:13Z] finished updating resource database.node::n1 (took 61.75µs)
[2026-06-16T17:45:13Z] updating resource monitor.instance::storefront-n1-689qacsi
[2026-06-16T17:45:13Z] finished updating resource monitor.instance::storefront-n1-689qacsi (took 16.260667ms)
[2026-06-16T17:45:13Z] updating resource common.pgbackrest_stanza::n1
[2026-06-16T17:45:13Z] updating resource database.postgres_database::n1:storefront
[2026-06-16T17:45:13Z] finished updating resource database.postgres_database::n1:storefront (took 28.9685ms)
[2026-06-16T17:45:13Z] finished updating resource common.pgbackrest_stanza::n1 (took 212.427625ms)
[2026-06-16T17:45:14Z] updating resource database.replication_slot::n1:n2:storefront
[2026-06-16T17:45:14Z] finished updating resource database.replication_slot::n1:n2:storefront (took 17.041µs)
[2026-06-16T17:45:14Z] updating resource database.subscription::n2:n1:storefront
[2026-06-16T17:45:14Z] updating resource database.subscription::n1:n2:storefront
[2026-06-16T17:45:14Z] finished updating resource database.subscription::n1:n2:storefront (took 7.1165ms)
[2026-06-16T17:45:14Z] finished updating resource database.subscription::n2:n1:storefront (took 12.722084ms)
[2026-06-16T17:45:15Z] updating resource common.pgbackrest_config::storefront-n2-9ptayhma-backup
[2026-06-16T17:45:15Z] finished updating resource common.pgbackrest_config::storefront-n2-9ptayhma-backup (took 602.292µs)
[2026-06-16T17:45:15Z] updating resource swarm.patroni_config::storefront-n2-9ptayhma
[2026-06-16T17:45:20Z] finished updating resource swarm.patroni_config::storefront-n2-9ptayhma (took 4.465855918s)
[2026-06-16T17:45:20Z] updating resource swarm.postgres_service_spec::storefront-n2-9ptayhma
[2026-06-16T17:45:20Z] finished updating resource swarm.postgres_service_spec::storefront-n2-9ptayhma (took 184.208µs)
[2026-06-16T17:45:20Z] updating resource swarm.check_will_restart::storefront-n2-9ptayhma
[2026-06-16T17:45:20Z] finished updating resource swarm.check_will_restart::storefront-n2-9ptayhma (took 4.197291ms)
[2026-06-16T17:45:20Z] updating resource swarm.switchover::storefront-n2-9ptayhma
[2026-06-16T17:45:20Z] finished updating resource swarm.switchover::storefront-n2-9ptayhma (took 101.417µs)
[2026-06-16T17:45:21Z] updating resource swarm.postgres_service::storefront-n2-9ptayhma
[2026-06-16T17:45:26Z] finished updating resource swarm.postgres_service::storefront-n2-9ptayhma (took 5.017264586s)
[2026-06-16T17:45:26Z] updating resource database.instance::storefront-n2-9ptayhma
[2026-06-16T17:45:43Z] finished updating resource database.instance::storefront-n2-9ptayhma (took 16.52490459s)
[2026-06-16T17:45:43Z] updating resource database.node::n2
[2026-06-16T17:45:43Z] finished updating resource database.node::n2 (took 323.291µs)
[2026-06-16T17:45:43Z] updating resource monitor.instance::storefront-n2-9ptayhma
[2026-06-16T17:45:43Z] finished updating resource monitor.instance::storefront-n2-9ptayhma (took 14.51175ms)
[2026-06-16T17:45:43Z] updating resource database.postgres_database::n2:storefront
[2026-06-16T17:45:43Z] updating resource common.pgbackrest_stanza::n2
[2026-06-16T17:45:43Z] finished updating resource database.postgres_database::n2:storefront (took 35.02925ms)
[2026-06-16T17:45:43Z] finished updating resource common.pgbackrest_stanza::n2 (took 205.802959ms)
[2026-06-16T17:45:44Z] updating resource database.replication_slot::n2:n1:storefront
[2026-06-16T17:45:44Z] finished updating resource database.replication_slot::n2:n1:storefront (took 14.417µs)
[2026-06-16T17:45:44Z] updating resource database.subscription::n2:n1:storefront
[2026-06-16T17:45:44Z] updating resource database.subscription::n1:n2:storefront
[2026-06-16T17:45:44Z] finished updating resource database.subscription::n2:n1:storefront (took 9.080042ms)
[2026-06-16T17:45:44Z] finished updating resource database.subscription::n1:n2:storefront (took 12.552083ms)
database entity storefront task 019ed188-da66-7e75-8d18-2312d20346c9 completed
➜ control-plane git:(fix/PLAT-610/update-global-patroni-params-swarm) cp1-req backup-database-node storefront n1 '{ "type": "full" }' | cp-follow-task
{
"task": {
"created_at": "2026-06-16T17:46:14Z",
"database_id": "storefront",
"entity_id": "storefront",
"scope": "database",
"status": "pending",
"task_id": "019ed18a-7bf7-77ca-8fb3-45c4dbe60169",
"type": "node_backup"
}
}
[2026-06-16T17:46:20Z] P00 INFO: backup command begin 2.58.0: --config=/opt/pgedge/configs/pgbackrest.backup.conf --exec-id=1180-883ded32 --lock-path=/tmp/pgbackrest/storefront-n1-689qacsi --log-level-console=info --no-log-timestamp --pg1-path=/opt/pgedge/data/pgdata --pg1-port=5432 --pg1-user=pgedge --repo1-cipher-type=none --repo1-path=/backups/databases/storefront/original/n1 --repo1-retention-full=7 --repo1-retention-full-type=time --repo1-type=posix --stanza=db --start-fast --type=full
[2026-06-16T17:46:20Z] P00 INFO: execute backup start: backup begins after the requested immediate checkpoint completes
[2026-06-16T17:46:20Z] P00 INFO: backup start archive = 000000010000000000000009, lsn = 0/9000028
[2026-06-16T17:46:20Z] P00 INFO: check archive for prior segment 000000010000000000000008
[2026-06-16T17:46:22Z] P00 INFO: execute backup stop and wait for all WAL segments to archive
[2026-06-16T17:46:22Z] P00 INFO: backup stop archive = 000000010000000000000009, lsn = 0/9000810
[2026-06-16T17:46:22Z] P00 INFO: check archive for segment(s) 000000010000000000000009:000000010000000000000009
[2026-06-16T17:46:22Z] P00 INFO: new backup label = 20260616-174620F
[2026-06-16T17:46:22Z] P00 INFO: full backup size = 30.3MB, file total = 1334
[2026-06-16T17:46:22Z] P00 INFO: backup command end: completed successfully
[2026-06-16T17:46:22Z] P00 INFO: expire command begin 2.58.0: --config=/opt/pgedge/configs/pgbackrest.backup.conf --exec-id=1180-883ded32 --lock-path=/tmp/pgbackrest/storefront-n1-689qacsi --log-level-console=info --no-log-timestamp --repo1-cipher-type=none --repo1-path=/backups/databases/storefront/original/n1 --repo1-retention-full=7 --repo1-retention-full-type=time --repo1-type=posix --stanza=db
[2026-06-16T17:46:22Z] P00 INFO: repo1: time-based archive retention not met - archive logs will not be expired
[2026-06-16T17:46:22Z] P00 INFO: expire command end: completed successfully
database entity storefront task 019ed18a-7bf7-77ca-8fb3-45c4dbe60169 completed
➜ control-plane git:(fix/PLAT-610/update-global-patroni-params-swarm)
Extracts the Postgres and Patroni container ports into constants. These represent the ports exposed on the container. These are separate from the published ports, which are user-specified and optional. PLAT-610
This change now applies to all running database clusters. PLAT-610
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When we implemented the systemd orchestrator, we identified several resources that could be shared between orchestrator implementations. For expedience and to avoid conflicts, we added duplicate implementations of those resources to the
orchestrator/commonpackage without refactoring theorchestrator/swarmpackage to use them.This PR addresses that tech debt by:
orchestrator/swarmorchestrator/commonimplementationsChanges
This PR is split into atomic commits. The most notable of these are:
feat: use common resources in swarm- switches to theorchestrator/commonresources in theorchestrator/swarmpackage.feat: v1.2.0 state migration- adds the state migration for existing databasestest: add heartbeat utility for e2es- adds a heartbeat process to the add node testsTesting
Our E2Es cover new database sufficiently well. We need to manually test the migration process for existing databases
PLAT-610