Add tag-based image resolution to the controller (milestone 5a)#75
Conversation
|
@jlebon I opened this draft PR to discuss with you a couple of design points, and I'd love to hear you opinion. First, how to we expose the time when the tag resolution will take place. In this version, the users see when the next resolution will take place in the bootc pool status in the field Secondly, the polling time interval is configurable on the operator side by a flag. IMO, this is something the cluster admin needs to configure. But an alternative could be that it is the user who decides. In this case, we could expose the polling time under the pool spec. Additionally, I added the supported for untrusted registry if the cluster admin configures the operator with Please, let me know what you think |
|
We also need to move away from alicefr/bink repository and include the fix bootc-dev/bink#71 |
➡️ #76 |
b9c79a8 to
e3d8661
Compare
Seems fine to me.
I touched on this a bit in another discussion, but yeah I think there might be a story for a distinction between a cluster admin persona and a "bootc-operator admin" persona. I'm thinking of a managed K8s or HyperShift-like setup, where you may have a user own nodes and schedule their workloads on their nodes but not have full cluster access. But AFAIK actual HyperShift hosted clusters don't actually have edit access to their nodes. The nodes are managed via NodePools, which reside at the management cluster level. And in the case of a managed K8s cluster like EKS, you do actually have full access to the cluster. So, there might be distinction to draw there in the future, but it's not clear to me yet how prevalent it is in the real world. So I would for now assume that cluster admin = bootc-operator admin. To circle back to the topic at hand though, I think let's leave it as a flag option for now.
Seems fine to me! (I assume you had to do this for testing?) Aside but I think at some point, we should also support configuring all operator knobs via a configmap instead. That makes it easier to separate manifest definitions from configs (related to #14). The configmap keys and CLI flags should just match clearly to each other. |
jlebon
left a comment
There was a problem hiding this comment.
Initial pass but looks sane overall!
|
Some of the later commits could be squashed. |
aa943dd to
fbb600c
Compare
|
I solved some of the comments I resolved. Squashed some of the latest commits. But I have added the commit 2eef5ec which introduces the helper functions and resolve in a more elegant way the patching of the option in the controller. Previously, it was completely replacing all the flags for the controller. Now, it replaces existing flags with the flags that the tests needs. In this way, if the there are unrelated flags, they are preserved and tested |
48e3c51 to
9294316
Compare
jlebon
left a comment
There was a problem hiding this comment.
LGTM overall. Mostly minor nits. Fine to merge as is but I guess you have to rebase anyway for conflicts. :)
Also, let's checkmark the completed milestone steps in the implementation plan.
Nice work!
| ) | ||
|
|
||
| func TestResolveValidTag(t *testing.T) { | ||
| tags := []string{"latest", "v1.0"} |
There was a problem hiding this comment.
Sorry, what I mean here is to test without any tag at all (i.e. implied :latest).
| srv := httptest.NewServer(registry.New()) | ||
| defer srv.Close() | ||
|
|
||
| // Build a minimal OCI image (empty base + one random layer) to push. | ||
| layer, err := random.Layer(256, types.OCILayer) | ||
| g.Expect(err).NotTo(HaveOccurred()) | ||
| img, err := mutate.AppendLayers(empty.Image, layer) | ||
| g.Expect(err).NotTo(HaveOccurred()) | ||
|
|
||
| ref, err := name.ParseReference(fmt.Sprintf("%s/test/image:%s", srv.Listener.Addr().String(), tag)) | ||
| g.Expect(err).NotTo(HaveOccurred()) | ||
| g.Expect(remote.Write(ref, img)).To(Succeed()) | ||
|
|
||
| want, err := img.Digest() | ||
| g.Expect(err).NotTo(HaveOccurred()) | ||
|
|
||
| resolver := &GGCRResolver{} | ||
| got, err := resolver.Resolve(context.Background(), ref.String()) | ||
| g.Expect(err).NotTo(HaveOccurred()) | ||
| g.Expect(got).To(Equal(want.String())) |
There was a problem hiding this comment.
Does all this have to happen on every iteration? Feels like some of the setup stuff could be lifted out of the loop.
| return resolveResult, nil | ||
| } | ||
|
|
||
| if pool.Status.TargetDigest == "" { |
There was a problem hiding this comment.
The comment you had here before was good (re. first tag resolution failure).
| return newInvalidSpecError(fmt.Sprintf("image ref %q has no digest (tag resolution not yet supported)", pool.Spec.Image.Ref)) | ||
| if ok { | ||
| pool.Status.TargetDigest = digested.Digest().String() | ||
| // Rest the NextTagResolutionTime in case we pass from a tag referenced image to a digested one. |
| func TestResolveTargetDigestClearsNextTagResolutionTime(t *testing.T) { | ||
| g := NewWithT(t) | ||
|
|
||
| pool := testutil.NewPool("tag-to-digest", testImageDigestRefA) | ||
| next := metav1.NewTime(time.Now().Add(time.Hour)) | ||
| pool.Status.NextTagResolutionTime = &next | ||
|
|
||
| r := &BootcNodePoolReconciler{} | ||
| _, err := r.resolveTargetDigest(context.Background(), pool) | ||
| g.Expect(err).NotTo(HaveOccurred()) | ||
| g.Expect(pool.Status.TargetDigest).To(Equal(testDigestA)) | ||
| g.Expect(pool.Status.NextTagResolutionTime).To(BeNil()) | ||
| } |
There was a problem hiding this comment.
Minor: this test is too artificial IMO. I would just drop it, but fine to keep too.
| return d.DigestStr(), nil | ||
| } | ||
| desc, err := remote.Get(parsed, remote.WithContext(ctx)) | ||
| if err != nil && r.AllowInsecure { |
There was a problem hiding this comment.
Minor: is there a specific typed error we could check for here instead of a blanket retry?
Introduce GGCRResolver to resolve container image tags to digests via remote.Get. Assisted-by: AI Signed-off-by: Alice Frosi <afrosi@redhat.com>
Test tag resolution against GGCR's in-memory registry, invalid reference parsing, and unreachable registry errors. Assisted-by: AI Signed-off-by: Alice Frosi <afrosi@redhat.com>
New Degraded condition reason for when the controller fails to resolve a tag from the container registry. Assisted-by: AI Signed-off-by: Alice Frosi <afrosi@redhat.com>
Tracks when the controller will next resolve a tag-based image ref to a digest. Assisted-by: AI Signed-off-by: Alice Frosi <afrosi@redhat.com>
Add TagResolver and TagResolutionInterval to the reconciler struct. The TagResolver defines an interface to solve the image digest. Rewrite resolveTargetDigest to resolve tag refs via the registry with periodic re-resolution, and set Degraded/RegistryError on failure. Wire GGCRResolver and --tag-resolution-interval flag in main.go. Assisted-by: AI Signed-off-by: Alice Frosi <afrosi@redhat.com>
Allow falling back to HTTP when resolving tag-based image refs against registries that do not serve TLS. Assisted-by: AI Signed-off-by: Alice Frosi <afrosi@redhat.com>
Introduce helper functions for the image handling. The NodeImageTagRef returns the tag-based image reference for the seeded node image, for use in tag resolution e2e tests. The RetagImage takes an image and retags it. Assisted-by: AI Signed-off-by: Alice Frosi <afrosi@redhat.com>
The test TestTagResolution verifies that controller is triggering a bootc switch if the image reference by a tag changes in the registry. The test retags the updated image with the same referenced tag. Additionally, the polling time for the controller is reduce the polling time and it configures the insecure registries. Assisted-by: AI Signed-off-by: Alice Frosi <afrosi@redhat.com>
TestTagResolution adds ~5m to the suite, pushing past the old 10m limit. Signed-off-by: Alice Frosi <afrosi@redhat.com>
Allow insecure registries and reduce the polling time for the registry resolution tag for the controller. This allows the controller to talk to the bink registry and reduce the polling time to speed up the tests. Signed-off-by: Alice Frosi <afrosi@redhat.com>
Signed-off-by: Alice Frosi <afrosi@redhat.com>
Pools can now reference node images by tag (e.g. registry.example.com/node:latest) instead of requiring a pinned digest. The controller periodically resolves tags to digests using go-containerregistry and updates the pool status with the resolved digest.
The resolution interval is configurable via --tag-resolution-interval (default 5m). Resolution failures are surfaced as a PoolRegistryError degraded condition. An
--allow-insecure-registryflag enables fallback to HTTP for registries that do not serve TLS.