Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -336,7 +336,7 @@ local-stovepipe-gateway-start: build-stovepipe-gateway-linux ## Start Stovepipe

mocks: ## Generate mock files using mockgen
@echo "Generating mocks..."
@$(BAZEL) run @rules_go//go -- generate ./submitqueue/extension/storage/... ./submitqueue/extension/buildrunner/... ./submitqueue/extension/changeprovider/... ./extension/counter/... ./extension/messagequeue/... ./submitqueue/extension/queueconfig/... ./submitqueue/extension/mergechecker/... ./submitqueue/extension/pusher/... ./submitqueue/extension/scorer/... ./submitqueue/extension/conflict/... ./submitqueue/core/consumer/...
@$(BAZEL) run @rules_go//go -- generate ./submitqueue/extension/storage/... ./submitqueue/extension/buildrunner/... ./submitqueue/extension/changeprovider/... ./extension/counter/... ./extension/messagequeue/... ./submitqueue/extension/queueconfig/... ./submitqueue/extension/mergechecker/... ./submitqueue/extension/pusher/... ./submitqueue/extension/scorer/... ./submitqueue/extension/conflict/... ./submitqueue/core/consumer/... ./submitqueue/core/changeset/...
@echo "Mocks generated successfully!"

proto: ## Generate protobuf files from .proto definitions
Expand Down
28 changes: 28 additions & 0 deletions submitqueue/core/changeset/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
load("@rules_go//go:def.bzl", "go_library", "go_test")

go_library(
name = "changeset",
srcs = [
"changeset.go",
"resolver.go",
],
importpath = "github.com/uber/submitqueue/submitqueue/core/changeset",
visibility = ["//visibility:public"],
deps = [
"//submitqueue/entity",
"//submitqueue/extension/storage",
],
)

go_test(
name = "changeset_test",
srcs = ["resolver_test.go"],
embed = [":changeset"],
deps = [
"//submitqueue/entity",
"//submitqueue/extension/storage/mock",
"@com_github_stretchr_testify//assert",
"@com_github_stretchr_testify//require",
"@org_uber_go_mock//gomock",
],
)
18 changes: 18 additions & 0 deletions submitqueue/core/changeset/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# changeset

`changeset` resolves batch identity into the changes a batch contains. It is the single place the orchestrator walks batch → requests → changes, consolidating the resolution the build, merge, and score controllers each performed privately.

## Why it exists

A `Batch` is a thin reference entity: it carries the IDs of the requests it contains, not their changes. Decision and action extensions (the scorer, build runner, pusher, and future detail-aware conflict analyzers) are handed that identity and resolve the granular content themselves through an injected `Resolver`, rather than depending on a controller to pre-resolve and pass the data in. The resolver depends only on the two resolution-target stores — the request store (to walk a batch's contained requests) and the change store (to attach provider details) — and nothing else.

## Two fidelities

The resolver offers the same walk at two levels of detail, and both preserve batch boundaries — neither flattens across batches, so a caller that wants a flat list flattens the result itself:

- The raw view returns each batch's contained changes as URIs only, one group per input batch, in input order. It performs no change-store read. The build stage uses it for base and head inputs; the merge stage uses it for the pusher.
- The detailed view returns a single batch's normalized, batch-level changes: one entry per claimed URI, each carrying the provider details recorded in the change store, aggregated across every request in the batch. Because the change store returns rows for every request that ever claimed a URI, the resolver selects the row owned by the requesting request. The score stage uses it, as will any analyzer that needs changed-file or line-count facts.

## Testing

A programmable in-memory fake lives in `fake/`: seed per-batch results and inject errors without a real store. A generated mock lives in `mock/` for tests that assert on exact call expectations. Extensions that take a `Resolver` can be exercised against either.
51 changes: 51 additions & 0 deletions submitqueue/core/changeset/changeset.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// Copyright (c) 2025 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Package changeset resolves batch identity into the changes a batch contains.
// It is the single place the orchestrator walks batch -> requests -> changes,
// consolidating what the build, merge, and score controllers each did privately.
// Decision/action extensions (scorer, buildrunner, pusher, and future
// detail-aware conflict analyzers) take thin identity entities and resolve their
// granular content through an injected Resolver instead of being handed
// pre-resolved data by a controller.
package changeset

//go:generate mockgen -source=changeset.go -destination=mock/changeset_mock.go -package=mock

import (
"context"

"github.com/uber/submitqueue/submitqueue/entity"
)

// Resolver turns batch identity into the changes the batch contains. Both methods
// operate on a single batch — callers with several batches (a build's base, a
// merge train) loop and keep the per-batch boundary by holding a slice per batch.
// The two methods differ only in fidelity: ChangesForBatch is the cheap URI-only
// view; DetailedForBatch reads the change store for provider details.
type Resolver interface {
// ChangesForBatch resolves a batch's contained requests into their raw
// changes (URIs only; no change-store read), in batch.Contains order. A batch
// with no requests yields an empty slice. Used by the build (base/head) and
// merge stages.
ChangesForBatch(ctx context.Context, batch entity.Batch) ([]entity.Change, error)

// DetailedForBatch resolves a batch into its normalized, batch-level view:
// one entity.ChangeInfo per claimed URI (URI plus the provider details read
// from the change store), aggregated across every request in the batch. For
// each URI it selects the record owned by the request, since the change store
// returns rows for all requests that ever claimed the URI. Used by the score
// stage and detail-aware analyzers.
DetailedForBatch(ctx context.Context, batch entity.Batch) (entity.BatchChanges, error)
}
23 changes: 23 additions & 0 deletions submitqueue/core/changeset/fake/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
load("@rules_go//go:def.bzl", "go_library", "go_test")

go_library(
name = "fake",
srcs = ["fake.go"],
importpath = "github.com/uber/submitqueue/submitqueue/core/changeset/fake",
visibility = ["//visibility:public"],
deps = [
"//submitqueue/core/changeset",
"//submitqueue/entity",
],
)

go_test(
name = "fake_test",
srcs = ["fake_test.go"],
embed = [":fake"],
deps = [
"//submitqueue/entity",
"@com_github_stretchr_testify//assert",
"@com_github_stretchr_testify//require",
],
)
86 changes: 86 additions & 0 deletions submitqueue/core/changeset/fake/fake.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
// Copyright (c) 2025 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Package fake provides an in-memory changeset.Resolver for tests and examples.
// Seed per-batch results with Set (raw changes) and SetDetailed (detailed view),
// keyed by batch ID; the resolver serves what was seeded. A batch with no seeded
// entry resolves to empty rather than an error, matching a batch whose requests
// carry no changes. FailWith injects an error on every call to exercise the error
// path without a real store. It is intended for examples and tests only, never
// production.
package fake

import (
"context"

"github.com/uber/submitqueue/submitqueue/core/changeset"
"github.com/uber/submitqueue/submitqueue/entity"
)

// Resolver is a programmable in-memory changeset.Resolver.
type Resolver struct {
changes map[string][]entity.Change
detailed map[string]entity.BatchChanges
err error
}

// New returns an empty fake Resolver. Seed it with Set / SetDetailed.
func New() *Resolver {
return &Resolver{
changes: map[string][]entity.Change{},
detailed: map[string]entity.BatchChanges{},
}
}

// Set seeds the raw changes returned by Changes for the given batch ID.
func (r *Resolver) Set(batchID string, changes ...entity.Change) *Resolver {
r.changes[batchID] = changes
return r
}

// SetDetailed seeds the detailed view returned by Detailed for the given batch ID.
func (r *Resolver) SetDetailed(batchID string, detailed entity.BatchChanges) *Resolver {
r.detailed[batchID] = detailed
return r
}

// FailWith makes every Changes and Detailed call return err.
func (r *Resolver) FailWith(err error) *Resolver {
r.err = err
return r
}

// ChangesForBatch returns the seeded raw changes for the batch, in seeded order.
// An unseeded batch resolves to a nil slice.
func (r *Resolver) ChangesForBatch(_ context.Context, batch entity.Batch) ([]entity.Change, error) {
if r.err != nil {
return nil, r.err
}
return r.changes[batch.ID], nil
}

// DetailedForBatch returns the seeded detailed view for the batch. An unseeded
// batch resolves to an empty entity.BatchChanges carrying the batch's identity.
func (r *Resolver) DetailedForBatch(_ context.Context, batch entity.Batch) (entity.BatchChanges, error) {
if r.err != nil {
return entity.BatchChanges{}, r.err
}
if detailed, ok := r.detailed[batch.ID]; ok {
return detailed, nil
}
return entity.BatchChanges{BatchID: batch.ID, Queue: batch.Queue}, nil
}

// ensure the fake satisfies the interface.
var _ changeset.Resolver = (*Resolver)(nil)
70 changes: 70 additions & 0 deletions submitqueue/core/changeset/fake/fake_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
// Copyright (c) 2025 Uber Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package fake

import (
"context"
"errors"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"

"github.com/uber/submitqueue/submitqueue/entity"
)

func TestResolverChanges(t *testing.T) {
r := New().
Set("q/batch/1", entity.Change{URIs: []string{"u1"}}).
Set("q/batch/2", entity.Change{URIs: []string{"u2"}}, entity.Change{URIs: []string{"u3"}})

got, err := r.ChangesForBatch(context.Background(), entity.Batch{ID: "q/batch/2"})
require.NoError(t, err)
assert.Equal(t, []entity.Change{{URIs: []string{"u2"}}, {URIs: []string{"u3"}}}, got)

unseeded, err := r.ChangesForBatch(context.Background(), entity.Batch{ID: "q/batch/unseeded"})
require.NoError(t, err)
assert.Empty(t, unseeded)
}

func TestResolverDetailed(t *testing.T) {
want := entity.BatchChanges{
BatchID: "q/batch/1",
Queue: "q",
Changes: []entity.ChangeInfo{{URI: "u1"}},
}
r := New().SetDetailed("q/batch/1", want)

got, err := r.DetailedForBatch(context.Background(), entity.Batch{ID: "q/batch/1", Queue: "q"})
require.NoError(t, err)
assert.Equal(t, want, got)
}

func TestResolverDetailedUnseeded(t *testing.T) {
got, err := New().DetailedForBatch(context.Background(), entity.Batch{ID: "q/batch/9", Queue: "q"})
require.NoError(t, err)
assert.Equal(t, entity.BatchChanges{BatchID: "q/batch/9", Queue: "q"}, got)
}

func TestResolverFailWith(t *testing.T) {
sentinel := errors.New("boom")
r := New().FailWith(sentinel)

_, err := r.ChangesForBatch(context.Background(), entity.Batch{ID: "q/batch/1"})
require.ErrorIs(t, err, sentinel)

_, err = r.DetailedForBatch(context.Background(), entity.Batch{ID: "q/batch/1"})
require.ErrorIs(t, err, sentinel)
}
12 changes: 12 additions & 0 deletions submitqueue/core/changeset/mock/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
load("@rules_go//go:def.bzl", "go_library")

go_library(
name = "mock",
srcs = ["changeset_mock.go"],
importpath = "github.com/uber/submitqueue/submitqueue/core/changeset/mock",
visibility = ["//visibility:public"],
deps = [
"//submitqueue/entity",
"@org_uber_go_mock//gomock",
],
)
72 changes: 72 additions & 0 deletions submitqueue/core/changeset/mock/changeset_mock.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading