Skip to content

Add changelog generation tool for GitHub milestones#13063

Open
cmcfarlen wants to merge 5 commits into
apache:masterfrom
cmcfarlen:new-changelog-tool
Open

Add changelog generation tool for GitHub milestones#13063
cmcfarlen wants to merge 5 commits into
apache:masterfrom
cmcfarlen:new-changelog-tool

Conversation

@cmcfarlen

Copy link
Copy Markdown
Contributor

Replaces tools/git/changelog.pl with a Python implementation that generates changelogs from merged PRs in a milestone using the GitHub API or gh CLI. Default output matches the existing CHANGELOG-* file format. The --doc mode includes merge SHAs, labels, and full PR descriptions to guide AI-assisted release documentation updates. Supports text and YAML output formats.

@cmcfarlen cmcfarlen added this to the 11.0.0 milestone Apr 6, 2026
@cmcfarlen cmcfarlen self-assigned this Apr 6, 2026
@cmcfarlen cmcfarlen added the Tools label Apr 6, 2026
@cmcfarlen

Copy link
Copy Markdown
Contributor Author

hmm, the uv.lock file is making the RAT check mad. Should I remove uv.lock?

@bneradt

bneradt commented Apr 7, 2026

Copy link
Copy Markdown
Contributor

hmm, the uv.lock file is making the RAT check mad. Should I remove uv.lock?

I think it's recommended to add uv.lock to ensure the exact packages are added. Let's add it back. The problem wasn't your patch adding the lock, the problem is the RAT check incorrectly failing on the lock file. I'll update CI to allow it.


Update

#13066

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new Python-based changelog generator under tools/changelog/ to produce CHANGELOG-*-style output from GitHub milestones (via direct REST API calls or the gh CLI), and updates the release-process documentation to use it.

Changes:

  • Add tools/changelog/changelog.py with text/YAML output and an extended --doc mode for richer metadata.
  • Add tools/changelog/pyproject.toml and tools/changelog/uv.lock for dependency management/execution via uv.
  • Update release-process docs to use the new tool (with --use-gh).

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 7 comments.

File Description
tools/changelog/changelog.py Implements milestone PR collection via REST API or gh, plus output formatting.
tools/changelog/pyproject.toml Defines the Python project and console script entry point.
tools/changelog/uv.lock Pins Python dependencies for uv-managed execution.
doc/developer-guide/release-process/index.en.rst Updates the documented release workflow to generate changelogs via the new tool.

Comment thread doc/developer-guide/release-process/index.en.rst
Comment thread doc/developer-guide/release-process/index.en.rst
Comment thread tools/changelog/changelog.py Outdated
Comment thread tools/changelog/changelog.py Outdated
Comment thread tools/changelog/changelog.py
Comment thread tools/changelog/changelog.py
Comment thread tools/changelog/changelog.py
@bryancall bryancall self-requested a review April 20, 2026 22:56
cmcfarlen and others added 4 commits April 21, 2026 19:24
Replaces tools/git/changelog.pl with a Python implementation
that generates changelogs from merged PRs in a milestone using
the GitHub API or gh CLI. Default output matches the existing
CHANGELOG-* file format. The --doc mode includes merge SHAs,
labels, and full PR descriptions to guide AI-assisted release
documentation updates. Supports text and YAML output formats.

Co-Authored-By: Claude <noreply@anthropic.com>
Replace reference to tools/git/changelog.pl with the new
tools/changelog/changelog.py invocation using uv run.

Co-Authored-By: Claude <noreply@anthropic.com>
@cmcfarlen cmcfarlen force-pushed the new-changelog-tool branch from e208668 to cd07fa5 Compare April 22, 2026 00:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Replaces the legacy Perl-based milestone changelog generator with a new Python tool under tools/changelog/ that can pull merged PRs for a milestone via the GitHub REST API or the gh CLI, and updates the release process docs accordingly.

Changes:

  • Removed tools/git/changelog.pl (Perl implementation).
  • Added a Python-based changelog generator (tools/changelog/changelog.py) with a pyproject.toml + uv.lock for dependency management.
  • Updated release-process documentation to use the new tool.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tools/git/changelog.pl Removes the old Perl changelog generator.
tools/changelog/changelog.py New Python tool for generating milestone changelogs via GitHub API or gh, with optional doc/YAML output.
tools/changelog/pyproject.toml Defines the Python tool project and dependencies.
tools/changelog/uv.lock Locks Python dependencies for reproducible runs via uv.
doc/developer-guide/release-process/index.en.rst Updates release instructions to use the new Python tool.

Comment thread tools/changelog/pyproject.toml
Comment thread doc/developer-guide/release-process/index.en.rst
Comment thread tools/changelog/changelog.py
Comment on lines +281 to +283
parser.add_argument("-m", "--milestone", required=True, help="Milestone title")
parser.add_argument("-a", "--auth", default=None, help="GitHub auth token (or set GH_TOKEN env var)")
parser.add_argument("-v", "--verbose", action="store_true", help="Verbose output")
Comment thread tools/changelog/changelog.py
@cmcfarlen

Copy link
Copy Markdown
Contributor Author

[approve ci autest 2]

@bryancall bryancall left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to see this move off Perl. The default httpx path looks solid: explicit rate-limit checks, raise_for_status, distinct exit codes, and a good unauthenticated-token warning.

One thing I want fixed before merge. In the --use-gh path, the per-PR merge check treats any non-zero gh api .../merge exit as "not merged" and skips the PR (changelog.py L130). That collapses a genuine 404 (really not merged) together with 403 secondary rate limiting, 5xx, and network errors into the same outcome. Since this makes one API call per PR across a whole milestone, secondary rate limiting is exactly the failure to expect, and when it hits you get a silently incomplete release changelog with a zero exit code. The httpx path already does this right in _is_merged (204 vs 404 vs raise_for_status). Please have the gh path distinguish 404 from other failures and error out on the rest instead of silently skipping. Same goes for the --doc detail fetch, which substitutes empty sha/body on failure rather than surfacing it.

Smaller items, not blocking:

  • --doc help and the module docstring say "full commit message" but the code stores the PR body. Fix the wording (or fetch the actual commit message).
  • The -a/--auth token is visible in ps and shell history. Carried over from the old script, but for new code prefer GH_TOKEN only and mark -a as discouraged.
  • pyproject.toml is missing license = "Apache-2.0" that the other tool packages set, and main() is missing a -> None return annotation.

The Copilot notes about milestone state=all and a --format yaml JSON fallback are already handled in the current code: both milestone lookups use state=all, and yaml exits with a clear error when PyYAML is missing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

4 participants