Skip to content

perf(vulnfeeds): make vulnfeeds tests go faster#5680

Open
jess-lowe wants to merge 2 commits into
google:masterfrom
jess-lowe:perf/vulnfeed-tests-go-vroom
Open

perf(vulnfeeds): make vulnfeeds tests go faster#5680
jess-lowe wants to merge 2 commits into
google:masterfrom
jess-lowe:perf/vulnfeed-tests-go-vroom

Conversation

@jess-lowe

Copy link
Copy Markdown
Contributor
  1. TestReposFromReferences (in vulnfeeds/conversion):

    • Issue: Was taking ~25 seconds. One subtest involving a cgit repository was taking over 20 seconds, likely due to network timeouts or slow responses during go-git operations or canonical link validation.
    • Fix: Modified the test to use the VCR client for git.FindCanonicalLink calls (which perform HTTP HEAD requests). This allows recording and replaying these requests.
    • Result: Test time reduced to ~3 seconds.
  2. cve5 Package Tests (vulnfeeds/conversion/cve5):

    • Issue: TestCVE5Snapshot and TestExtractVersions were doing live network calls during version extraction and Git version-to-commit conversion.
    • Fix: Integrated testutils.SetupGitVCR(t) into these tests to record and replay Git remote interactions.
    • Result: While initial recording still takes time, playback significantly reduces execution time. TestExtractVersions reduced from ~8s to ~3.8s (played back). TestCVE5Snapshot took around 10-12s, processing multiple files sequentially.

@another-rex another-rex left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Slightly hesitant on encoding git responses, but let's try it and see.

Can you extend/explain the final line though?

Result: While initial recording still takes time, playback significantly reduces execution time. TestExtractVersions reduced from ~8s to ~3.8s (played back). TestCVE5Snapshot took around 10-12s, processing multiple files sequentially.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants