Skip to content

[AMORO-4274] Remove Roaring64Bitmap.isEmpty() check in CombinedDelete…#4275

Open
byungnam wants to merge 1 commit into
apache:masterfrom
byungnam:AMORO-4274-remove-isempty-check
Open

[AMORO-4274] Remove Roaring64Bitmap.isEmpty() check in CombinedDelete…#4275
byungnam wants to merge 1 commit into
apache:masterfrom
byungnam:AMORO-4274-remove-isempty-check

Conversation

@byungnam

Copy link
Copy Markdown

Why are the changes needed?

CombinedDeleteFilter builds a per-file position-delete predicate that runs once per scanned record. For each record it looks up the file path in positionMap and checks posSet.isEmpty() before posSet.contains(...):

Roaring64Bitmap posSet = positionMap.get(structLikeForDelete.filePath());
if (posSet == null || posSet.isEmpty()) {
  return false;
}
return posSet.contains(structLikeForDelete.getPosition());

The isEmpty() check is unnecessary and wastes CPU on the hot path:

  1. positionMap is populated only via computeIfAbsent(...).add(...) while reading position-delete files, so every value stored in the map is a non-empty Roaring64Bitmap. positionMap.get(...) therefore returns either null (key absent) or a non-empty bitmap — never an empty one, so the isEmpty() branch is unreachable.
  2. Roaring64Bitmap.isEmpty() is backed by getLongCardinality(), which traverses all high-to-low containers. It is not an O(1) check, so paying it once per record is measurable overhead on tables with large position-delete sets.

Close #4274

Brief change log

  • amoro-format-icebergCombinedDeleteFilter: in the position-delete predicate, drop the unreachable posSet.isEmpty() branch, keeping only the posSet == null guard.

How was this patch tested?

  • Existing tests: covered by the existing delete-filter / reader tests in amoro-format-iceberg; behavior is unchanged because the removed branch was unreachable.
  • Add screenshots for manual tests if appropriate
  • Run test locally before making a pull request

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? (not applicable)

…Filter

positionMap is populated only via computeIfAbsent(...).add(...) while reading
position-delete files, so any stored value is a non-empty Roaring64Bitmap.
positionMap.get(...) therefore returns null or a non-empty bitmap, never an
empty one, making the isEmpty() branch unreachable.

Roaring64Bitmap.isEmpty() is backed by getLongCardinality(), which traverses
all containers, so calling it once per scanned record wastes CPU on the hot
path. Drop the check and keep only the null guard; behavior is unchanged.

Close apache#4274

@zhoujinsong zhoujinsong left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Thanks a lot for the work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement]: Remove Roaring64Bitmap.isEmpty() check in CombinedDeleteFilter

2 participants