Skip to content

Pipe: stop historical source supply after drop#17877

Open
Caideyipi wants to merge 1 commit into
masterfrom
fix-drop-pipe-historical-source-close
Open

Pipe: stop historical source supply after drop#17877
Caideyipi wants to merge 1 commit into
masterfrom
fix-drop-pipe-historical-source-close

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

What changed

  • Drop the processor and sink stages before cleaning up the source stage, so a dropped pipe stops consuming and forwarding events first.
  • Make historical source close visible before synchronized cleanup, allowing concurrent supply/start paths to stop returning events after drop.
  • Track pinned historical TsFile resources explicitly and release only those pinned resources during drop cleanup.

Why

For pure historical sync, DROP PIPE can remove ConfigNode metadata before DataNode task cleanup finishes. The historical source close path could then wait behind a concurrent supply() call while processor/sink continued forwarding historical TsFile events, so SHOW PIPES was empty but the receiver still observed data transfer.

Verification

  • mvn -pl iotdb-core/datanode spotless:apply
  • mvn -pl iotdb-core/datanode compile reached javac after checkstyle and spotless passed, then failed due local native memory/pagefile exhaustion; generated hs_err/replay files were removed.

@sonarqubecloud

sonarqubecloud Bot commented Jun 9, 2026

Copy link
Copy Markdown

@codecov

codecov Bot commented Jun 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 13.46154% with 45 lines in your changes missing coverage. Please review.
✅ Project coverage is 40.62%. Comparing base (3d98ea5) to head (4787a27).
⚠️ Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
...peHistoricalDataRegionTsFileAndDeletionSource.java 13.72% 44 Missing ⚠️
...che/iotdb/db/pipe/agent/task/PipeDataNodeTask.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17877      +/-   ##
============================================
- Coverage     40.85%   40.62%   -0.23%     
+ Complexity     2611     2568      -43     
============================================
  Files          5201     5240      +39     
  Lines        354010   361992    +7982     
  Branches      45313    46586    +1273     
============================================
+ Hits         144626   147055    +2429     
- Misses       209384   214937    +5553     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant