-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fix](load) Fix the issue of high-concurrency single-replica load getting stuck #42297
Conversation
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
clang-tidy review says "All clean, LGTM! 👍" |
run buildall |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR approved by at least one committer and no changes requested. |
PR approved by anyone and no changes requested. |
TeamCity be ut coverage result: |
TPC-H: Total hot run time: 41560 ms
|
TPC-DS: Total hot run time: 192797 ms
|
ClickBench: Total hot run time: 32.52 s
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
PR approved by at least one committer and no changes requested. |
…ting stuck (apache#42297) In high-concurrency single-replica load, the tablet_writer_add_block RPC may occupy the _heavy_work_pool completely, causing the response_slave_tablet_pull_rowset RPC to have no available threads for processing. As a result, tablet_writer_add_block waits indefinitely for a response from the slave tablet, leading to the import getting stuck until it times out. response_slave_tablet_pull_rowset is relatively lightweight, so it can be handled by the _light_work_pool.
…ting stuck (apache#42297) In high-concurrency single-replica load, the tablet_writer_add_block RPC may occupy the _heavy_work_pool completely, causing the response_slave_tablet_pull_rowset RPC to have no available threads for processing. As a result, tablet_writer_add_block waits indefinitely for a response from the slave tablet, leading to the import getting stuck until it times out. response_slave_tablet_pull_rowset is relatively lightweight, so it can be handled by the _light_work_pool.
…ting stuck (apache#42297) In high-concurrency single-replica load, the tablet_writer_add_block RPC may occupy the _heavy_work_pool completely, causing the response_slave_tablet_pull_rowset RPC to have no available threads for processing. As a result, tablet_writer_add_block waits indefinitely for a response from the slave tablet, leading to the import getting stuck until it times out. response_slave_tablet_pull_rowset is relatively lightweight, so it can be handled by the _light_work_pool.
In high-concurrency single-replica load, the tablet_writer_add_block RPC may occupy the _heavy_work_pool completely, causing the response_slave_tablet_pull_rowset RPC to have no available threads for processing. As a result, tablet_writer_add_block waits indefinitely for a response from the slave tablet, leading to the import getting stuck until it times out.
response_slave_tablet_pull_rowset is relatively lightweight, so it can be handled by the _light_work_pool.