YQ-4779 fixed ACK sending for sys view scan #26552

GrigoriyPA · 2025-10-08T11:00:00Z

Changelog entry

Fixed ACK sending for sys view scan

Changelog category

Bugfix

Description for reviewers

Bugfix ticket: YQ-4779

Copilot

Pull Request Overview

This PR fixes ACK sending for system view scan operations by introducing a new TScanActorWithoutBackPressure base class and preventing duplicate scan initialization across multiple system view scan actors.

Introduced TScanActorWithoutBackPressure template class to handle ACK processing without backpressure
Added guards to prevent duplicate scan requests in tablet and query stats scan actors
Refactored multiple system view scan classes to use the new base class and simplified ACK handling

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
ydb/core/sys_view/common/scan_actor_base_impl.h	Added new `TScanActorWithoutBackPressure` template class
ydb/core/sys_view/tablets/tablets.cpp	Added guard to prevent duplicate tablet ID requests
ydb/core/sys_view/query_stats/query_stats.cpp	Added guard to prevent duplicate old scan initialization
ydb/core/sys_view/storage/base.h	Migrated to use `TScanActorWithoutBackPressure` base class
ydb/core/sys_view/show_create/show_create.cpp	Migrated to use `TScanActorWithoutBackPressure` base class
ydb/core/sys_view/resource_pools/resource_pools.cpp	Migrated to use `TScanActorWithoutBackPressure` base class
ydb/core/sys_view/resource_pool_classifiers/resource_pool_classifiers.cpp	Migrated to use `TScanActorWithoutBackPressure` base class
ydb/core/sys_view/nodes/nodes.cpp	Migrated to use `TScanActorWithoutBackPressure` base class
ydb/core/sys_view/compile_cache/compile_cache.cpp	Added guard for pending proxy node requests and improved initialization
ydb/core/sys_view/auth/users.cpp	Migrated to use `TScanActorWithoutBackPressure` base class
ydb/core/sys_view/auth/auth_scan_base.h	Added guard to prevent duplicate navigation path requests
ydb/core/kqp/compute_actor/kqp_pure_compute_actor.h	Added new methods for source polling and free space calculation
ydb/core/kqp/compute_actor/kqp_pure_compute_actor.cpp	Implemented source polling logic and refactored free space calculation

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

ydb/core/sys_view/common/scan_actor_base_impl.h

ydb/core/kqp/compute_actor/kqp_pure_compute_actor.cpp

github-actions · 2025-10-08T11:01:25Z

🟢 2025-10-08 11:01:24 UTC The validation of the Pull Request description is successful.

github-actions · 2025-10-08T11:03:05Z

⚪ 2025-10-08 11:03:05 UTC Pre-commit check linux-x86_64-relwithdebinfo for 9c96d7e has started.
⚪ 2025-10-08 11:03:35 UTC Artifacts will be uploaded here
⚫ 2025-10-08 11:06:32 UTC Check cancelled

github-actions · 2025-10-08T11:04:37Z

⚪ 2025-10-08 11:04:36 UTC Pre-commit check linux-x86_64-release-asan for 9c96d7e has started.
⚪ 2025-10-08 11:04:53 UTC Artifacts will be uploaded here
⚫ 2025-10-08 11:06:32 UTC Check cancelled

github-actions · 2025-10-08T11:09:50Z

⚪ 2025-10-08 11:09:49 UTC Pre-commit check linux-x86_64-release-asan for a86c33c has started.
⚪ 2025-10-08 11:10:03 UTC Artifacts will be uploaded here
⚪ 2025-10-08 11:13:44 UTC ya make is running...
🟡 2025-10-08 13:45:30 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15722	15247	0	179	273	23

🟢 2025-10-08 13:45:37 UTC Build successful.
🟡 2025-10-08 13:46:04 UTC ydbd size 3.7 GiB changed* by +311.1 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash	main: `c38c114`	merge: `a86c33c`	diff	diff %
ydbd size	4 021 180 208 Bytes	4 021 498 728 Bytes	+311.1 KiB	+0.008%
ydbd stripped size	1 494 048 320 Bytes	1 494 062 784 Bytes	+14.1 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-10-08T11:12:48Z

⚪ 2025-10-08 11:12:48 UTC Pre-commit check linux-x86_64-relwithdebinfo for a86c33c has started.
⚪ 2025-10-08 11:12:52 UTC Artifacts will be uploaded here
⚪ 2025-10-08 11:16:22 UTC ya make is running...
🟡 2025-10-08 12:57:08 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
38673	35909	0	3	2739	22

⚪ 2025-10-08 12:57:20 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-10-08 13:14:29 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
263 (only retried tests)	245	0	1	0	17

⚪ 2025-10-08 13:14:32 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-10-08 13:23:49 UTC Tests successful.

Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
188 (only retried tests)	173	0	0	0	15

🟢 2025-10-08 13:23:52 UTC Build successful.
🟢 2025-10-08 13:24:10 UTC ydbd size 2.3 GiB changed* by +12.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `c38c114`	merge: `a86c33c`	diff	diff %
ydbd size	2 418 004 960 Bytes	2 418 017 768 Bytes	+12.5 KiB	+0.001%
ydbd stripped size	515 030 344 Bytes	515 032 968 Bytes	+2.6 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-10-09T17:48:03Z

⚪ 2025-10-09 17:48:02 UTC Pre-commit check linux-x86_64-release-asan for 5aea80f has started.
⚪ 2025-10-09 17:48:16 UTC Artifacts will be uploaded here
⚪ 2025-10-09 17:52:09 UTC ya make is running...
🟡 2025-10-09 19:57:13 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15750	15266	0	190	274	20

🟢 2025-10-09 19:57:21 UTC Build successful.
🟢 2025-10-09 19:57:43 UTC ydbd size 3.7 GiB changed* by +42.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `a446a72`	merge: `5aea80f`	diff	diff %
ydbd size	4 023 586 624 Bytes	4 023 629 872 Bytes	+42.2 KiB	+0.001%
ydbd stripped size	1 494 478 880 Bytes	1 494 500 064 Bytes	+20.7 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-10-09T17:57:26Z

⚪ 2025-10-09 17:57:25 UTC Pre-commit check linux-x86_64-relwithdebinfo for 5aea80f has started.
⚪ 2025-10-09 18:00:56 UTC Artifacts will be uploaded here
⚪ 2025-10-09 18:04:38 UTC ya make is running...
🟡 2025-10-09 19:34:35 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
38703	35918	0	6	2747	32

⚪ 2025-10-09 19:34:55 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-10-09 19:50:32 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
640 (only retried tests)	621	0	0	0	19

🟢 2025-10-09 19:50:35 UTC Build successful.
🟢 2025-10-09 19:50:52 UTC ydbd size 2.3 GiB changed* by +19.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `a446a72`	merge: `5aea80f`	diff	diff %
ydbd size	2 418 700 232 Bytes	2 418 719 976 Bytes	+19.3 KiB	+0.001%
ydbd stripped size	515 107 272 Bytes	515 113 608 Bytes	+6.2 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

Copilot

Pull Request Overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 2 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

ydb/library/testlib/common/test_utils.h

ydb/core/sys_view/common/scan_actor_base_impl.h

ydb/library/testlib/common/test_utils.h

ydb/library/testlib/pq_helpers/mock_pq_gateway.cpp

ydb/core/sys_view/query_stats/query_stats.cpp

github-actions · 2025-10-14T11:24:49Z

⚪ 2025-10-14 11:24:48 UTC Pre-commit check linux-x86_64-relwithdebinfo for 8480094 has started.
⚪ 2025-10-14 11:25:02 UTC Artifacts will be uploaded here
⚪ 2025-10-14 11:28:45 UTC ya make is running...

github-actions · 2025-10-14T11:26:09Z

⚪ 2025-10-14 11:26:08 UTC Pre-commit check linux-x86_64-release-asan for 8480094 has started.
⚪ 2025-10-14 11:26:22 UTC Artifacts will be uploaded here
⚪ 2025-10-14 11:29:54 UTC ya make is running...
🟡 2025-10-14 13:33:16 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15784	15359	0	157	250	18

🟢 2025-10-14 13:33:21 UTC Build successful.
🟢 2025-10-14 13:33:44 UTC ydbd size 3.8 GiB changed* by +28.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `10a225c`	merge: `8480094`	diff	diff %
ydbd size	4 031 470 192 Bytes	4 031 498 960 Bytes	+28.1 KiB	+0.001%
ydbd stripped size	1 497 304 704 Bytes	1 497 319 936 Bytes	+14.9 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

ydb/library/testlib/pq_helpers/mock_pq_gateway.cpp

github-actions · 2025-10-15T08:00:38Z

⚪ 2025-10-15 08:00:37 UTC Pre-commit check linux-x86_64-relwithdebinfo for 00e7f59 has started.
⚪ 2025-10-15 08:00:41 UTC Artifacts will be uploaded here
⚪ 2025-10-15 08:04:20 UTC ya make is running...
🟡 2025-10-15 09:40:26 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
38952	36162	0	12	2750	28

⚪ 2025-10-15 09:40:36 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-10-15 09:59:14 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
847 (only retried tests)	827	0	0	0	20

🟢 2025-10-15 09:59:17 UTC Build successful.
🟢 2025-10-15 09:59:34 UTC ydbd size 2.3 GiB changed* by +11.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `a116ac4`	merge: `00e7f59`	diff	diff %
ydbd size	2 423 988 336 Bytes	2 424 000 024 Bytes	+11.4 KiB	+0.000%
ydbd stripped size	515 997 032 Bytes	515 999 912 Bytes	+2.8 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-10-15T08:01:19Z

⚪ 2025-10-15 08:01:19 UTC Pre-commit check linux-x86_64-release-asan for 00e7f59 has started.
⚪ 2025-10-15 08:01:23 UTC Artifacts will be uploaded here
⚪ 2025-10-15 08:05:00 UTC ya make is running...
🟡 2025-10-15 10:05:56 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15791	15298	0	190	284	19

🟢 2025-10-15 10:06:02 UTC Build successful.
🟢 2025-10-15 10:06:26 UTC ydbd size 3.8 GiB changed* by +28.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `a116ac4`	merge: `00e7f59`	diff	diff %
ydbd size	4 032 935 280 Bytes	4 032 963 992 Bytes	+28.0 KiB	+0.001%
ydbd stripped size	1 497 801 344 Bytes	1 497 816 512 Bytes	+14.8 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

Co-authored-by: Copilot <[email protected]>

github-actions · 2025-10-16T07:28:30Z

⚪ 2025-10-16 07:28:30 UTC Pre-commit check linux-x86_64-relwithdebinfo for 4b6ba62 has started.
⚪ 2025-10-16 07:28:44 UTC Artifacts will be uploaded here
⚪ 2025-10-16 07:32:25 UTC ya make is running...
🟡 2025-10-16 09:05:09 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
38962	36183	0	2	2752	25

⚪ 2025-10-16 09:05:20 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-10-16 09:25:31 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
574 (only retried tests)	554	0	0	0	20

🟢 2025-10-16 09:25:34 UTC Build successful.
🟢 2025-10-16 09:25:50 UTC ydbd size 2.3 GiB changed* by +11.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `69954fb`	merge: `4b6ba62`	diff	diff %
ydbd size	2 424 057 760 Bytes	2 424 069 384 Bytes	+11.4 KiB	+0.000%
ydbd stripped size	516 043 304 Bytes	516 046 120 Bytes	+2.8 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

github-actions · 2025-10-16T07:28:50Z

⚪ 2025-10-16 07:28:49 UTC Pre-commit check linux-x86_64-release-asan for 4b6ba62 has started.
⚪ 2025-10-16 07:29:02 UTC Artifacts will be uploaded here
⚪ 2025-10-16 07:32:39 UTC ya make is running...
🟡 2025-10-16 09:29:22 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
15801	15289	0	220	275	17

🟢 2025-10-16 09:29:27 UTC Build successful.
🟢 2025-10-16 09:29:49 UTC ydbd size 3.8 GiB changed* by +32.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: `69954fb`	merge: `4b6ba62`	diff	diff %
ydbd size	4 033 062 440 Bytes	4 033 095 304 Bytes	+32.1 KiB	+0.001%
ydbd stripped size	1 497 932 672 Bytes	1 497 952 000 Bytes	+18.9 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

Co-authored-by: Copilot <[email protected]>

GrigoriyPA requested a review from a team as a code owner October 8, 2025 11:00

github-actions bot added the bugfix label Oct 8, 2025

GrigoriyPA requested review from Copilot and gridnevvvit October 8, 2025 11:00

Copilot AI reviewed Oct 8, 2025

View reviewed changes

ydb/core/sys_view/common/scan_actor_base_impl.h Show resolved Hide resolved

ydb/core/sys_view/common/scan_actor_base_impl.h Show resolved Hide resolved

ydb/core/kqp/compute_actor/kqp_pure_compute_actor.cpp Outdated Show resolved Hide resolved

GrigoriyPA requested a review from a team as a code owner October 9, 2025 17:43

GrigoriyPA requested a review from Copilot October 9, 2025 18:00

Copilot AI reviewed Oct 9, 2025

View reviewed changes

ydb/library/testlib/common/test_utils.h Show resolved Hide resolved

ydb/core/sys_view/common/scan_actor_base_impl.h Show resolved Hide resolved

GrigoriyPA requested review from APozdniakov and yumkam October 10, 2025 07:21

uzhastik reviewed Oct 13, 2025

View reviewed changes

ydb/library/testlib/common/test_utils.h Show resolved Hide resolved

uzhastik reviewed Oct 13, 2025

View reviewed changes

ydb/library/testlib/pq_helpers/mock_pq_gateway.cpp Outdated Show resolved Hide resolved

yumkam reviewed Oct 13, 2025

View reviewed changes

ydb/core/sys_view/query_stats/query_stats.cpp Show resolved Hide resolved

yumkam reviewed Oct 14, 2025

View reviewed changes

ydb/library/testlib/pq_helpers/mock_pq_gateway.cpp Outdated Show resolved Hide resolved

yumkam previously approved these changes Oct 14, 2025

View reviewed changes

GrigoriyPA dismissed yumkam’s stale review via 519db31 October 15, 2025 07:58

GrigoriyPA force-pushed the YQ-4779-fix-ack-sending-for-sys-view-scan branch from 323b477 to 519db31 Compare October 15, 2025 07:58

GrigoriyPA requested a review from yumkam October 15, 2025 10:34

yumkam previously approved these changes Oct 15, 2025

View reviewed changes

GrigoriyPA and others added 5 commits October 16, 2025 10:24

Fixed ACK sending for sys view scan

c2f5af7

Update ydb/core/kqp/compute_actor/kqp_pure_compute_actor.cpp

d5c271c

Co-authored-by: Copilot <[email protected]>

Added unit test on sys view back pressure

64eff9e

Fixed issues

e4a9268

Fixed issues

b86b770

GrigoriyPA dismissed yumkam’s stale review via b86b770 October 16, 2025 07:25

GrigoriyPA force-pushed the YQ-4779-fix-ack-sending-for-sys-view-scan branch from 519db31 to b86b770 Compare October 16, 2025 07:25

APozdniakov approved these changes Oct 16, 2025

View reviewed changes

gridnevvvit approved these changes Oct 16, 2025

View reviewed changes

GrigoriyPA merged commit ed73f44 into ydb-platform:main Oct 16, 2025
13 checks passed

GrigoriyPA deleted the YQ-4779-fix-ack-sending-for-sys-view-scan branch October 16, 2025 10:48

GrigoriyPA added a commit to GrigoriyPA/ydb that referenced this pull request Oct 21, 2025

YQ-4779 fixed ACK sending for sys view scan (ydb-platform#26552)

2c1463f

Co-authored-by: Copilot <[email protected]>

GrigoriyPA mentioned this pull request Oct 21, 2025

merge to stable-25-3 YQ-4312 features and fixes for streaming 3 #27212

Merged

GrigoriyPA added a commit to GrigoriyPA/ydb that referenced this pull request Oct 22, 2025

YQ-4779 fixed ACK sending for sys view scan (ydb-platform#26552)

5ae3cc0

Co-authored-by: Copilot <[email protected]>

YQ-4779 fixed ACK sending for sys view scan #26552

YQ-4779 fixed ACK sending for sys view scan #26552

Uh oh!

Conversation

GrigoriyPA commented Oct 8, 2025

Changelog entry

Changelog category

Description for reviewers

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 8, 2025

Uh oh!

github-actions bot commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

github-actions bot commented Oct 8, 2025 •

edited

Loading

github-actions bot commented Oct 8, 2025 •

edited

Loading

github-actions bot commented Oct 8, 2025 •

edited

Loading

github-actions bot commented Oct 8, 2025 •

edited

Loading

github-actions bot commented Oct 9, 2025 •

edited

Loading

github-actions bot commented Oct 9, 2025 •

edited

Loading

github-actions bot commented Oct 14, 2025 •

edited

Loading

github-actions bot commented Oct 14, 2025 •

edited

Loading

github-actions bot commented Oct 15, 2025 •

edited

Loading

github-actions bot commented Oct 15, 2025 •

edited

Loading

github-actions bot commented Oct 16, 2025 •

edited

Loading

github-actions bot commented Oct 16, 2025 •

edited

Loading