Skip to content

Commit 85af0ca

Browse files
authored
[fix](s3-client) Fix s3-client max-connection default value which causes s3 to be slow and unable to handle high concurrency in multi-file scenarios. (#58512)
### What problem does this PR solve? Problem Summary: ### Release note Cherry-pick #58511 ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
1 parent 16b2d1c commit 85af0ca

File tree

1 file changed

+1
-2
lines changed

1 file changed

+1
-2
lines changed

be/src/util/s3_util.cpp

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -369,8 +369,7 @@ std::shared_ptr<io::ObjStorageClient> S3ClientFactory::_create_s3_client(
369369
if (s3_conf.max_connections > 0) {
370370
aws_config.maxConnections = s3_conf.max_connections;
371371
} else {
372-
// AWS SDK max concurrent tcp connections for a single http client to use. Default 25.
373-
aws_config.maxConnections = std::max(config::doris_scanner_thread_pool_thread_num, 25);
372+
aws_config.maxConnections = 102400;
374373
}
375374

376375
aws_config.requestTimeoutMs = 30000;

0 commit comments

Comments
 (0)