Skip to content

Remove large-memory CopyTo and parser overflow tests#131280

Merged
tannergooding merged 2 commits into
dotnet:mainfrom
tannergooding:tannergooding-large-memory-test-minimize
Jul 24, 2026
Merged

Remove large-memory CopyTo and parser overflow tests#131280
tannergooding merged 2 commits into
dotnet:mainfrom
tannergooding:tannergooding-large-memory-test-minimize

Conversation

@tannergooding

@tannergooding tannergooding commented Jul 23, 2026

Copy link
Copy Markdown
Member

Fixes #24139.

Removes the three large-memory [OuterLoop] tests in System.Memory.Tests — the two CopyToLargeSizeTest (Span + ReadOnlySpan) and TestParser2GiBOverflow — rather than trying to keep them alive.

These allocated 4–16GB of memory (or repeatedly filled a 2GB buffer) and were flaky on the 14GB outerloop machines, which is what #24139 reports. After looking at what they actually cover, they aren't worth that cost:

  • There is no special >2GB/>4GB code path. Span<T>.CopyTo just funnels into the native Memmove, and the parser case only validates that a full 2GB buffer is consumed and succeeds/fails as expected.
  • Span<T> is itself length-limited, so there is no overflow edge case here that isn't already covered — and better bounded — by the existing bounds/overflow tests.
  • The behavior can be validated manually if this logic is ever touched again, which matches the conclusion reached previously for a similar 2GB test in Remove 2GB allocations from MemoryStream_SeekOverflow_Throws. #118400.

AllocationHelper is left as-is; it is still used by the remaining large-allocation tests (Overflow, Clear, BinarySearch, Base64).

Note

This PR description and the code changes were drafted with GitHub Copilot.

The three disabled/expensive large-memory tests committed 8GB (CopyTo) or
refilled 2GB five times (parser), thrashing swap on 14GB CI agents. Halve the
CopyTo footprint to a single overlapping ~4GB buffer, verify only the boundaries
(the tail still catches a truncated length), drop the redundant 4GB+256B case,
and fill the parser buffer once while restoring only the bytes each case touches.
Switch AllocationHelper from Marshal.AllocHGlobal to NativeMemory, and re-enable
the two CopyTo tests by removing the dotnet#24139 ActiveIssue.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts several System.Memory.Tests [OuterLoop] scenarios to reduce peak memory and per-test execution cost while keeping coverage of large-buffer behaviors (notably >4GB CopyTo and ~2GB parser overflow).

Changes:

  • Updates the large-buffer Span<T>.CopyTo / ReadOnlySpan<T>.CopyTo tests to use a single overlapping >4GB native allocation instead of two separate >4GB allocations.
  • Refactors the 2GiB parser overflow test to fill the 2GB buffer once and restore only the bytes touched per test case.
  • Switches test native allocation helpers from Marshal.AllocHGlobal/FreeHGlobal to NativeMemory.Alloc/Free.
Show a summary per file
File Description
src/libraries/System.Memory/tests/Span/CopyTo.cs Reworks >4GB Span.CopyTo test to use one overlapping native buffer and fewer assertions.
src/libraries/System.Memory/tests/ReadOnlySpan/CopyTo.cs Mirrors the Span change for ReadOnlySpan.CopyTo with the same allocation strategy.
src/libraries/System.Memory/tests/ParsersAndFormatters/Parser/ParserTests.2gbOverflow.cs Avoids repeated full-buffer refills by reusing a single 2GB span and restoring only mutated bytes.
src/libraries/System.Memory/tests/AllocationHelper.cs Changes native allocation/free implementation to NativeMemory and keeps a global mutex for large allocations.

Copilot's findings

  • Files reviewed: 4/4 changed files
  • Comments generated: 3

Comment thread src/libraries/System.Memory/tests/AllocationHelper.cs
Comment thread src/libraries/System.Memory/tests/Span/CopyTo.cs
Comment thread src/libraries/System.Memory/tests/ReadOnlySpan/CopyTo.cs
- AllocationHelper: release the mutex on any failed allocation via finally, not
  just OutOfMemoryException, so a failure can't leave it held and hang later tests.
- CopyTo tests: initialize only the boundary elements instead of filling the whole
  >4GB buffer, dropping an extra full memory pass while keeping truncation detection.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 23, 2026 18:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 4/4 changed files
  • Comments generated: 0 new

@tannergooding tannergooding changed the title Minimize large-memory CopyTo and parser overflow tests Remove large-memory CopyTo and parser overflow tests Jul 23, 2026
@tannergooding

Copy link
Copy Markdown
Member Author

@vcsjones any other feedback on this one?

@tannergooding
tannergooding merged commit 911ee58 into dotnet:main Jul 24, 2026
81 of 85 checks passed
@tannergooding
tannergooding deleted the tannergooding-large-memory-test-minimize branch July 24, 2026 23:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Large memory test fails on outerloop

4 participants