Skip to content

Speed up exception handling#1038

Open
akx wants to merge 2 commits into
pydantic:mainfrom
akx:retire-map-exceptions-hx2
Open

Speed up exception handling#1038
akx wants to merge 2 commits into
pydantic:mainfrom
akx:retire-map-exceptions-hx2

Conversation

@akx

@akx akx commented Jun 16, 2026

Copy link
Copy Markdown

Summary

@contextmanager is slow, and it's used in hot paths for every request both in httpcore2 and httpx2.

This PR replaces uses of map_exceptions and map_httpcore_exceptions with faster alternatives.

On my machine, a pytest-benchmark test shows a nearly 10% speed increase on a microbenchmark that retrieves a file from a local Caddy (see source in the details block below)

----------------- benchmark: 2 tests, 2 sources ------------------
Name (time in ms)     0010_82b9e2d Min  0042_384ae31 Min      ΔMin
------------------------------------------------------------------
test_connpool                   3.8101            3.4641     -9.1%
test_client                     4.0599            3.7075     -8.7%
------------------------------------------------------------------
test_bench1.py
import pytest
from httpx2 import Client
from httpcore2 import ConnectionPool


def test_connpool(benchmark):
    with ConnectionPool() as pool:
        def b():
            resp = pool.request(
                method="GET",
                url="http://127.0.0.1:9339/bigfile2",
            )
            assert len(resp.content) == 10 * 1024 * 1024
        benchmark(b)


def test_client(benchmark):
    with Client() as c:
        def b():
            resp = c.request(
                method="GET",
                url="http://127.0.0.1:9339/bigfile2",
            )
            assert len(resp.content) == 10 * 1024 * 1024
        benchmark(b)

and a hyperfine check for the same says ~5%, but that's with all of the overhead:

$ hyperfine --warmup=5 --min-runs=20 'git checkout main && uv sync && uv pip install ../pytest-benchmark && pytest --benchmark-autosave test_bench1.py' 'git checkout retire-map-exceptions-hx2 && uv sync && uv pip install ../pytest-benchmark && pytest --benchmark-autosave test_bench1.py'
[...]
Summary
  git checkout retire-map-exceptions-hx2 && uv sync && uv pip install ../pytest-benchmark && pytest --benchmark-autosave test_bench1.py ran
    1.05 ± 0.06 times faster than git checkout main && uv sync && uv pip install ../pytest-benchmark && pytest --benchmark-autosave test_bench1.py

Rebase of

Checklist

  • I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
  • I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
    • No test changes needed. Some pragma: no covers were added to the exception branches that were actually not covered originally either.
  • I've updated the documentation accordingly.
    • No public documentation changes needed, I think.

akx added 2 commits June 16, 2026 13:33
`contextlib.contextmanager`s are much slower than `try:except:`, and here they occur in very hot paths.
`try: except:` blocks are free on modern Pythons when exceptions
aren't raised. Entering a `@contextmanager`'d block is much less free,
and there are hot paths (e.g. reading from a sync socket) where it's
worth avoiding that overhead.

For consistency, this retires the use of `map_exceptions` everywhere.
New exception blocks are marked nocover, since they weren't actually tested
before either.
@codspeed-hq

codspeed-hq Bot commented Jun 16, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 15 untouched benchmarks
⏩ 7 skipped benchmarks1


Comparing akx:retire-map-exceptions-hx2 (384ae31) with main (82b9e2d)

Open in CodSpeed

Footnotes

  1. 7 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 8 files

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant