Add path-scoped JSON collection strategies

[akshat62]

Problem statement

DeepDiff compares arbitrary Python and JSON-like structures, but nested API collections are often unordered and keyed by business identity rather than list position. Positional comparison can create cascading false positives when records are inserted, removed, or reordered.

This PR adds an opt-in, reusable normalization layer for declaring collection-specific semantics without changing existing DeepDiff behavior.

Proposed solution

The PR introduces DeepJSONDiff and CollectionStrategy. Both inputs are converted into canonical caller-isolated views, then compared by the existing DeepDiff engine.

from deepdiff import CollectionStrategy, DeepJSONDiff

result = DeepJSONDiff(
    expected,
    actual,
    collection_strategies=[
        CollectionStrategy(
            path="$.orders[*].items",
            match_by=("product.id", "warehouse.code"),
            filter_func=lambda item: item.get("active", True),
            exclude_fields=("generatedAt", "requestId"),
        )
    ],
)

The existing DeepDiff constructor and comparison semantics remain unchanged.

Selector semantics

Supported selectors include:

$.users
$.groups[0].users
$.groups[*].users
$.*.users
$['a-b']

• [*] matches exactly one array index.
• .* matches exactly one object key.
• Quoted bracket selectors address keys that cannot be represented safely in dot form.
• Wildcards do not cross path levels.
• Higher priority wins.
• At equal priority, the pattern with more exact tokens wins.
• Equally specific matches are rejected as ambiguous.

JSON object keys are expected to be strings. Diagnostic paths use the same quoted-key syntax and can be reused as selectors.

Collection behavior

Identity matching

match_by supports single, composite, and nested fields. Identity values must resolve after normalization to finite JSON scalars: None, booleans, integers, finite floats, or strings.

Identity labels use canonical type-preserving encoding, so integer 1, float 1.0, and string "1" remain distinct and composite identities cannot collide through delimiter characters.

Filtering, normalization, sorting, and exclusion

Filters and normalizers receive defensive copies and may mutate them without modifying caller-owned payloads. Copies are created only when callbacks are configured.

Processing order is:

1. apply the filter;
2. apply normalizers;
3. capture identity and sort fields;
4. remove excluded fields;
5. canonicalize nested content.

Identity and sort fields may therefore be excluded from the compared record while still controlling matching or ordering.

sort_by provides a total, type-stable order for JSON-compatible values. Integers and floats remain distinct. Finite values, infinities, and NaN values have deterministic positions without heterogeneous comparison failures.

Structured sort keys are restricted to JSON lists and mappings with string keys. Unsupported values are rejected instead of being ordered through unstable repr() output.

Order-insensitive scalar arrays

compare_as_set=True performs multiset/bag comparison:

• order is ignored;
• duplicate counts remain significant;
• integer and float values remain type-distinct;
• only finite JSON scalar values are accepted;
• it cannot be combined with match_by or sort_by.

Missing and duplicate identities

Missing identity policies:

• fallback: retain records in an ordered fallback collection;
• exclude: omit them;
• error: raise IdentityExtractionError with input side and path context.

Duplicate policies:

• error: raise DuplicateIdentityError;
• group: retain every record under the identity and optionally order the group with sort_by.

Enum members and string values are accepted.

Diagnostics

get_stats() separates diagnostics by input side:

stats["left"]["$.orders[0].items"]
stats["right"]["$.orders[0].items"]

Each entry reports the selected strategy, input item count, filtered count, missing-identity count, and duplicate-group count. Separating sides prevents unrelated parent records from being conflated after parent reordering.

DeepDiff keyword compatibility

Identity matching changes selected arrays into canonical mappings. Options whose behavior depends on caller-visible paths, object paths, or iterable positions are rejected instead of being interpreted against a different structure:

• include_paths
• exclude_paths
• exclude_regex_paths
• ignore_order_func
• iterable_compare_func
• custom_operators
• exclude_obj_callback
• exclude_obj_callback_strict

Other keyword arguments are forwarded to the underlying DeepDiff instance.

Result interface

DeepJSONDiff is a composition-based facade rather than a complete DeepDiff subclass. It implements the read-only Mapping interface, delegates to_dict() and to_json(), and exposes the underlying result through result.diff.

Backward compatibility

The feature is fully opt-in:

• existing DeepDiff behavior is unchanged;
• no strategy is applied unless the caller constructs DeepJSONDiff;
• caller-owned inputs are not mutated;
• existing views, Delta behavior, serialization, and multiprocessing paths remain untouched.

Files changed

• deepdiff/json_diff.py — facade, strategy model, selector parsing, canonicalization, identity matching, sorting, policies, diagnostics, and compatibility boundaries.
• deepdiff/__init__.py — public exports.
• tests/test_json_diff.py — primary functional, regression, and edge-case coverage.
• tests/test_json_diff_merge_contracts.py — final sorting/exclusion and callback-compatibility contract tests.
• docs/json_collection_strategies.rst — public behavior and API documentation.
• deepdiff/docstrings/index.rst — documentation navigation and unreleased note.

No custom GitHub Actions workflow is included.

Validation performed

The implementation is validated across Python 3.10–3.14 and includes focused tests for:

• reordered nested collections;
• additions, removals, and modifications together;
• exact one-level wildcards and quoted-key selectors;
• collision-safe composite identities;
• scalar identity enforcement;
• identity and sort extraction before exclusion;
• callback isolation;
• finite and non-finite numeric sorting;
• type-stable integer/float multiset comparison;
• structured JSON sorting;
• duplicate-group ordering before exclusion;
• side-separated nested diagnostics;
• missing and duplicate policies on both inputs;
• invalid strategy rejection;
• complete path-sensitive DeepDiff option rejection;
• deterministic serialization and mapping behavior.

Scope and limitations

This PR provides deterministic exact reconciliation. It does not infer business identity or perform fuzzy matching. The selector grammar is intentionally smaller than full JSONPath. Relative identity and sort fields use dotted extraction and numeric list indexes.

For identity-matched collections, result paths contain canonical identity keys rather than unstable original list indexes.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 94.38503% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.07%. Comparing base (c59636c) to head (a5faa5d).

Files with missing lines	Patch %	Lines
deepdiff/json_diff.py	94.38%	21 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #612      +/-   ##
==========================================
- Coverage   95.10%   95.07%   -0.04%     
==========================================
  Files          17       18       +1     
  Lines        4900     5274     +374     
==========================================
+ Hits         4660     5014     +354     
- Misses        240      260      +20     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.