Create benchmarks for the flat index (full-precision in-memory implementation)

[arrayka]

Establishes a baseline benchmark for FlatIndex::knn_search, enabling evaluation of new capabilities and innovations in DiskANN's flat-scan implementation.

Why Add Flat Index Benchmarks if Exhaustive Exists?

Exhaustive benchmarks are lightweight benchmarks focused on quantizer speed and accuracy: they measure compress + distance performance in isolation, without involving the FlatIndex machinery or any data provider. They operate entirely outside the DiskANN provider/index abstraction.

Flat benchmarks are end-to-end benchmarks that exercise FlatIndex with different strategy configurations. They exercise FlatIndex through the actual provider and search strategy abstractions, so they capture realistic runtime behavior, and integration overhead that exhaustive benchmarks intentionally skip.

This PR enables full-precision in-memory flat search. Future work may add:

• Quantized flat search
• Quantized flat search with reranking (disk-based or in-memory)
• Batched SIMD scan
• Integration with diversity search

Changes

• New diskann-benchmark/src/flat benchmark module with:
  • InMemProvider wrapping FastMemoryVectorProviderAsync (cache-line-aligned vector storage) with identity DataProvider mapping
  • FlatScanStrategy and FlatVisitor implementing SearchStrategy/DistancesUnordered
  • FlatSearcher implementing the Search trait from benchmark_core
• Input schema (diskann-benchmark/src/inputs/flat.rs) with dataset, distance metric, queries, groundtruth, k, thread counts, and reps
• Integration test (flat_search_integration) and example input (diskann-benchmark/example/flat-index.json)
• Performance test input for wikipedia-100K dataset (diskann-benchmark/perf_test_inputs/wikipedia-100K-flat-index.json)
• Unrelated fix: renamed temp file in run_integration_test from graph-index.json to input.json - the helper is shared by all backends, not just graph-index

Follows established patterns from the graph-index and exhaustive backends.

Output example:

cargo run --package diskann-benchmark --release -- run --input-file diskann-benchmark\perf_test_inputs\wikipedia-100K-flat-index.json --output-file ./target/tmp/flat-index-output.json     
                                                                                                                                                                                
######################
# Running Job 1 of 1 #
######################

              Data: target/tmp\wikipedia_cohere/wikipedia_base.bin.crop_nb_100000
         Data Type: float32
          Distance: inner_product
           Queries: target/tmp\wikipedia_cohere/wikipedia_query.bin
       Groundtruth: target/tmp\wikipedia_cohere/wikipedia-100K
                 K: 100
           Threads: 4, 8
              Reps: 1

Loading dataset...
  Loaded 100000 vectors of dimension 768
  Queries: 5000, Groundtruth: 5000x100


  K,   Avg cmps,   QPS - mean(max),             Avg Latency,           p99 Latency,               Recall,   Threads
===================================================================================================================
100,     100000,     123.0 (123.0),   32430.0us (32430.0us),   35711.0us (35711us),   0.9999899999999999,         4
100,     100000,     250.4 (250.4),   31909.2us (31909.2us),   33323.0us (33323us),   0.9999899999999999,         8

[codecov-commenter]

Codecov Report

❌ Patch coverage is 95.95960% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.67%. Comparing base (3aa44ac) to head (c33cdfb).
⚠️ Report is 27 commits behind head on main.

Files with missing lines	Patch %	Lines
diskann-benchmark/src/flat/search.rs	95.28%	15 Missing ⚠️
diskann-benchmark/src/inputs/flat.rs	98.52%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1170      +/-   ##
==========================================
+ Coverage   89.46%   89.67%   +0.21%     
==========================================
  Files         487      506      +19     
  Lines       92170    96155    +3985     
==========================================
+ Hits        82460    86229    +3769     
- Misses       9710     9926     +216     

Flag	Coverage Δ
miri	89.67% <95.95%> (+0.21%)	⬆️
unittests	89.34% <95.95%> (+0.22%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
diskann-benchmark-core/src/lib.rs	53.33% <ø> (ø)
diskann-benchmark-core/src/utils.rs	100.00% <100.00%> (ø)
diskann-benchmark/src/flat/mod.rs	100.00% <100.00%> (ø)
diskann-benchmark/src/inputs/mod.rs	81.25% <ø> (ø)
diskann-benchmark/src/main.rs	91.64% <100.00%> (+0.28%)	⬆️
diskann-benchmark/src/inputs/flat.rs	98.52% <98.52%> (ø)
diskann-benchmark/src/flat/search.rs	95.28% <95.28%> (ø)

... and 56 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR adds a new end-to-end benchmark backend for diskann::flat::FlatIndex::knn_search, establishing a baseline for full-precision in-memory brute-force kNN performance (recall + latency) to support future flat-search optimizations.

Changes:

• Registers a new flat-index benchmark backend and adds an integration test wired to a new example input.
• Introduces a FlatSearch input schema and JSON examples/perf-test inputs for running flat-index benchmarks.
• Implements an in-memory DataProvider + SearchStrategy that performs a full sequential scan using FastMemoryVectorProviderAsync.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
diskann-benchmark/src/main.rs	Registers the new flat backend; adds a flat-search integration test; renames the temp input file used by the shared integration helper.
diskann-benchmark/src/inputs/mod.rs	Exposes the new flat benchmark input module.
diskann-benchmark/src/inputs/flat.rs	Adds FlatSearch / SearchPhase input schema, validation hooks, and display formatting.
diskann-benchmark/src/flat/mod.rs	Adds the flat benchmark module entry point and registration function.
diskann-benchmark/src/flat/search.rs	Implements the flat benchmark backend (provider + scan strategy + search runner + aggregation + formatting).
diskann-benchmark/perf_test_inputs/wikipedia-100K-flat-index.json	Adds a perf-test job configuration for wikipedia-100K flat search.
diskann-benchmark/example/flat-index.json	Adds an example job configuration used by the new integration test.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[arkrishn94]

Thanks for working on this Alex. Just have one non-minor comment - we should be using a provider, visitor and DistancesUnordered implementation as is in the benchmark. Maybe this PR can include the implementation of these for the full precision provider?

[arkrishn94]

Same comment as above, we should have a visitor implementation for FastmemoryVectorProviderAsync and that should exist along with its definition.