|
| 1 | +# HDRHistogram |
| 2 | + |
| 3 | +HDRHistogram (High Dynamic Range Histogram) is a logarithmic-bucketed histogram implementation designed for efficient tracking of values across multiple orders of magnitude. |
| 4 | + |
| 5 | +## Overview |
| 6 | + |
| 7 | +HDRHistogram offers the following features: |
| 8 | + |
| 9 | +* **Wide Value Range**: Efficiently tracks values across multiple orders of magnitude |
| 10 | +* **Configurable Precision**: Adjustable number of buckets for different accuracy needs |
| 11 | +* **Memory Efficient**: Uses logarithmic bucketing to minimize memory usage |
| 12 | +* **Summary Statistics**: Provides comprehensive statistics including min, max, quartiles, and count |
| 13 | +* **Visualization**: Built-in support for plotting distributions with logarithmic scales |
| 14 | +* **Serialization**: Supports converting histograms to/from dictionaries for storage and transmission |
| 15 | +* **Value Range Control**: Configurable minimum and maximum trackable values |
| 16 | + |
| 17 | +## Main Class |
| 18 | + |
| 19 | +```{eval-rst} |
| 20 | +.. autoclass:: QuantileFlow.HDRHistogram |
| 21 | + :members: |
| 22 | + :undoc-members: |
| 23 | + :special-members: __init__ |
| 24 | + :exclude-members: __dict__, __weakref__ |
| 25 | + :noindex: |
| 26 | + :show-inheritance: |
| 27 | +``` |
| 28 | + |
| 29 | +## Mathematical Background |
| 30 | + |
| 31 | +HDRHistogram works by maintaining a fixed number of logarithmic buckets to efficiently track values across a wide range. The key aspects of the implementation are: |
| 32 | + |
| 33 | +1. **Logarithmic Bucketing**: Values are assigned to buckets based on their logarithm, allowing efficient tracking of values across multiple orders of magnitude |
| 34 | +2. **Value Range Control**: Configurable minimum and maximum values ensure memory efficiency |
| 35 | +3. **Linear Interpolation**: Within each bucket, linear interpolation is used to estimate quantiles |
| 36 | + |
| 37 | +### Bucket Calculation |
| 38 | + |
| 39 | +The bucket index for a value $x$ is calculated as: |
| 40 | + |
| 41 | +$$ |
| 42 | +\text{bucket_index} = \lfloor \log_2(x) \rfloor |
| 43 | +$$ |
| 44 | + |
| 45 | +This ensures that: |
| 46 | +- Values in the same order of magnitude are grouped together |
| 47 | +- The number of buckets grows logarithmically with the value range |
| 48 | +- Memory usage remains constant regardless of the number of values inserted |
| 49 | + |
| 50 | +### Quantile Estimation |
| 51 | + |
| 52 | +Quantiles are estimated by: |
| 53 | +1. Finding the bucket containing the target quantile |
| 54 | +2. Using linear interpolation within the bucket to estimate the exact value |
| 55 | + |
| 56 | +For a target quantile $q$ and total count $N$, the estimated value is: |
| 57 | + |
| 58 | +$$ |
| 59 | +\text{value} = \text{lower_bound} + (\text{upper_bound} - \text{lower_bound}) \times \frac{qN - \text{cumulative_count}}{\text{bucket_count}} |
| 60 | +$$ |
| 61 | + |
| 62 | +Where: |
| 63 | +- $\text{lower_bound}$ and $\text{upper_bound}$ are the bucket boundaries |
| 64 | +- $\text{cumulative_count}$ is the count of values in previous buckets |
| 65 | +- $\text{bucket_count}$ is the count of values in the current bucket |
| 66 | + |
| 67 | +## Performance Characteristics |
| 68 | + |
| 69 | +- **Memory Usage**: O(num_buckets) |
| 70 | +- **Insertion Time**: O(1) per value |
| 71 | +- **Query Time**: O(num_buckets) for quantile queries |
| 72 | +- **Merge Time**: O(num_buckets) for merging histograms |
| 73 | + |
| 74 | +## Use Cases |
| 75 | + |
| 76 | +HDRHistogram is particularly useful for: |
| 77 | +- Tracking metrics with wide value ranges (e.g., response times, request sizes) |
| 78 | +- Monitoring systems with multiple orders of magnitude in measurements |
| 79 | +- Applications requiring configurable precision and value range control |
| 80 | +- Distributed systems where histograms need to be merged |
| 81 | +- Real-time monitoring and alerting systems |
| 82 | + |
| 83 | +## Usage Examples |
| 84 | + |
| 85 | +For basic usage, see the [Getting Started](../getting-started.md) guide. |
| 86 | + |
| 87 | +For more advanced examples, see the [Examples](../examples.md) page. |
0 commit comments