Backtest RS

A high-performance backtesting system implemented in Rust, designed to process and analyze Kraken's complete trading history with support for custom strategies and sophisticated execution simulation.

Project Structure

This is a Rust workspace containing:

• crates/backtest/ - The main backtesting library crate
• crates/xmr_swing/ - XMR/USD swing trading strategy implementation

Historical Market Data

This project uses Kraken's historical market data for backtesting. Here's how to set up the data:

1. Download Kraken_Trading_History.zip from Kraken's official support page: https://support.kraken.com/hc/en-us/articles/360047543791-Downloadable-historical-market-data-time-and-sales-

2. Place the downloaded Kraken_Trading_History.zip in the data/ directory.

3. Run the unzip script:

   cd data
   ./unzip_data.sh

   This will extract the data into a data/Trading_History directory.

Data Organization

After running the unzip script, you'll have:

• data/Kraken_Trading_History.zip - The original downloaded file
• data/Trading_History/ - Directory containing the extracted CSV files

Note: Both the zip file and extracted directory are gitignored to prevent committing large data files to the repository.

Data Description

The dataset contains complete time and sales (trade) data for all trading pairs on Kraken's platform. Each CSV file follows this format:

timestamp,price,volume,buysell
1568062578.0428,10084.9,0.00113146,s

Fields:

• timestamp: Unix timestamp with microsecond precision
• price: The price at which the trade executed
• volume: The volume of the trade
• buysell: Trade direction ('b' for buy, 's' for sell)

Files are organized by trading pair and year, e.g., XBT_USD_trades_2019.csv for Bitcoin-USD trades from 2019.

Running Examples

To run the XMR swing trading strategy:

# From the workspace root
cargo run -p xmr_swing

Library Status

Core Modules

1. Data Processing (backtest/src/data/)

• Trade data parser and validator
• OHLCV candle generator
• Flexible timeframe support
• Data cleaning and validation
• Stream processing interface

pub trait DataSource {
    fn next_trade(&mut self) -> Option<Result<Trade>>;
    fn seek_time(&mut self, timestamp: f64) -> Result<()>;
}

pub struct CandleBuilder {
    fn new(period_length: f64) -> Self;
    fn update(&mut self, trade: &Trade) -> Option<Candle>;
    fn finish(&mut self) -> Option<Candle>;
}

2. Market Simulation (backtest/src/market/)

• Order book simulation with synthetic depth
• Price impact modeling
• Market depth tracking
• Tick-by-tick price updates

pub trait MarketSimulator {
    fn process_trade(&mut self, trade: Trade);
    fn current_price(&self) -> Option<Price>;
    fn get_depth(&self, levels: usize) -> OrderBookDepth;
}

// Example usage:
let mut sim = SimpleMarketSimulator::new(
    100,     // Keep 100 trades of history
    0.1,     // 0.1% spread
    5,       // Maintain 5 levels of depth
    2.0,     // Each level has 2x avg trade size
);

// Process some trades
sim.process_trade(trade);

// Get current price and market depth
if let Some(price) = sim.current_price() {
    println!("Current price: {}", price.raw());
}

let depth = sim.get_depth(5);
println!("Best bid: {}, Best ask: {}", 
    depth.bids[0].price.raw(),
    depth.asks[0].price.raw());

3. Order Execution (backtest/src/execution/)

• Order types (Market, Limit, Stop, Take-Profit)
• Position tracking
• Leverage and margin handling
• Liquidation mechanics
• Fee calculation

// Example usage of the ExecutionEngine
let mut engine = ExecutionEngine::new(
    100_000.0,     // Initial cash balance
    10.0,          // Maximum leverage
    10.0,          // Maximum position size
);

// Place a leveraged long order
let order = Order::new(Side::Buy, OrderType::Market, 1.0)?
    .with_leverage(5.0)?;
let order_id = engine.place_order(order, &mut market)?;

// Check order status
if let Some(status) = engine.get_order_status(order_id) {
    match status {
        OrderStatus::Filled { size, avg_price } => {
            println!("Filled {} units at {}", size, avg_price.raw());
        }
        OrderStatus::PartiallyFilled { filled_size, avg_price } => {
            println!("Partially filled {} units at {}", filled_size, avg_price.raw());
        }
        _ => println!("Order is {:?}", status),
    }
}

// Monitor positions and check for liquidations
engine.update_positions(&market);
let liquidations = engine.check_liquidations(&market);
for (symbol, position) in liquidations {
    println!("Position {} liquidated at {}", symbol.0, position.entry_price.raw());
}

4. Strategy Framework (backtest/src/strategy/)

• Strategy interface
• Signal generation
• Position sizing
• Risk management rules

pub trait Strategy {
    fn on_trade(&mut self, trade: &Trade);
    fn on_candle(&mut self, candle: &Candle);
    fn generate_signals(&self) -> Vec<Signal>;
    fn analyze_market(&self, market: &dyn MarketSimulator) -> Vec<String>;
    fn analyze_positions(&self, positions: &[Position]) -> Vec<String>;
}

pub trait RiskManager {
    fn analyze_risk(
        &self,
        portfolio: &dyn Portfolio,
        positions: &[Position],
        market: &dyn MarketSimulator,
    ) -> RiskAnalysis;
    fn validate_signal(
        &self,
        signal: &Signal,
        portfolio: &dyn Portfolio,
        positions: &[Position],
    ) -> bool;
}

5. Analysis (backtest/src/analysis/)

• Technical indicators
  • SMA - Simple Moving Average
  • EMA - Exponential Moving Average
  • MACD - Moving Average Convergence Divergence
  • RSI - Relative Strength Index
  • BB - Bollinger Bands
• Custom indicator framework
• Window-based calculations
• Performance metrics

// Trading metrics
let mut stats = TradeStats::new();
stats.update(100.0, 1.0); // Add a trade with P&L and fees
println!("Win rate: {:.2}%", stats.win_rate() * 100.0);
println!("Profit factor: {:.2}", stats.profit_factor());

// Position analysis
let mut metrics = PositionMetrics::new();
metrics.update(&position, intended_entry, intended_exit);
println!("Avg leverage: {:.2}x", metrics.avg_leverage);
println!("Slippage: {:.2}%", metrics.entry_slippage * 100.0);

// Portfolio risk metrics
let mut portfolio = PortfolioMetrics::new();
portfolio.update_exposure(symbol, exposure);
portfolio.calculate_metrics();
println!("Diversity: {:.2}", portfolio.diversity_score);
pub trait Indicator {
    /// Update indicator with a new candle
    fn update(&mut self, candle: &Candle);
    /// Get current indicator value
    fn value(&self) -> Option<f64>;
}

/// Example usage of technical analysis:
let mut analysis = TechnicalAnalysis::new();
analysis.update(&candle);

// Access indicator values
if let Some(sma) = analysis.sma20() {
    println!("SMA-20: {}", sma);
}

if let Some(rsi) = analysis.rsi() {
    println!("RSI-14: {}", rsi);
}

if let Some((middle, upper, lower)) = analysis.bollinger_bands() {
    println!("Bollinger Bands - Middle: {}, Upper: {}, Lower: {}", middle, upper, lower);
}

// Get comprehensive analysis
let analysis_report = analysis.analyze();
for insight in analysis_report {
    println!("{}", insight);
}

6. Portfolio Management (backtest/src/portfolio/)

• Multi-asset tracking
• Balance management
• Exposure calculation
• Portfolio metrics

pub trait Portfolio {
    fn init_balance(&self) -> f64;
    fn current_balance(&self) -> f64;
    fn total_value(&self) -> f64;
    fn max_drawdown(&self) -> f64;
}

pub struct BasicPortfolio {
    fn new(init_balance: f64) -> Self;
    fn update_position(&mut self, position: Position);
    fn close_position(&mut self, symbol: &Symbol, exit_price: Price) -> f64;
    fn available_margin(&self) -> f64;
    fn margin_usage_ratio(&self) -> f64;
    fn unrealized_pnl(&self) -> f64;
    fn asset_exposure(&self, symbol: &Symbol) -> f64;
    fn max_drawdown_ratio(&self) -> f64;
}

Additional Components

7. Configuration (backtest/src/config/)

• Strategy parameters
• Risk limits
• Execution settings
• Fee structures

// Risk Configuration
pub struct RiskConfig {
    pub max_drawdown: f64,      // % as 0.0 to 1.0
    pub max_leverage: f64,      // e.g., 3.0 for 3x
    pub max_position_size: f64, // % as 0.0 to 1.0
    pub strict_enforcement: bool,
}

// Fee Configuration
pub struct FeeConfig {
    pub maker_fee: f64,       // e.g., 0.0002 for 0.02%
    pub taker_fee: f64,       // e.g., 0.0005 for 0.05%
    pub custom_fees: HashMap<Symbol, (f64, f64)>,
    pub include_fees: bool,
}

// Strategy Parameters
pub struct StrategyParams {
    pub fn new() -> Self;
    pub fn with_param(self, name: &str, value: f64) -> Self;
    pub fn get(&self, name: &str) -> Option<f64>;
    pub fn set(&mut self, name: &str, value: f64);
}

8. Reporting (backtest/src/report/)

• Trade logs
• Performance reports
• Data export
• Visualization tools

use backtest::{
    report::{
        TradeLog, TradeRecord, PerformanceReport,
        ChartData, ChartFormat, ChartOptions,
        generate_equity_curve, generate_position_chart,
    },
};

// Create a trade log with file output
let mut log = TradeLog::with_log_file("trades.log")?;

// Record trades
log.record(TradeRecord::new(
    timestamp,
    symbol,
    "buy",
    1.0,
    price,
    100.0, // P&L
    1.0,   // Fees
))?;

// Generate performance report
let report = PerformanceReport::new(
    stats,
    position_metrics,
    portfolio_metrics,
    positions,
);

// Export report to JSON
report.export_json("performance.json")?;

// Print formatted report
println!("{}", report.format());

// Generate and export charts
let mut chart_options = ChartOptions::default();
chart_options.format = ChartFormat::TradingView;

// Generate equity curve chart
let equity_chart = generate_equity_curve(&portfolio_metrics, &chart_options);
equity_chart.export("equity_curve.pine", &chart_options)?;

// Generate position visualization with candles and trades
let position_chart = generate_position_chart(&positions, &candles, &chart_options);
position_chart.export("trades.pine", &chart_options)?;

// Export data in other formats
chart_options.format = ChartFormat::Csv;
position_chart.export("trades.csv", &chart_options)?;

chart_options.format = ChartFormat::Json;
position_chart.export("trades.json", &chart_options)?;

9. Utilities (backtest/src/utils/)

• Time handling (timestamps, durations, time series)
• Data structures (ring buffers, time series buffers)
• Statistical functions (core stats, financial metrics)
• Logging and error handling

// Time handling
let t1 = Timestamp::from_unix_secs(1600000000.0);
let d = Duration::from_secs(60.0);
let t2 = t1 + d;
assert_eq!(t2.as_unix_secs(), 1600000060.0);

// Data structures
let mut buffer = RingBuffer::new(100);
buffer.push(1.0);
let time_series = TimeSeriesBuffer::new(100);
time_series.push(t1.as_unix_secs(), 1.0);

// Statistical functions
let mut stats = Statistics::new();
stats.update(100.0);
println!("Mean: {}, StdDev: {}", stats.mean(), stats.std_dev());

let mut metrics = FinancialMetrics::new(100);
metrics.update(100.0);
println!("Sharpe: {}, Sortino: {}", 
    metrics.sharpe_ratio(0.02),
    metrics.sortino_ratio(0.02));

// Logging
info!("Processing trade", "execution");
error!("Invalid order", "validation");
debug!("Position updated", "portfolio");

Performance Goals

• Process 1M trades per second on standard hardware

  // Benchmark results:
  Data Processing/process_100k_trades: 264.04 μs
  // = 378M trades/sec for basic processing
  
  Data Processing/process_100k_trades_with_market: 10.972 ms
  // = 9M trades/sec with market simulation
  
  Portfolio/update_100_positions: 2.726 μs
  // = 36.6M position updates/sec

• Memory-efficient data structures (ring buffers, time series buffers)
• Parallel processing support (sync collections, thread-safe logging)
• Efficient data storage and retrieval (CSV parsing, indexed access)

Testing Goals

• Unit tests for data processing components
• Unit tests for all other components (87 tests total)
• Integration tests for full system

// Integration test categories:
test_trading_workflow       // End-to-end trading workflow
test_risk_management       // Portfolio risk limits
test_market_behavior      // Market simulation realism
test_analysis_signals     // Technical indicators
test_market_impact        // Order execution & liquidity
test_market_correlation   // Multi-asset correlation
test_portfolio_risk       // Portfolio risk metrics

• Benchmark suite

cargo bench  # Run all benchmarks
cargo bench data  # Run only data processing benchmarks
cargo bench portfolio  # Run only portfolio benchmarks

• Known strategy validation tests

// XMR Swing Strategy validation tests:
#[test]
fn test_strategy_signals() {
    // Validates signal generation in oversold/overbought conditions
    let mut strategy = XmrSwingStrategy::new(70.0, 30.0, 1.0);
    
    // Generate test data and verify signal triggers
    // ...
}

#[test]
fn test_strategy_workflow() {
    // Tests complete strategy execution workflow
    let mut strategy = XmrSwingStrategy::default();
    let mut market = SimpleMarketSimulator::new();
    let mut portfolio = BasicPortfolio::new(100_000.0);
    
    // Run backtest simulation and verify results
    // ...
}

#[test]
fn test_strategy_risk_management() {
    // Validates position sizing and risk limits
    let mut strategy = XmrSwingStrategy::new(70.0, 30.0, 2.0);
    let mut engine = ExecutionEngine::new(100_000.0, 2.0, 5.0);
    
    // Test position size limits and risk management
    // ...
}

#[test]
fn test_market_correlation() {
    // Tests alignment of signals with technical indicators
    let mut strategy = XmrSwingStrategy::default();
    let mut technical = TechnicalAnalysis::new();
    
    // Generate price trends and verify signal correlation
    // ...
}

Usage

Currently under development. The data processing module is ready for use:

use backtest::{
    data::{DataSource, CandleBuilder},
    Candle, Trade,
};

// Create a data source
let mut source = KrakenCsvDataSource::new("data/Trading_History/XBT_USD_trades_2019.csv")?;

// Create a candle builder for 1-minute candles
let mut builder = CandleBuilder::new(60.0);

// Process trades and generate candles
while let Some(Ok(trade)) = source.next_trade() {
    if let Some(candle) = builder.update(&trade) {
        // Process completed candle
        println!("Completed candle: {:?}", candle);
    }
}

// Get the last incomplete candle if any
if let Some(last_candle) = builder.finish() {
    println!("Final incomplete candle: {:?}", last_candle);
}