Webcam Convolution Filter - WASM Learning Project

A real-time webcam application that applies convolution filters using a 5x5 matrix. This project is designed as a stepping stone for learning WebAssembly (WASM) by starting with a pure JavaScript implementation that can be easily replaced with WASM.

What You'll Learn

JavaScript Foundation

• Real-time video processing using Canvas API
• Image data manipulation with getImageData() and putImageData()
• Convolution algorithm implementation
• Performance optimization techniques

WASM Migration Path

• Clean function signatures designed for WASM interop
• Memory management patterns for image data
• Performance comparison between JS and WASM
• Gradual migration strategy from JS to compiled code

Project Structure

├── index.html          # Main application UI
├── style.css           # Styling
├── main.js             # Application logic and webcam handling
├── convolution.js      # Pure JS convolution implementation
└── README.md           # This file

Key Learning Concepts

Convolution Filters

• Edge Detection: Sobel, Laplacian operators
• Blur Effects: Box blur, Gaussian blur
• Sharpening: Unsharp mask, high-pass filters
• Custom Effects: Emboss, outline, artistic filters

Performance Considerations

• Memory Layout: How image data is stored and accessed
• Algorithm Complexity: O(n²) operations on pixel data
• Browser Optimization: Canvas rendering pipeline
• Bottleneck Analysis: CPU vs GPU vs memory bandwidth

WASM Learning Path

Phase 1: Understand the JS Implementation

1. Study the convolution algorithm in convolution.js
2. Experiment with different kernel matrices
3. Measure performance with browser dev tools
4. Understand memory patterns and data flow

Phase 2: Prepare for WASM

1. Identify the hot path (convolution function)
2. Understand the function signature and data types
3. Learn about linear memory and pointer arithmetic
4. Study how to pass image data between JS and WASM

Phase 3: Implement WASM Version

1. Choose your language (C, Rust, AssemblyScript)
2. Set up the build toolchain
3. Implement the convolution function
4. Create JS bindings and memory management
5. Compare performance with the pure JS version

Phase 4: Optimize and Extend

1. Add SIMD instructions for vectorized operations
2. Implement larger kernel sizes (5x5, 7x7)
3. Add multiple filter passes
4. Experiment with different memory layouts

Getting Started

npm install
npm run dev

Open your browser and:

1. Start the webcam
2. Modify the matrix values to see different effects
3. Open browser dev tools to see performance metrics

Common Convolution Kernels to Try

Identity (no change):

0 0 0 0 0
0 0 0 0 0
0 0 1 0 0
0 0 0 0 0
0 0 0 0 0

Gaussian Blur (σ≈1.0):

1  4   6  4 1
4 16  24 16 4
6 24  36 24 6
4 16  24 16 4
1  4   6  4 1

(Divide by 256 for normalization)

Edge Detection (Laplacian 5×5):

0  0 -1  0  0
0 -1 -2 -1  0
-1 -2 16 -2 -1
0 -1 -2 -1  0
0  0 -1  0  0

Motion Blur (Diagonal):

1 0 0 0 0
0 1 0 0 0
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1

(Divide by 5 for normalization)

Emboss (45°):

-2 -1 0 1 2
-1  0 1 2 1
 0  1 2 1 0
 1  2 1 0 -1
 2  1 0 -1 -2

(Add offset +128 to bring values into visible range)

Sharpen (Un-sharp mask style):

0  0  -1  0  0
0 -1  -1 -1  0
-1 -1  25 -1 -1
0 -1  -1 -1  0
0  0  -1  0  0

Performance Tips

• Use requestAnimationFrame() for smooth rendering
• Consider downscaling video for better performance
• Profile your code to find bottlenecks
• Understand when to use ImageData vs direct pixel manipulation

Next Steps for WASM

Once you're comfortable with the JavaScript implementation:

1. Research WASM toolchains (Emscripten, wasm-pack, etc.)
2. Learn about linear memory and how to share data
3. Understand the performance characteristics of different approaches
4. Experiment with SIMD and other optimization techniques

The goal is to replace the applyConvolution function in convolution.js with a WASM equivalent while keeping the same interface.