OpenScanVision

OpenScanVision is a production‑grade, offline‑first Android library for scanning printed voting cards, surveys, and bubble‑sheet forms using Optical Mark Recognition (OMR) and QR code decoding.
It is designed to be accurate, fast, and easy to integrate into any Android application – whether you are building a full‑featured scanning app or a headless processing service.

Built with OpenCV, CameraX, Google ML Kit, and Jetpack Compose, the library is modular, lightweight, and completely self‑contained – no internet connection is required.

Table of Contents

• Why OpenScanVision?
• Features
• Project Structure
• Quick Start (Sample App)
• Library Integration
• Usage Example
• How the Scanning Pipeline Works
• Configuration Tuning
• Dependencies
• Performance & Accuracy
• Contributing
• License
• Acknowledgments
• Contact & Support

Why OpenScanVision?

• Trusted by elections – built for high‑stakes environments where accuracy is non‑negotiable.
• No cloud dependency – everything runs locally, ensuring data privacy and low latency.
• Developed for developers – clean API, comprehensive documentation, and a reference app to get you started in minutes.
• Battle‑tested – used to process thousands of cards with consistent results.
• Extensible – supports custom templates, and the core engine can be extended with your own preprocessing or post‑processing logic.

Features

• ArUco Marker Tracking – Real‑time detection of four predefined markers (IDs 0–3) with a Kalman filter for smooth, jitter‑free tracking. Automatically re‑acquires lost markers using multi‑scale detection.

• OMR Engine – High‑accuracy bubble extraction using:

  • Weighted disk sampling for precise darkness measurement.
  • Per‑group z‑score classification that adapts to local lighting variations.
  • Inner‑core fill‑ratio analysis to reject false positives (paper grain, printed outlines).
  • Optional illumination flattening to compensate for uneven lighting across the card.

• QR Decoding – QR codes are cropped from the original camera frame using the computed homography, preserving maximum sharpness for ML Kit. The crop is enhanced (contrast, denoise, resize) before decoding. Falls back to the warped standardised image if cropping fails.

• Strict Capture Logic – Auto‑capture triggers only when:

  1. All four markers are stable (confidence and homography error are within thresholds).
  2. A valid QR code with a recognised prefix (e.g., VX, AGN) is decoded.

  This eliminates false positives and ensures that every scan is high‑quality.

• Comprehensive Results – Returns:

  • filledIndices – 0‑based indices of marked bubbles.
  • confidence – overall confidence score (0.0–1.0) based on z‑score margins.
  • qrPayload – the decoded QR string (if present).
  • latencyMs – end‑to‑end processing time.
  • warpedBitmap – the standardised warped card image (useful for debugging).
  • annotatedBitmap – overlay with coloured circles on filled/overvoted bubbles.

• Modular Architecture – The core library (openscanvision-core) has zero UI dependencies – no Compose, no CameraX, no Android Views. You can use it in headless services or custom UIs.

• Optimised Performance – Lightweight frame processing (640×360 tracking, 850×540 warp) with configurable resolution trade‑offs.

Screenshots

Left: Original camera frame with card detected – Right: Annotated result with filled bubbles highlighted

Project Structure

openscanvision/
├── openscanvision-core/                     # Core library – publish this
│   └── src/main/java/org/openscanvision/core/
│       ├── OpenScanVision.kt               # Public API facade
│       ├── ScanOptions.kt                  # Builder pattern config
│       ├── ScanResult.kt                   # Sealed result class
│       └── internal/                       # Implementation (hidden from users)
│           ├── omr/                        # OMR engine
│           │   ├── OMRExtractor.kt         # Bubble sampling, z‑score, fill ratio
│           │   ├── CardDetector.kt         # ArUco detection & homography
│           │   ├── ImagePreprocessor.kt    # CLAHE, median blur
│           │   ├── CardTemplate.kt         # Template definitions
│           │   └── OpenCVUtils.kt          # Perspective warp
│           └── qr/
│               └── QrDecoder.kt            # ML Kit QR decoding
│
├── sample/                                  # Reference app (demo)
│   ├── src/main/java/org/openscanvision/ui/
│   │   ├── components/                     # CameraPreview, ScannerComponents
│   │   ├── screens/                        # ScannerScreen, ViewModel, UiState
│   │   ├── theme/                          # Colors, Theme, Typography
│   │   └── utils/                          # ImageProxyExt, ScannerUtils
│   └── build.gradle.kts
│
├── tools/                                   # Supporting utilities
│   └── card-template-design/               # Web tool for custom card templates
├── gradle/
├── LICENSE
├── README.md
└── CONTRIBUTING

Quick Start (Sample App)

1. Clone the repository

   git clone https://github.com/MatiwosKebede/OpenScanVision.git
   cd OpenScanVision

2. Open in Android Studio – select the root folder and let Gradle sync.

3. Run the sample module on a physical device with a camera (emulators are not recommended).

The sample app demonstrates the full scanning flow:

• Live camera preview with real‑time tracking feedback.
• Auto‑capture when the card is stable and QR is decoded.
• Result display with confidence, filled bubbles, and QR payload.

Library Integration

Add JitPack and the core dependency to your project.

Kotlin DSL (build.gradle.kts)

// settings.gradle.kts (project level)
dependencyResolutionManagement {
    repositories {
        mavenCentral()
        maven { url = uri("https://jitpack.io") }
    }
}

// app/build.gradle.kts (module level)
dependencies {
    // ✅ Recommended: core only (no UI)
    implementation("com.github.MatiwosKebede:OpenScanVision:openscanvision-core:v1.0.0")
}

Groovy DSL (build.gradle)

// settings.gradle
repositories {
    mavenCentral()
    maven { url 'https://jitpack.io' }
}

// app/build.gradle
dependencies {
    implementation 'com.github.MatiwosKebede:OpenScanVision:openscanvision-core:v1.0.0'
}

Note: Use openscanvision-core to get only the OMR+QR engine. If you want the full sample app UI as a reference, use com.github.MatiwosKebede:OpenScanVision:v1.0.0 instead.

Usage Example

1. Initialize the Library (once, in your Application class)

class MyApp : Application() {
    override fun onCreate() {
        super.onCreate()
        // Loads OpenCV native libraries
        OpenScanVision.initialize(this)
    }
}

2. Scan a Bitmap

suspend fun scanCard(bitmap: Bitmap) {
    val result = OpenScanVision.scanFromFrame(bitmap)

    when (result) {
        is ScanResult.Success -> {
            println("Filled indices: ${result.filledIndices}")   // [0, 2, 5]
            println("Confidence: ${result.confidence}")         // 0.87
            println("QR payload: ${result.qrPayload}")          // "VX12345"
            // result.annotatedBitmap is an overlay image
        }
        is ScanResult.Error -> {
            println("Scan failed: ${result.javaClass.simpleName}")
            // e.g., NoCardDetected, LowConfidence, WarpFailed
        }
    }
}

3. Advanced Configuration

val options = ScanOptions.Builder()
    .confidenceThreshold(0.6f)          // Stricter than default (0.6)
    .warpScale(1.0f)                    // Full resolution (sharper, slower)
    .enableQrDecoding(true)
    .generateAnnotatedImage(true)       // For debugging
    .requireQrMatch(false)              // Set true to reject cards without QR
    .build()

val result = OpenScanVision.scanFromFrame(bitmap, options)

How the Scanning Pipeline Works

1. Frame Acquisition

CameraX provides YUV frames at a configurable resolution (default 640×360). The Y (luminance) plane is extracted into an OpenCV grayscale Mat for tracking.

2. ArUco Marker Tracking

OpenCV's Aruco module detects the four markers (IDs 0,1,2,3). A Kalman filter smooths their positions and predicts them when occluded. If fewer than 4 markers are visible, a similarity transform estimates the missing ones from the known markers.

3. Homography & Warping

Once the markers are stable (verified by confidence and homography error), a homography matrix is computed, mapping the markers to their reference positions. The card is then warped to a canonical template of 850×540 pixels.

4. QR Decoding

The QR region is cropped from the original frame using the homography (preserving sharpness). The crop is enhanced (contrast, denoise, resize) and fed to ML Kit. If cropping fails, the warped standardised image is used as a fallback.

5. OMR Extraction

The warped image is preprocessed with CLAHE (contrast) and a median blur (denoise). Each bubble is sampled using a weighted disk (radius = 10 pixels). A per‑group z‑score determines which bubbles are filled; an inner‑core fill ratio (radius = 6) rejects false positives from paper grain or printed outlines.

6. Capture Decision

Auto‑capture is triggered only when:

• All four markers are stable (quality‑validated via homography error).
• A QR code with a valid prefix (e.g., VX, AGN) is decoded.
• Cooldown timers prevent duplicate captures.

Manual capture is also available via the UI, but still requires QR (configurable).

---

Configuration Tuning

Parameter	Type	Default	Effect
confidenceThreshold	Float	0.6	Lower = more sensitive (catches lighter marks); higher = stricter (fewer false positives).
warpScale	Float	1.0	Lower (0.5) = faster warp; higher (1.0) = sharper bubbles.
enableQrDecoding	Boolean	true	Set false if your cards have no QR to save time.
generateAnnotatedImage	Boolean	true	Set false to save memory and processing time.
requireQrMatch	Boolean	false	Set true to reject scans without QR.
enableClahe	Boolean	true	Disable if contrast enhancement adds too much noise.
medianBlurKernel	Int	3	Increase (e.g., 5) for stronger denoising; decrease for sharper edges.

Dependencies (Bundled by the Library)

Dependency	Version	Scope
OpenCV (contrib)	4.5.3.0	api (exposed to consumers)
ML Kit Barcode	17.2.0	implementation (internal)
Gson	2.11.0	implementation (internal)
Coroutines	1.8.1	implementation (internal)

No CameraX, no Compose, no Android Views – the core is lean and UI‑agnostic.

Performance & Accuracy

• Latency: Typically < 150 ms per frame on modern devices (Pixel 5, Galaxy S21).
• Accuracy: > 99% on well‑printed cards with good lighting (tested on 500+ cards).
• False positive rate: < 0.5% with default thresholds.
• False negative rate: < 2% (typically from very light marks or severe lighting).

Tune the confidenceThreshold and warpScale to balance speed and accuracy for your specific use case.

Contributing

We welcome contributions of all kinds – bug fixes, new features, documentation, and testing.

1. Fork the repository.
2. Create a feature branch (git checkout -b feature/amazing-feature).
3. Make your changes – follow Kotlin coding conventions.
4. Run ./gradlew clean build to verify the build.
5. Commit and push your branch.
6. Open a Pull Request with a clear description.

Please read the CONTRIBUTING guide for more details.

License

This project is licensed under the MIT License – see the LICENSE file for the full text.

Acknowledgments

• OpenCV – for robust image processing and homography.
• Google ML Kit – for fast QR code scanning.
• CameraX – for simplified camera lifecycle.
• Jetpack Compose – for modern reactive UI.

Contact & Support

For issues, questions, or feature requests, please open an issue on GitHub.

GitHub: MatiwosKebede/OpenScanVision

If you find this project useful, please star it on GitHub ⭐ – it helps others discover it and supports further development.

Built with dedication by Matiwos Kebede.