The HOPE Deduplication Engine (HDE) is a service in the HOPE ecosystem that detects duplicate individuals in a dataset by comparing their facial photographs. Client systems register batches of images through a REST API, trigger asynchronous processing (face quality assessment, encoding, and comparison), and read back duplicate findings with similarity scores.
Full documentation is at https://unicef.github.io/hope-dedup-engine/:
- Concepts — architecture, set lifecycle, data model, face pipeline
- Integration Guide — for client systems using the API
- Administration — deployment, configuration, operations
- Development — contributing to the engine itself
Requirements: uv and Docker, or see the full setup guide.
git clone https://github.com/unicef/hope-dedup-engine.git
cd hope-dedup-engine
docker compose up --buildThen open http://localhost:8000/admin/ (adm@hde.org / 123) and the API docs at http://localhost:8000/api/rest/swagger/.
For a native (non-Docker) environment:
uv venv .venv
uv sync
./manage.py env --develop > .envrc # create initial development configuration
direnv allow . # enable environment
createdb hope_dedup_engine # create postgres database on localhost
./manage.py upgrade
./manage.py runserver