Smart Integration Microservice

Smart Integration Microservice for schema matching, mapping, delineation and correlation, built with FastAPI.

Project structure

Directory structure:

• src - production source code
• src/modules - domain oriented modules with API routes and implementation
• src/common - common code and utils
• test/unit - unit tests
• test/integration - integration tests

Important files:

• server.py - hypercorn server entry point
• src/app.py - FastAPI entry point
• src/config.py - project configuration
• pyproject.toml - dependencies, tools, tasks
• Dockerfile - docker file

API Documentation

App exposes API documentation on the URLs below:

• OpenAPI UI: http://localhost:8090/docs
• ReDoc: http://localhost:8090/redoc

Configuration

App is configured using environment variables, see the default configuration in src/config.py and samples in .env-example.

For local development you can take advantage of dotenv plugins that read .env file from the project root.

Make sure you have LLM__OPENAI_API_KEY secret configured as there is no default for it and it is required to run the applictaion.

# copy and use example dev configuration
cp .env-example .env

# copy and use configuration for unit/integration tests
cp .env.test-example .env.test

Logging

Set LOGGING__LEVEL to debug, info (default), warning, error, or critical. Application request logs use INFO for successful responses, WARNING for 4xx responses, and ERROR for 5xx responses and unhandled failures. They include the HTTP method, path, status, and elapsed time until the response headers are available. Successful /health requests are logged only at DEBUG.

Use LOGGING__LEVEL=debug to see request starts and LLM chain starts/completions, including chain duration. Application logs include a generated request_id to correlate processing within a request; responses handled by the request middleware also expose it in the X-Request-ID header. The new request summaries omit query parameters, headers, and request/response bodies. Existing module error logs may include exception details; DEBUG is not a payload-redaction setting.

Hypercorn access logs are controlled separately with LOGGING__ACCESS_LOG (default true). Set it to false if the application request summaries are sufficient. Use LOGGING__COLORS=true for colored terminal output.

Running with Docker

Requirements

• Docker
• Docker Compose

Run using docker compose

# build image
docker compose build

# start container
docker compose up

Local Development

Prerequisites

• Python 3.12+
• UV dependency manager

NOTE: tasks are run with a poethepoet tool and configured in pyproject.toml

Run the app

uv run poe start
# access the service at http://localhost:8090
# e.g. `curl http://0.0.0.0:8090/health`

Installing dependencies

dev dependency group is used to distinguish from production ones.

# install production dependency
uv add mydep

# install dev dependency
uv add --dev mydep

Quality assurance

All code should adhere to the quality checks below. It is highly recommended to instal pre-commit hook and also integrate these tools below in the IDE.

Linting and formatting

Quality checks using ruff and mypy.

uv run poe typecheck
uv run poe lint
uv run poe stylecheck

# optionally run all quality checks (including unit tests)
uv run poe qa

# attempt to fix formatting and lint errors
uv run poe fix

Tests

Running tests using pytest.

# run all tests
uv run poe test

# run in watch mode
uv run poe test-watch

# run unit tests only
uv run poe test test/unit

# run integration tests only
uv run poe test test/integration

Pre-commit hooks

This project uses pre-commit to ensure consistent code style and other quality checks.

# install the hooks from .pre-commit-config.yaml
# this needs to be done just once when setting up project
uv run pre-commit install

Once installed, the hooks will automatically run every time you commit changes. If any issues are found or files are modified, the commit will be aborted until fixed.

Langfuse configuration

For development and testing purposes is every api request and llm call traced with Langfuse. By default is langfuse tracing disabled, you can enable it by configuration:

# configure langfuse host
LANGFUSE__HOST=https://my-langfuse-host.xyz
# configure correct langfuse project keys
LANGFUSE__SECRET_KEY=project-secret-key
LANGFUSE__PUBLIC_KEY=project-public-key
# when using for development define your own environment
LANGFUSE__ENVIRONMENT=dev-myname
# enable langfuse
LANGFUSE__TRACING_ENABLED=true

Technical notes

• API endpoints and parameters have to follow camel case convention
• LLM model can be configured via any OpenAI-compatible chat API