DB Toolkit

A consolidated collection of database utilities: ETL, schema tooling, Oracle/SQL Server inspection, and banking data helpers. Built for large datasets with chunked loaders and schema-driven workflows.

Highlights

• Bank file inventory with Jalali date extraction
• Excel/CSV to SQL Server loaders
• Schema extraction, table creation, and data loading from JSON
• Oracle schema dump and SQL Server explorer

Modules

• Bank: Inventory generator and CSV loader for SQL Server
• ETL: Excel/CSV loaders with chunked ingestion
• Schema: Extract schema from files, build tables, load data
• Oracle: Schema dump to a structured report
• SQL Server: Sample row explorer

Requirements

• Python 3.10+
• Database drivers (ODBC Driver 17 for SQL Server)
• See requirements.txt for Python packages

Install

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

Configuration

Templates live in configs/. Copy the template you need and rename to config.json or pass it with --config.

Run

$env:PYTHONPATH = "src"

# Bank inventory
python -m db_toolkit.cli bank inventory --root D:/Data --out inventory.csv --metadata configs/bank_metadata.example.json

# ETL
python -m db_toolkit.cli etl load-excel --db configs/etl_pipeline.example.json --excel D:/Data/file.xlsx
python -m db_toolkit.cli etl load-csv --db configs/etl_pipeline.example.json --csv D:/Data/file.csv --table MyTable

# Schema tools
python -m db_toolkit.cli schema extract --input D:/Data --output schema.json
python -m db_toolkit.cli schema build --config schema.json
python -m db_toolkit.cli schema load --config schema.json

# Oracle schema dump
python -m db_toolkit.cli oracle --config configs/oracle_schema_dumper.example.json

# SQL Server explorer
python -m db_toolkit.cli sqlserver explore --config configs/sqlserver_schema_explorer.example.json

You can also use:

.\scripts\run.ps1

Project Layout

• src/db_toolkit/bank/: Bank inventory and helpers
• src/db_toolkit/etl/: ETL pipeline
• src/db_toolkit/schema/: Schema tools
• src/db_toolkit/oracle/: Oracle schema dumper
• src/db_toolkit/tools/: SQL Server explorer and utilities
• configs/: Template configs
• sql/: SQL scripts

Security

• Never commit config.json.
• Use templates only in configs/.

Roadmap

• Unified YAML config with tool profiles
• Automated data quality checks
• Incremental loads and upserts
• CI with Dockerized databases