EnginBERT is a domain-specific BERT model designed to create high-quality text embeddings for engineering literature. It's specifically trained on engineering research papers to better understand technical and scientific content.
- Domain-specific BERT model trained on engineering papers
- Automated data collection from arXiv
- Custom preprocessing pipeline for academic papers
- Fine-tuning with Masked Language Modeling (MLM)
- Sequence classification capabilities
- Evaluation metrics for clustering, citations, and information retrieval
- Command-line interface for easy training and evaluation
- Python 3.10 or higher
- pip (Python package installer)
To install EnginBERT locally, use:
pip install -e .For development purposes, install with additional dependencies:
pip install -e ".[dev]"This includes testing, linting, and development tools.
To build and work with the documentation:
pip install -e ".[docs]"EnginBERT provides a convenient CLI for all major operations:
Train the model from scratch:
enginbert trainSkip specific training steps:
enginbert train --skip data preprocessRun all evaluation metrics:
enginbert evaluateRun specific evaluation metrics:
enginbert evaluate --metrics clustering ir citationsRun the entire pipeline (training and evaluation):
enginbert run-allEnginBERT/
├── scripts/
│ ├── data_processing/ # Data collection and preprocessing
│ ├── evaluation_metrics/ # Model evaluation tools
│ ├── helpers/ # Utility functions
│ ├── tokenizer/ # Custom tokenization
│ └── train/ # Training scripts
The project follows the Black code style. To format your code:
black .Run flake8 for code quality checks:
flake8 .MIT license
Jan Heimann - jan_heimann@icloud.com | Tristan Kruse - krusetristan1@gmail.com