-
Notifications
You must be signed in to change notification settings - Fork 282
add mxfp8 qat example. #2316
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add mxfp8 qat example. #2316
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds a new example for MXFP8 Quantization Aware Training (QAT) for large language models using Hugging Face transformers. The example demonstrates how to perform QAT on LLMs with the Neural Compressor library.
- Provides a complete QAT workflow with utilities for data processing and model quantization
- Implements training pipeline with MXFP8 quantization scheme integration
- Includes comprehensive documentation and configuration files for distributed training
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| utils.py | Data processing utilities for the Daring-Anteater dataset and metrics calculation |
| requirements.txt | Dependencies for the QAT example including neural-compressor and transformers |
| main.py | Main training script with quantization integration and model export functionality |
| accelerate_config/fsdp1.yaml | FSDP configuration for distributed training |
| accelerate_config/ddp.yaml | DDP configuration for multi-GPU training |
| README.md | Documentation with step-by-step QAT workflow instructions |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
examples/pytorch/nlp/huggingface_models/language-modeling/quantization/llm_qat/utils.py
Outdated
Show resolved
Hide resolved
examples/pytorch/nlp/huggingface_models/language-modeling/quantization/llm_qat/main.py
Outdated
Show resolved
Hide resolved
examples/pytorch/nlp/huggingface_models/language-modeling/quantization/llm_qat/main.py
Outdated
Show resolved
Hide resolved
…tization/llm_qat/main.py Co-authored-by: Copilot <[email protected]>
…tization/llm_qat/main.py Co-authored-by: Copilot <[email protected]>
…tization/llm_qat/utils.py Co-authored-by: Copilot <[email protected]>
Type of Change
add mxfp8 qat example
Description
mxfp8 qat