Skip to content

feat: implement local Data Lake stack (Athena, Glue, and Firehose)#429

Merged
hectorvent merged 5 commits intomainfrom
feat/athena-firehose-glue
Apr 15, 2026
Merged

feat: implement local Data Lake stack (Athena, Glue, and Firehose)#429
hectorvent merged 5 commits intomainfrom
feat/athena-firehose-glue

Conversation

@hectorvent
Copy link
Copy Markdown
Collaborator

@hectorvent hectorvent commented Apr 14, 2026

Summary

  • Implement AWS Glue Data Catalog for metadata and schema management
  • Implement Amazon Athena query engine
  • Implement Amazon Data Firehose for streaming ingestion with Parquet conversion
  • Add comprehensive end-to-end integration tests for the full Data Lake lifecycle

Closes #318
Closes #118

Type of change

  • Bug fix (fix:)
  • New feature (feat:)
  • Breaking change (feat!: or fix!:)
  • Docs / chore

AWS Compatibility

Checklist

  • ./mvnw test passes locally
  • New or updated integration test added
  • Commit messages follow Conventional Commits

@hectorvent hectorvent marked this pull request as draft April 14, 2026 06:21
@hectorvent hectorvent marked this pull request as ready for review April 15, 2026 06:19
@hectorvent hectorvent merged commit ea597c6 into main Apr 15, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEAT] Add support for Amazon Athena [FEAT] Support Amazon Data Firehose

1 participant