PacketX Lakehouse is a Cloud & Local based medium-sized data platform designed to handle and analyze network packets traffic data. The project follows a modern medium-sized lakehouse architecture, integrating Apache Iceberg & AWS S3 Bucket Storage (Lakehouse), Redshift & Postgres (Warehouse), DynamoDB, Docker, and Apache Airflow.
postgres data airflow sqlite lambda-functions data-engineering s3-storage warehouse iceberg etl-pipeline flyway-migrations lakehouse duckdb
- Updated
Apr 10, 2025 - Python