base docker compose to setup the data engineering env in local
APACHE-2.0 License
Welcome to StreamlineDE, an end-to-end data engineering project designed to demonstrate real-time...
Personal docker images for various data science software stacks
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage...
The project aims to automate content classification and knowledge retrieval, as well as to perfor...
Apache Spark docker image
This project integrates real-time data processing and analytics using Apache NiFi, Kafka, Spark, ...
Quickly setup and simulate a multi node spark cluster using docker and docker-compose.
Dockerizing an Apache Spark Standalone Cluster
A Spark cluster setup running on Docker containers
Some typical docker compose templates.
Study project for big data (Hadoop, Zookeeper, Kafka, Flink, Spark)
An end-to-end data engineering pipeline to collect, store, process, and analyze property and crim...
Apache Hadoop docker image
Utilizing my background and love for Apache Airflow and Data to build a real-time data streaming ...