End-to-end-realtime-data-streaming

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage...

hdfs-stream-processing

Streaming data processing using Hadoop HDFS, Spark, Kafka, Minio, Elasticsearch

pyspark-realtime-streaming-sentiment-analysis

⏱ Real-Time Sentiment Analysis using PySpark and simulation of Twitter/X API using FastAPI

Spark-Structured-Streaming-Examples

Spark Structured Streaming / Kafka / Cassandra / Elastic

StreamlineDE-

Welcome to StreamlineDE, an end-to-end data engineering project designed to demonstrate real-time...

Lambda-Arch-Spark

spark-structured-streaming

Spark structured streaming with Kafka data source and writing to Cassandra

e2e-data-engineering

An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage...

SparkPythonBigData

Big-Data with Apache Spark and Python.

spark-application

Java Application, uses Apache Spark, handles batch as well as streaming processing

Reddit-Analysis

Sentiment Analysis on streaming data and batch data from Reddit

DataStreamingETL

Utilizing my background and love for Apache Airflow and Data to build a real-time data streaming ...

e2e-structured-streaming

End-to-end data pipeline that ingests, processes, and stores data. It uses Apache Airflow to sche...

ECommerce-Dive-Deep-Sales-Analysis

In this project, we developed an ETL pipeline using Apache Airflow to process delivery data and t...

Credit-Card-Fraud-Detection-Spark