base docker compose to setup the data engineering env in local
APACHE-2.0 License
The Namenode is the master node which persist metadata in HDFS and the datanode is the slave node which store the data. When you insert data or create objects into Hive tables, data will be stored in HDFS on Hadoop DataNodes and the NameNode will keep the tracking of which DataNode has the data.
Hue is an open source SQL Assistant for Databases & Data Warehouses, It is not necessary for a big data ecosystem, but it can help you visualize data in HDFS faster, and other notable features.
shanks@pc cd sparkini/docker
docker-compose up -d
Took the inspirations from https://github.com/fabiogjardim