title	author
docker - Big data tools project	Abdellatif Ahammad

Docker Big data tools

:flag-ma: the main principle of this repo is to have the complete tools of big data run on the way and ready to use.

🎈 there are two ways there so you can choose anyone you want .

1- the Cloudera Manager and CDH quick start

this offers a Hadoop cluster from 2014, with Cloudera express manager (free version), that gives a nice experience to launch and stop any tool from its UI.

✒️ the tools that exist in this cluster

Name	Port
HDFS	8002
Cloudera Manager	7180
Hue	8888
Hive	10002
Oozie	11000
Zookeeper	2181
Solr	8983
Sqoop Metastore	16000
Impala	-
Spark Master	7077

quickstart 💯

use only the hadoop cluster without the cloudera manager :first_place_medal:

    git clone https://github.com/abdellatifAhammad/Dockerized-bigdata-tools
    cd docker-bigdata/cloudera_express
    sudo docker-compose up

you will see something like this at the beginning

use the cluster with the cloudera manager ⛳ inside the cloudera_express folder use this

    sudo docker attach abdo_cdh
    /home/cloudera/cloudera-manager --express

it takes some time to be launched

Examples services UI :

Hue Hive inside Hue (hive editor) Impala cloudera_manager Interface you have to launch the service that you want manually from this dashboard HBase

2 - big data tools without CDH (only Docker Images from dockerhub )

this one is a Hadoop cluster that contains multiple data nodes (just 3 in this docker-compose file you can make more of them)

I want to say thanks to the creators of these images in the docker hub 🤙

• fjardim
• streamset
• fmantuano

quick start

    git clone https://github.com/abdellatifAhammad/Dockerized-bigdata-tools
    cd docker-bigdata/big_data
    sudo dcoker-compose up -d

then you can check all these nice tools (check ports from the docker-compose file)

- Hive
- hue
- zookeper
- mysql
- kafka
- hbase
- mongo
- sqoop metabase
- streamsets
- storm