This project helps users to initialize a new cluster.
For example, you can watch a use case vedio on Youtube: https://youtu.be/OvVaib5xllo
It is suitable for:
The only conditions you should have are:
it seems a little harder than above steps. But believe me, you deserve it!
on Mac, install easy_install and then fabric:
on Linux (Ubuntu), the same way (notice, do not use pip to install fabric, you will be disappointed; besides, apt-get install fabric is ok, but the version is 1.8 now (2016.4))
on Windows,... Good luck, guy. I have not tried how to install fabric on Windows. But it should not be a probelm. init EC2 cluster, for free-password-login(ubuntu and root). for hostname, for hosts file.
##how to use
modify passwd.ini
this is an example
[default]
#which section is actived
activeSession=
#the sudo user, only used for sudo tasks
root =
#passwd of sudoer
passwd =
#example of section
[server]
#uesr name, if use createUser task, this user name will be the new user
newuser =
#password of newuser
passwd =
#cluster ips, use comma to split
hosts=
#hostnames of cluster , the number of hostnames must equal with hosts. the hostnames is used for changeHostname tass
hostnames=
#locally jdk file. the related path is ./
jdk_source_file=
# when the jdk.tar.gz is unziped, the folder in the tar file.
jdk_folder=
# the ntp server address you want to sync
ntp_server=
# the allowed net that can sync with your ntp server
ntp_net=
# the allowed net mask that can sync with your ntp server
ntp_net_mask=
this is an instance:
[default]
activeSession=server
root = fit
passwd =111
[server]
newuser = hxd
passwd = 1112
hosts=192.168.130.3, 192.168.130.5 hostnames=s3, s5,
jdk_source_file=files/jdk1.8.77.tar.gz
jdk_folder=jdk1.8.0_77
ntp_server=192.168.130.2
[client]
newuser = hxd
passwd = 1112
hosts = 192.168.130.2
hostnames = s2
jdk_source_file=files/jdk1.8.77.tar.gz
jdk_folder=jdk1.8.0_77
ntp_net=192.168.130.0
ntp_net_mask=255.255.255.0
ntp_server=192.168.130.2
[all]
newuser = hxd
passwd = 1112
hosts=192.168.130.2,192.168.130.3, 192.168.130.5
hostnames=s2,s3, s5
jdk_source_file=files/jdk1.8.77.tar.gz
jdk_folder=jdk1.8.0_77
you can use this script like this
fab task_name
you can use fab help
to show how many tasks supportted.
current supportted tasks:
ssh hxd@node1 "jps"
, the terminal says: jps command not found, though we have set $JAVA_HOME and $PATH in .bashrc file. The reason is that .bashrc is ignored when we run a non-interactive login command. To solve that, use this task to repair the bashrc file.fab addDisk:device,location
. e.g., fab addDisk:/dev/sdb,/datab
some bigdata system and monitor system. you can use it like fab -f fab_bigdata taskname
current supported task:
installCollectd
you need add a section in passwd.ini
[collectd] #server ip tell the node that where to send the collected data #the receiver should have a receive process. e.g. influxdb and others #the most easy receiver is collected itself. In this way, you should modify the receiver's collectd configuration file yourself: /etc/collectd/collectd.conf #in network plugin: replace "Server" by "Listen" server_ip= server_port= #collect interval unit: second interval=5
runCollectd
fab -f fab_bigdata runCollectd:stop
or fab -f fab_bigdata runCollectd:start
installGangliaClient
installCouch
fab -f fab_bigdata installCouch:y
runCouch
fab -f fab_bigdata runCouch:'start'
or fab -f fab_bigdata runCouch:'stop'
or fab -f fab_bigdata runCouch:'start','y'
or fab -f fab_bigdata runCouch:'stop','y'
. 'y' means you have couchdb on each node##Amazon Ec2 (or other pem based ssh) in Amazon Ec2, we use a pem file to login instead of password.
To use the script on Amazon EC2. you need modify the script now. But do not worry, just a little thing you need to do: