Play deployment

Novu

readme for deployment with ansible

Instructions and logs

• Please open 2_Frontend folder, and firstly edit play-docker-deploy/run.sh to update the hostname to IP address mapping using --add-host flag, e.g. --add-host workstation1:192.168.0.131, where workstation1 is the hostname. Please add the hostname to IP address mapping for all machines in the cluster.

• Secondly edit play-docker-deploy/config/config.properties, to update all the values accordingly.

  • For cluster with infiniband, please add IP_TO_IBIP_MAPPING into the config.properties. This is used to map ethernet IP addresses to infiniband IP addresses. The following is an example,

  IP_TO_IBIP_MAPPING=172.27.10.142->192.168.56.142,172.27.10.143->192.168.56.143,172.27.10.144->192.168.56.144,172.27.10.145->192.168.56.145

  • For the machines with multiple NUMA zones, please add the NUM_NUMA_ZONE and CPU_SET_NUMA properties into the config.properties file. NUM_NUMA_ZONE is used to configure hwo many NUMA zones. And, CPU_SET_NUMA is used to configure cpu-set of each zone. The following is an example,

  NUM_NUMA_ZONE=2
  CPU_SET_NUMA=0-5,12-17;6-11,18-23

• Edit Ansible hosts2 file (you can share the same host file as 1.Cluster). For more information, please refer to http://docs.ansible.com/ansible/intro_inventory.html

• Then run the ansible configuration command to copy frontend server docker image/config files and execute the server docker image:

cd 2_Frontend
ansible-playbook -i hosts2 ansible/run_play.yml -k -K

• The log is:

p1@p1-M32CD:~/Novu/mesos_mpi_caffe/ansible⟫ ansible-playbook -i ~/hosts2 run_play.yml -k -K                                                                                                                                             
SSH password:
SUDO password[defaults to SSH password]:
PLAY [masters] *************************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] *****************************************************************************************************************************************************************************************************************
Monday 19 June 2017  16:06:40 -0700 (0:00:00.019)       0:00:00.019 ***********
ok: [192.168.0.245]
TASK [play_server : Create /opt/NovuPlay folder in server node] ************************************************************************************************************************************************************************
Monday 19 June 2017  16:06:41 -0700 (0:00:00.873)       0:00:00.893 ***********
ok: [192.168.0.245] => (item=absent)
changed: [192.168.0.245] => (item=directory)
TASK [play_server : Copy image, shell from ansible machine to server node (might be master node)] **************************************************************************************************************************************
Monday 19 June 2017  16:06:42 -0700 (0:00:01.180)       0:00:02.073 ***********
changed: [192.168.0.245]
TASK [play_server : Load image from archive] *******************************************************************************************************************************************************************************************
Monday 19 June 2017  16:10:53 -0700 (0:04:10.274)       0:04:12.348 ***********
changed: [192.168.0.245]
TASK [play_server : Run play-java docker container shell script in remote shell] *******************************************************************************************************************************************************
Monday 19 June 2017  16:15:22 -0700 (0:04:29.014)       0:08:41.363 ***********
changed: [192.168.0.245]
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
192.168.0.245              : ok=5    changed=4    unreachable=0    failed=0   
Monday 19 June 2017  16:15:25 -0700 (0:00:02.847)       0:08:44.211 ***********
===============================================================================
play_server : Load image from archive --------------------------------- 269.01s
play_server : Copy image, shell from ansible machine to server node (might be master node) - 250.27s
play_server : Run play-java docker container shell script in remote shell --- 2.85s
play_server : Create /opt/NovuPlay folder in server node ---------------- 1.18s
Gathering Facts --------------------------------------------------------- 0.87s
p1@p1-M32CD:~/Novu/mesos_mpi_caffe/ansible⟫

• Now login to server and check docker container: docker ps -a.

[hengz@c0 NovuPlay]$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
play-novu           1.0-SNAPSHOT        91fb2f9d3018        59 minutes ago      4.81 GB
prom/prometheus     latest              4da113bb6ae3        10 days ago         74.5 MB
grafana/grafana     latest              f79ce49a1309        3 weeks ago         286 MB
zookeeper           latest              5291027d4199        6 weeks ago         143 MB
[hengz@c0 NovuPlay]$ docker ps
CONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                                                    NAMES
79056ca8ec20        play-novu:1.0-SNAPSHOT   "bin/play-novu"          28 minutes ago      Up 28 minutes                                                                                play-9000
1cddff7fae69        grafana/grafana          "/run.sh"                3 days ago          Up 3 days           0.0.0.0:3000->3000/tcp                                                   grafana
15e52620b436        prom/prometheus          "/bin/prometheus -..."   3 days ago          Up 3 days           0.0.0.0:9090->9090/tcp                                                   prometheus
0ea29be7a4b9        zookeeper                "/docker-entrypoin..."   6 days ago          Up 4 days           0.0.0.0:2181->2181/tcp, 0.0.0.0:2888->2888/tcp, 0.0.0.0:3888->3888/tcp   novu_zookeeper
[hengz@c0 NovuPlay]$ exit

• Now you can access the web app: [master_ip]:9000. modify the ip addr according to your case.

FAQ

• How to stop the docker container in case of failure.
  
  • Ans: goto server node and find the docker container's ID by docker ps -a, copy the ID for play-novu:1.0-SNAPSHOT and do command: docker stop {put the ID here}.
• How to restart play frontend server docker?
  
  • Ans: goto server node and type command: sh /opt/NovuPlay/run.sh.