JStorm的搭建文档

一、下载jstorm的jar包
　　https://github.com/alibaba/jstorm/releases

二、解压jstorm的包
　　tar -xvf jstorm-2.4.0.tgz
　　mv jstorm-2.4.0 jstorm
修改环境变量：
　　vi /etc/profile
　　　　export JSTORM_HOME=/usr/local/jstorm
　　　　export PATH=$PATH:$JSTORM_HOME/bin
　　生效 source /etc/profile

二、上传并修改storm.yaml
　　mv storm.yaml storm.yaml.bak

　　vi storm.yaml
　　nimbus.host: 表示nimbus的地址， 填写ip

storm.zookeeper.servers: - "zk-datanode-01"
     - "zk-datanode-02"
     - "zk-datanode-03" nimbus.host: "192.168.46.160" storm.zookeeper.root: "/jstorm" storm.local.dir: "/usr/local/jstorm/data" supervisor.slots.ports: - 6800
    - 6801
    - 6802
    - 6803

三、部署到其余节点
　　scp -r jstorm zk-datanode-02:/usr/local/
　　scp -r jstorm zk-datanode-03:/usr/local/

四、启动storm
　　在nimbus 节点上执行
　　nohup jstorm nimbus & 查看$JSTORM_HOME/logs/nimbus.log检查有无错误

　　在supervisor节点上执行
　　nohup jstorm supervisor & 查看$JSTORM_HOME/logs/supervisor.log检查有无错误
　　若没有配置环境变量
　　nohup bin/jstorm supervisor &

五、webUI的部署
　　必须使用tomcat 7.x版本， 注意不要忘记拷贝 ~/.jstorm/storm.yaml
　　Web UI 能够和Nimbus不在同一个节点

　　

mkdir ~/.jstorm cp -f $JSTORM_HOME/conf/storm.yaml ~/.jstorm cd /usr/local/tomcat cd webapps cp $JSTORM_HOME/jstorm-ui-2.4.0.war ./            
        mv ROOT ROOT.old ln -s jstorm-ui-2.4.0 ROOT cd ../bin ./startup.sh

访问http://192.168.46.160:8080/ 便可查看到jstorm的界面

 

六、WebUI 增长新集群
　　注意： web ui使用的版本必须和集群中JStorm最高的版本一致

　　在运行Web UI的机器上， 修改~/.jstorm/storm.yaml
　　把默认的ui.clusters注释给去掉， 补充

# UI MultiCluster # Following is an example of multicluster UI configuration ui.clusters: - { name: "jstorm.share", zkRoot: "/jstorm", zkServers: [ "zk-datanode-01", "zk-datanode-02", "zk-datanode-03"], zkPort: 2181, } - { name: "jstorm.bu1", zkRoot: "/jstorm.dw", zkServers: [ "10.125.100.101", "10.125.100.101", "10.125.100.101"], zkPort: 2181, } 解释一下 - { name: "jstorm.bu1",    --- 这个集群的名字， 每一个集群的名字必须不同 zkRoot: "/jstorm.dw",  --- 这个集群 zk的根节点，能够参考$JSTORM_HOME/con/storm.yaml 中“storm.zookeeper.root” 字段 zkServers: [ "10.125.100.101", "10.125.100.101", "10.125.100.101"],   -- zk 机器列表 zkPort: 2181,                                                  -- zk 的客户端端口 }

 

提交jstorm的任务
　　jstorm jar /usr/local/jstormpack/JstormTest-0.0.1-SNAPSHOT.jar com.dinpay.bdp.rcp.mywordcount.WordCountTopologMain WordCountTopolog

中止jstorm的任务
　　jstorm kill KTTAndNormalTopology

nimbus.childopts: "-Xms1g -Xmx1g -Xmn512m -XX:PermSize=256m -XX:SurvivorRatio=4 -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSFullGCsBeforeCompaction=5 -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=100M -XX:+UseCMSCompactAtFullCollection -XX:CMSMaxAbortablePrecleanTime=5000" worker.childopts: "-Xms512m -Xmx512m -Xmn256m -XX:SurvivorRatio=2 -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=65" ui.childopts: "-Xms768m -Xmx768m -Xmn256m -XX:PermSize=96m -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSFullGCsBeforeCompaction=5 -XX:+HeapDumpOnOutOfMemoryError -XX:+UseCMSCompactAtFullCollection -XX:CMSMaxAbortablePrecleanTime=5000" supervisor.childopts: "-Xms2g -Xmx2g -Xmn1g -XX:PermSize=256m -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSFullGCsBeforeCompaction=5 -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=100M -XX:+UseCMSCompactAtFullCollection -XX:CMSMaxAbortablePrecleanTime=5000"