1. Install and start ZooKeeper

Why is ZK using 3.4.5, since it supports disk snapshots and periodic deletion of namenode to avoid disk overloading

Configuration:

# The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial synchronization phase can  take initLimit=10 # The number of ticks that can pass between sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just example sakes. dataDir=/zookeepertest/data autopurge.purgeInterval=1 # the port at which the clients will connect clientPort=2181 server.1=test:2888:3888 server.2=test1:2888:3888 server.3=test2:2888:388Copy the code

  • dataDi\

    • All ZK data is written under dataDir

  • autopurge.purgeInterval\

    • Autopurge. PurgeInterval =1 every hour, it cleans up the data in the dataDir, because ZK generates snapshots and binlogs so fast that they fill up the disk in a matter of days. We can effectively avoid this problem by cleaning once an hour. The rule of cleaning is that it keeps the latest 3 files when cleaning. Of course, this 3 can also be configured

  • server.1=test:2888:3888\

    • X number set in myID for server.x

Storm cluster

Install Storm

Download, unzip

2.1 Configuring Storm Configuration file storm.yaml

cd /usr/local/storm/conf
vi storm.yaml
Copy the code

Modify as follows: Add a cluster and set the primary node

Distribute to each node

scp

3. Start Storm

3.1. Start Nimbus

Start Nimbus on the primary node

cd /usr/local/storm
mkdir logs
./bin/storm nimbus >> logs/nimbus.out 2>&1 & 
Copy the code

3.2 start the UI

cd /usr/local/storm
./bin/storm ui >> logs/ui.out 2>&1 &
Copy the code

3.3 start the logviewer

Boot on port 8000

cd /usr/local/storm
./bin/storm logviewer  >> logs/logviewer 2>&1 &
Copy the code

3.4. Start Superviour

Start the Supervisor on each node

cd /usr/local/storm
./bin/storm supervisor >> logs/supervisor .out 2>&1 & 
Copy the code

3.5 Verifying the Browser

http://test:8080
Copy the code

3.6 submit the Topology

CD /usr/local/storm. /bin/storm jar examples/storm-starter/ storm-startertopology-0.94.jar storm.starter.WordCountTopology wordcountCopy the code