5、安装zookeeper
本博客所有内容是原创，如果转载请注明来源
http://blog.csdn.net/myhaspl/
myhaspl@aaayun:~/jzmq-master$cd ..
myhaspl@aaayun:~$wget http://apache.fayea.com/zookeeper/zookeeper-3.3.6/zookeeper-3.3.6.tar.gz
myhaspl@aaayun:~$tar -xzvf zookeeper-3.3.6.tar.gz
$cd zookeeper-3.3.6
yhaspl@aaayun:~/zookeeper-3.3.6$ cp./conf/zoo_sample.cfg ./conf/zoo.cfg
myhaspl@aaayun:~/zookeeper-3.3.6$bin/zkServer.sh start
JMX enabled by default
myhaspl@aaayun:~/zookeeper-3.3.6$ sudobin/zkServer.sh start
JMX enabled by default
Using config:/home/myhaspl/zookeeper-3.3.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
myhaspl@aaayun:~/zookeeper-3.3.6$bin/zkCli.sh -server 127.0.0.1:2181
Connecting to 127.0.0.1:2181
2014-12-17 16:22:17,187 - INFO [main:Environment@97] - Clientenvironment:zookeeper.version=3.3.6-1366786, built on 07/29/2012 06:22 GMT
2014-12-17 16:22:17,207 - INFO [main:Environment@97] - Clientenvironment:host.name=aaayun
2014-12-17 16:22:17,208 - INFO [main:Environment@97] - Clientenvironment:java.version=1.7.0_71
2014-12-17 16:22:17,209 - INFO [main:Environment@97] - Clientenvironment:java.vendor=Oracle Corporation
2014-12-17 16:22:17,210 - INFO [main:Environment@97] - Clientenvironment:java.home=/usr/lib/java/jre
2014-12-17 16:22:17,212 - INFO [main:Environment@97] - Clientenvironment:java.class.path=/home/myhaspl/zookeeper-3.3.6/bin/../build/classes:/home/myhaspl/zookeeper-3.3.6/bin/../build/lib/*.jar:/home/myhaspl/zookeeper-3.3.6/bin/../zookeeper-3.3.6.jar:/home/myhaspl/zookeeper-3.3.6/bin/../lib/log4j-1.2.15.jar:/home/myhaspl/zookeeper-3.3.6/bin/../lib/jline-0.9.94.jar:/home/myhaspl/zookeeper-3.3.6/bin/../src/java/lib/*.jar:/home/myhaspl/zookeeper-3.3.6/bin/../conf:.:/usr/lib/java/lib:/usr/lib/java/jre/lib
2014-12-17 16:22:17,213 - INFO [main:Environment@97] - Clientenvironment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2014-12-17 16:22:17,276 - INFO [main:Environment@97] - Clientenvironment:java.io.tmpdir=/tmp
2014-12-17 16:22:17,277 - INFO [main:Environment@97] - Clientenvironment:java.compiler=<NA>
2014-12-17 16:22:17,278 - INFO [main:Environment@97] - Clientenvironment:os.name=Linux
2014-12-17 16:22:17,278 - INFO [main:Environment@97] - Clientenvironment:os.arch=amd64
2014-12-17 16:22:17,279 - INFO [main:Environment@97] - Clientenvironment:os.version=3.2.0-53-generic
2014-12-17 16:22:17,279 - INFO [main:Environment@97] - Clientenvironment:user.name=myhaspl
2014-12-17 16:22:17,280 - INFO [main:Environment@97] - Clientenvironment:user.home=/home/myhaspl
2014-12-17 16:22:17,281 - INFO [main:Environment@97] - Clientenvironment:user.dir=/home/myhaspl/zookeeper-3.3.6
2014-12-17 16:22:17,286 - INFO [main:ZooKeeper@379] - Initiating clientconnection, connectString=127.0.0.1:2181 sessionTimeout=30000watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@6c0dc01
2014-12-17 16:22:17,408 - INFO [main-SendThread():ClientCnxn$SendThread@1058] - Opening socketconnection to server /127.0.0.1:2181
Welcome to ZooKeeper!
2014-12-17 16:22:17,796 - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@947] - Socketconnection established to localhost/127.0.0.1:2181, initiating session
JLine support is enabled
2014-12-17 16:22:18,204 - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@736] - Sessionestablishment complete on server localhost/127.0.0.1:2181, sessionid =0x14a575462220000, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:Nonepath:null
[zk: 127.0.0.1:2181(CONNECTED) 0] ls /
[zookeeper]
[zk: 127.0.0.1:2181(CONNECTED) 1] quit
6、storm安装
myhaspl@aaayun:~/zookeeper-3.3.6$ cd ..
myhaspl@aaayun:~$ wget http://mirrors.cnnic.cn/apache/storm/apache-storm-0.9.3/apache-storm-0.9.3.tar.gz
myhaspl@aaayun:~$ tar -xzvf apache-storm-0.9.3.tar.gz
启动storm
myhaspl@aaayun:~/apache-storm-0.9.3$ sudo ./bin/storm nimbus&
[1] 25871
myhaspl@aaayun:~/apache-storm-0.9.3$ sudo ./bin/storm supervisor&
[2] 25882
编辑参数配置
myhaspl@aaayun:~/apache-storm-0.9.3$
myhaspl@aaayun:~/apache-storm-0.9.3$ cdconf
myhaspl@aaayun:~/apache-storm-0.9.3/conf$ls
默认配置如下:
# Licensed to the Apache SoftwareFoundation (ASF) under one
# or more contributor licenseagreements. See the NOTICE file
# distributed with this work for additionalinformation
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version2.0 (the
# "License"); you may not usethis file except in compliance
# with the License. You may obtain a copy of the License at
#
#http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law oragreed to in writing, software
# distributed under the License isdistributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANYKIND, either express or implied.
# See the License for the specific languagegoverning permissions and
# limitations under the License.
########### These all have default valuesas shown
########### Additional configuration goesinto storm.yaml
java.library.path:"/usr/local/lib:/opt/local/lib:/usr/lib"
### storm.* configs are generalconfigurations
# the local dir is where jars are kept
storm.local.dir: "storm-local"
storm.zookeeper.servers:
-"localhost"
storm.zookeeper.port: 2181
storm.zookeeper.root: "/storm"
storm.zookeeper.session.timeout: 20000
storm.zookeeper.connection.timeout: 15000
storm.zookeeper.retry.times: 5
storm.zookeeper.retry.interval: 1000
storm.zookeeper.retry.intervalceiling.millis:30000
storm.cluster.mode: "distributed"# can be distributed or local
storm.local.mode.zmq: false
storm.thrift.transport:"backtype.storm.security.auth.SimpleTransportPlugin"
storm.messaging.transport:"backtype.storm.messaging.netty.Context"
### nimbus.* configs are for the master
nimbus.host: "localhost"
nimbus.thrift.port: 6627
nimbus.thrift.max_buffer_size: 1048576
nimbus.childopts: "-Xmx1024m"
nimbus.task.timeout.secs: 30
nimbus.supervisor.timeout.secs: 60
nimbus.monitor.freq.secs: 10
nimbus.cleanup.inbox.freq.secs: 600
nimbus.inbox.jar.expiration.secs: 3600
nimbus.task.launch.secs: 120
nimbus.reassign: true
nimbus.file.copy.expiration.secs: 600
nimbus.topology.validator:"backtype.storm.nimbus.DefaultTopologyValidator"
### ui.* configs are for the master
ui.port: 8080
ui.childopts: "-Xmx768m"
logviewer.port: 8000
logviewer.childopts: "-Xmx128m"
logviewer.appender.name: "A1"
drpc.port: 3772
drpc.worker.threads: 64
drpc.queue.size: 128
drpc.invocations.port: 3773
drpc.request.timeout.secs: 600
drpc.childopts: "-Xmx768m"
transactional.zookeeper.root:"/transactional"
transactional.zookeeper.servers: null
transactional.zookeeper.port: null
### supervisor.* configs are for nodesupervisors
# Define the amount of workers that can berun on this machine. Each worker is assigned a port to use for communication
supervisor.slots.ports:
-6700
-6701
-6702
-6703
supervisor.childopts: "-Xmx256m"
#how long supervisor will wait to ensurethat a worker process is started
supervisor.worker.start.timeout.secs: 120
#how long between heartbeats untilsupervisor considers that worker dead and tries to restart it
supervisor.worker.timeout.secs: 30
#how frequently the supervisor checks onthe status of the processes it's monitoring and restarts if necessary
supervisor.monitor.frequency.secs: 3
#how frequently the supervisor heartbeatsto the cluster state (for nimbus)
supervisor.heartbeat.frequency.secs: 5
supervisor.enable: true
### worker.* configs are for task workers
worker.childopts: "-Xmx768m"
worker.heartbeat.frequency.secs: 1
task.heartbeat.frequency.secs: 3
task.refresh.poll.secs: 10
zmq.threads: 1
zmq.linger.millis: 5000
zmq.hwm: 0
storm.messaging.netty.server_worker_threads:1
storm.messaging.netty.client_worker_threads:1
storm.messaging.netty.buffer_size: 5242880#5MB buffer
storm.messaging.netty.max_retries: 30
storm.messaging.netty.max_wait_ms: 1000
storm.messaging.netty.min_wait_ms: 100
### topology.* configs are for specificexecuting storms
topology.enable.message.timeouts: true
topology.debug: false
topology.optimize: true
topology.workers: 1
topology.acker.executors: null
topology.tasks: null
# maximum amount of time a message has tocomplete before it's considered failed
topology.message.timeout.secs: 30
topology.skip.missing.kryo.registrations:false
topology.max.task.parallelism: null
topology.max.spout.pending: null
topology.state.synchronization.timeout.secs:60
topology.stats.sample.rate: 0.05
topology.builtin.metrics.bucket.size.secs:60
topology.fall.back.on.java.serialization:true
topology.worker.childopts: null
topology.executor.receive.buffer.size: 1024#batched
topology.executor.send.buffer.size: 1024#individual messages
topology.receiver.buffer.size: 8 # settingit too high causes a lot of problems (heartbeat thread gets starved, throughputplummets)
topology.transfer.buffer.size: 1024 #batched
topology.tick.tuple.freq.secs: null
topology.worker.shared.thread.pool.size: 4
topology.disruptor.wait.strategy:"com.lmax.disruptor.BlockingWaitStrategy"
topology.spout.wait.strategy:"backtype.storm.spout.SleepSpoutWaitStrategy"
topology.sleep.spout.wait.strategy.time.ms:1
topology.error.throttle.interval.secs: 10
topology.max.error.report.per.interval: 5
topology.kryo.factory:"backtype.storm.serialization.DefaultKryoFactory"
topology.tuple.serializer:"backtype.storm.serialization.types.ListDelegateSerializer"
topology.trident.batch.emit.interval.millis:500
dev.zookeeper.path:"/tmp/dev-storm-zookeeper"
Storm发行版本解压目录下有一个conf/storm.yaml文件，用于配置Storm。默认配置在defaults.yaml可以查看。conf/storm.yaml中的配置选项将覆盖defaults.yaml中的默认配置。
启动后台ui
myhaspl@aaayun:~/apache-storm-0.9.3$ sudo ./bin/stormui&
[3] 25893