【Flume】 flume 负载均衡环境的搭建 load_balance
flume的负载均衡即每次按照一定的算法选择sink输出到指定地方,如果在文件输出量很大的情况下,负载均衡还是很有必要的,通过多个通道输出缓解输出压力
flume内置的负载均衡的算法默认是round robin,轮询算法,按序选择
下面看一下具体实例:
# Name the components on this agent a1.sources = r1 a1.sinks = k1 k2 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = exec a1.sources.r1.channels=c1 a1.sources.r1.command=tail -F /root/dev/biz/logs/bizlogic.log #define sinkgroups a1.sinkgroups=g1 a1.sinkgroups.g1.sinks=k1 k2 a1.sinkgroups.g1.processor.type=load_balance a1.sinkgroups.g1.processor.backoff=true a1.sinkgroups.g1.processor.selector=round_robin #define the sink 1 a1.sinks.k1.type=avro a1.sinks.k1.hostname=192.168.11.179 a1.sinks.k1.port=9876 #define the sink 2 a1.sinks.k2.type=avro a1.sinks.k2.hostname=192.168.11.178 a1.sinks.k2.port=9876 # Use a channel which buffers events in memory a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100 # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1 a1.sinks.k2.channel=c1
以上是flume agent client的配置,server端的配置跟上一篇文章一样http://blog.csdn.net/simonchi/article/details/42494461
同样通过脚本去模拟生成文件内容,触发tail -F,执行两次,发现第一次输出在179,第二次输出在178,说明负载均衡已经生效了!!
很简单!!
郑重声明:本站内容如果来自互联网及其他传播媒体,其版权均属原媒体及文章作者所有。转载目的在于传递更多信息及用于网络分享,并不代表本站赞同其观点和对其真实性负责,也不构成任何其他建议。