Advertisement

安装 nc、Flume 以及测试 Flume

阅读量:

安装Flume

解压文件

复制代码
    tar -zvxf apache-flume-1.9.0-bin.tar.gz -C ../pkg/

    
    
    sh
在这里插入图片描述

修改文件名

复制代码
     mv apache-flume-1.9.0-bin/ flume-1.9.0

    
    
    sh
在这里插入图片描述

配置环境变量

复制代码
    sudo vi /etc/profile.d/env.sh

    
    
    sh

添加如下配置

复制代码
    # Flume

    export FLUME_HOME=/opt/pkg/flume-1.9.0
    export PATH=$FLUME_HOME/bin:$PATH
    
    
    sh
在这里插入图片描述

执行配置文件使其生效

复制代码
    source /etc/profile.d/env.sh
    
    
    sh
在这里插入图片描述

配置 Flume

复制代码
    mv flume-env.sh.template flume-env.sh

    
    vi flume-env.sh
    
    
    sh
在这里插入图片描述

将Hadoop的包软连接到flume

复制代码
    ln -s /opt/pkg/hadoop-3.1.4/share/hadoop/common/hadoop-common-3.1.4.jar /opt/pkg/flume-1.9.0/lib/

    
    ln -s /opt/pkg/hadoop-3.1.4/share/hadoop/common/lib/commons-configuration2-2.1.1.jar /opt/pkg/flume-1.9.0/lib/
    
    ln -s /opt/pkg/hadoop-3.1.4/share/hadoop/common/lib/hadoop-auth-3.1.4.jar /opt/pkg/flume-1.9.0/lib/
    
    ln -s /opt/pkg/hadoop-3.1.4/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar /opt/pkg/flume-1.9.0/lib/
    
    ln -s /opt/pkg/hadoop-3.1.4/share/hadoop/common/lib/commons-io-2.5.jar /opt/pkg/flume-1.9.0/lib/
    
    ln -s /opt/pkg/hadoop-3.1.4/share/hadoop/hdfs/hadoop-hdfs-3.1.4.jar /opt/pkg/flume-1.9.0/lib/
    
    
    sh
![](https://ad.itadn.com/c/weblog/blog-img/images/2025-08-17/rqi8QTYsZMCJ9E1DBeKthXmNGUwP.png)
在这里插入图片描述

安装 netcat

复制代码
    sudo yum install nmap-ncat.x86_64
    
    
    sh
在这里插入图片描述

测试 Flume

创建 netcat_flume_console.conf

复制代码
    vi /opt/pkg/flume-1.9.0/conf/netcat_flume_console.comf

    
    
    sh
复制代码
    a1.sources = r1

    a1.sinks = k1
    a1.channels = c1
    
    a1.sources.r1.type = netcat
    a1.sources.r1.bind = localhost
    a1.sources.r1.port = 1999
    
    a1.sinks.k1.type =logger
    
    a1.channels.c1.type = memory
    a1.channels.c1.capacity = 1000
    a1.channels.c1.transactionCapacity = 100
    
    a1.sources.r1.channels = c1
    a1.sinks.k1.channel = c1
![](https://ad.itadn.com/c/weblog/blog-img/images/2025-08-17/Z0YMN8VhHDtg4q6TcGOfXQnFWkr2.png)

启动 flume

复制代码
    flume-ng agent \

    --conf conf \
    --conf-file $FLUME_HOME/conf/netcat_flume_console.conf \
    --name a1 \
    -Dflume.root.logger=INFO,console
    
    
    sh
在这里插入图片描述

全部评论 (0)

还没有任何评论哟~