hadoop集群 应用例子

例子博客

https://blog.csdn.net/huhui_cs/article/details/9907951

http://dbaplus.cn/news-21-1277-1.html

 

一 搭建好了集群环境以后，首先咱们先跑一下例子。

能够时刻关注http://172.17.0.2:50070/explorer.html信息

在ubuntu1 中建立三个文件

a.txt内容为

this is first file
one
two
three
four

b.txt

this is second file
aa
bb
cc
dd
ee
ff

c.txt

this is third file
11
22
33
44
55
one
two
aa
bb

在hdfs建立input目录，而且将三个文件上传。

root@ubuntu1:/home/software/hadoop# bin/hdfs dfs -mkdir /input
root@ubuntu1:/home/software/hadoop# vim a.txt
root@ubuntu1:/home/software/hadoop# vim b.txt
root@ubuntu1:/home/software/hadoop# vim c.txt
root@ubuntu1:/home/software/hadoop# 
root@ubuntu1:/home/software/hadoop# 
root@ubuntu1:/home/software/hadoop# 
root@ubuntu1:/home/software/hadoop# 
root@ubuntu1:/home/software/hadoop# bin/hdfs dfs -put  /input
LICENSE.txt  README.txt   b.txt        c.txt        etc/         lib/         logs/        share/       
NOTICE.txt   a.txt        bin/         data/        include/     libexec/     sbin/        
root@ubuntu1:/home/software/hadoop# bin/hdfs dfs -put a.txt b.txt c.txt /input
root@ubuntu1:/home/software/hadoop# bin/hdfs dfs -ls /input
Found 3 items
-rw-r--r--   3 root supergroup         38 2018-05-26 10:33 /input/a.txt
-rw-r--r--   3 root supergroup         38 2018-05-26 10:33 /input/b.txt
-rw-r--r--   3 root supergroup         48 2018-05-26 10:33 /input/c.txt

接下来查找三个文件里的单词

root@ubuntu1:/home/software/hadoop# bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.9.1.jar grep /input /output '[a-z]'

***********************************************************************************************************************

注意：这里有可能会卡在INFO mapreduce.Job: Running job: job_1532339328403_0003 ，

在http://172.17.0.2:8088/cluster/apps/ACCEPTED能够看到任务处于ACCEPTED状态，可是就不执行。

此时须要修改/etc/hosts文件

添加下面的内容  前面是ip  后面是主机名

172.17.0.2      ubuntu1
172.17.0.3      ubuntu2
172.17.0.4      ubuntu3

***********************************************************************************************************************

 

查看结果

root@ubuntu1:/home/software/hadoop# bin/hdfs dfs -get /output output
root@ubuntu1:/home/software/hadoop# cat output/*