Hadoop搭建_01

时间 2019-12-05

标签 hadoop 搭建栏目 Hadoop 繁體版

原文原文链接

安装vmware

安装linux

配置linux

配置IP

192.168.1.100 masterjava

192.168.1.102 slave2linux

192.168.1.103 slave1sql

配置主机名

[root@localhost ~]# vim /etc/sysconfig/networkshell

NETWORKING=yesvim

NETWORKING_IPV6=nodom

HOSTNAME=masterssh

[root@localhost ~]# hostname masteride

三台主机都配置主机名工具

[root@localhost ~]# vim /etc/sysconfig/networkoop

NETWORKING=yes

NETWORKING_IPV6=no

HOSTNAME=slave1

[root@localhost ~]# hostnameslave1

[root@localhost ~]# vim /etc/sysconfig/network

NETWORKING=yes

NETWORKING_IPV6=no

HOSTNAME=slave2

[root@localhost ~]# hostnameslave2

重启才能生效

Reboot

而后使用远程链接工具，这里使用xshell

关闭防火墙和selinux

[root@master ~]# vim /etc/sysconfig/selinux

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:

# enforcing - SELinux security policy is enforced.

# permissive - SELinux prints warnings instead of enforcing.

# disabled - SELinux is fully disabled.

SELINUX=disabled

# SELINUXTYPE= type of policy in use. Possible values are:

# targeted - Only targeted network daemons are protected.

# strict - Full SELinux protection.

SELINUXTYPE=targeted

[root@master ~]# serviceiptables status

表格：filter

Chain INPUT (policy ACCEPT)

num targetprot opt source destination

Chain FORWARD (policy ACCEPT)

num targetprot opt source destination

Chain OUTPUT (policy ACCEPT)

num targetprot opt source destination

域名解析

使得master和slave1 slave2 链接对方

三台都要作

[root@master ~]# vim /etc/hosts

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1 localhost.localdomainlocalhost

::1 localhost6.localdomain6 localhost6

192.168.7.100 master

192.168.7.101 slave1

192.168.7.102 slave2

配置java

启动ftp

[root@master ~]# /etc/init.d/vsftpd restart

关闭vsftpd： [失败]

为vsftpd启动vsftpd： [肯定]

默认状况下 root不容许使用ftp

[root@mastervsftpd]# pwd

/etc/vsftpd

[root@mastervsftpd]# ls

ftpusersuser_list

将这两个文件中的root注释掉。

而后重启ftp

在三个机器上建立文件夹

[root@master ~]# mkdir installer

[root@master ~]#

上传jdk

E:\开发工具

jdk-6u27-linux-i586-rpm.bin

到installer目录

[root@master installer]# ll

总计 78876

-rw-r--r-- 1 root root 80680219 12-01 12:50 jdk-6u27-linux-i586-rpm.bin

[root@master installer]# chmoda+x jdk-6u27-linux-i586-rpm.bin

[root@master installer]#

[root@master installer]# ./jdk-6u27-linux-i586-rpm.bin

Unpacking...

Checksumming...

Extracting...

UnZipSFX 5.50 of 17 February 2002, by Info-ZIP (Zip-Bugs@lists.wku.edu).

inflating: jdk-6u27-linux-i586.rpm

这时候能够再开一个链接进行远程拷贝

[root@master installer]# scp jdk-6u27-linux-i586-rpm.bin slave1:/root/installer

root@slave1's password:

jdk-6u27-linux-i586-rpm.bin

拷贝完在三个机器上都要安装jdk。

[root@master installer]# java -version

java version "1.6.0_27"

添加用户add

三台机器都作

[root@master ~]# useraddhadoop

[root@master ~]# passwdhadoop

Changing password for user hadoop.

New UNIX password:

BAD PASSWORD: it is based on a dictionary word

Retype new UNIX password:

passwd: all authentication tokens updated successfully.

[root@master ~]#

而且在slave1和slave2使用hadoop用户建立installers目录

[root@slave1 ~]# su - hadoop

[hadoop@slave1 ~]$ mkdir installer

配置ssh等效性

[hadoop@master ~]# ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /root/.ssh/id_rsa.

Your public key has been saved in /root/.ssh/id_rsa.pub.

The key fingerprint is:

5b:d6:30:95:6f:37:b1:d8:e0:ec:b4:cb:94:cc:3f:ccroot@master

[hadoop @master ~]#

一路回车

这一个动做在三台机器上都执行

[hadoop@master ~]$ cd .ssh/

[hadoop@master .ssh]$ ls

id_rsa id_rsa.pub

[hadoop@master .ssh]$

[hadoop@master .ssh]$ cat id_rsa.pub >authorized_keys

[hadoop@master .ssh]$ ls

authorized_keysid_rsa id_rsa.pub

[hadoop@master .ssh]$

将生成的authorized_keys文件拷贝到slave1和slave2

[hadoop@master .ssh]$ scpauthorized_keys slave1:~/.ssh/

The authenticity of host 'slave1 (192.168.1.103)' can't be established.

RSA key fingerprint is 61:e5:be:d1:92:41:b4:22:4e:88:ff:b1:b1:a1:64:bb.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'slave1,192.168.1.103' (RSA) to the list of known hosts.

hadoop@slave1's password:

authorized_keys 100% 395 0.4KB/s 00:00

[hadoop@master .ssh]$

而后进入slave1

[hadoop@slave1 .ssh]$ cat id_rsa.pub >>authorized_keys

[hadoop@slave1 .ssh]$ catauthorized_keys

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAw8taarZ+/ndWV04MqGsnT5cKcYs5LqMmtocWSsIxfUttYpMjwcgktjEPSByb/SFPE3alx0/Te7bjG8nFu2HHV4v++2jNfraqoBjIrO3/ITzHOSGduYmM4xbvBcXCAX5BSawwbpKn8RifPM5M1ZbExFhdZ0njsYSBlq6ZAMV+2F77enfwCI6jB/WhtfClj4QpWuMTQ8O/gqaMbM0OMrIuY84ssoYfDSpl2uUtGBBGY3cyyTDEbQukRH5doapSNPwZQs6lJSVIO7JWLGMfOQbvsqlS0r1nly57I1b7hAMZcGdVWZy2CGclQX3s8a7vjpJ8+iTFtwiAdydFsP+aQ9ldUw== hadoop@master

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAqhiMNhNlBZ1+aC+tU9O8HKTd7lSMmqhi7FcBKue/q/H37hyMp+PqS/BVYStvEhtHzcy+1/SJWKqSV0ut1Qh8zUo42w81KW/g1xCt5fAJLe61/XtC2WyTrwfVQbFVXCPTpAarYJTlgy+ZgarD8Qg4hS642dmXKbSUQf/Mjbxd7PpcAZx1GCVOX3wck+7LIQJuLInlAFIXhyP0rq+I80CX9u40utkgJQd6ZVvsqJdnB+eeXr08w16GEOSY8ER2Vksbw69PGJjjKz1eMFpCUNatlf3bgmLp+JBOnlbgEizc21ogwcnyTXKCP9j3ZHTO2pDxAaHJ2hYJnOjr2+GSALzeOw== hadoop@slave1

[hadoop@slave1 .ssh]$

而后在slave1 传输到slave2

[hadoop@slave1 .ssh]$ scpauthorized_keys slave2:~/.ssh

The authenticity of host 'slave2 (192.168.1.102)' can't be established.

RSA key fingerprint is 61:e5:be:d1:92:41:b4:22:4e:88:ff:b1:b1:a1:64:bb.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'slave2,192.168.1.102' (RSA) to the list of known hosts.

hadoop@slave2's password:

authorized_keys 100% 790 0.8KB/s 00:00

[hadoop@slave1 .ssh]$

到slave2上

[hadoop@slave2 .ssh]$ cat id_rsa.pub >>authorized_keys

[hadoop@slave2 .ssh]$ catauthorized_keys

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzyFZKYRXh1HIm+p//kh/P268u6CHQJ88M+vEcb0fEjpXhNoDaVDceuYhQZxc0E/3dJRd86jaRNWnV+G+IPN00ykV2+UJhE2yjsdMa+Yqwy6XU14H25lMaImJGtxpoXO+3kWKJZ1uGB0E2TU2nS+Epb8EI+6ezZ0ilQhgwpc0kQR/jN6d6hUKKK5yTxKZg4agn4QsOZhyBNQZX7tLofHELR970T5n7to19UejB1j09AVdME+TYf7q3reLYHtVA1NsD7+wQcPB3WOKCRhHU5Uas+Rd3ukIP2/H8h13mJ5NHhq5FzxdVa62OPw9BKZVVO2vXp7SvxJG0MW0Aw8fO+AuRQ== hadoop@slave2

[hadoop@slave2 .ssh]$

而后将这个文件传回slave1和master

[hadoop@slave2 .ssh]$ scpauthorized_keys master:~/.ssh/

The authenticity of host 'master (192.168.1.100)' can't be established.

RSA key fingerprint is 61:e5:be:d1:92:41:b4:22:4e:88:ff:b1:b1:a1:64:bb.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'master,192.168.1.100' (RSA) to the list of known hosts.

hadoop@master's password:

authorized_keys 100% 1185 1.2KB/s 00:00

[hadoop@slave2 .ssh]$

[hadoop@slave2 .ssh]$ scpauthorized_keys slave1:~/.ssh/

The authenticity of host 'slave1 (192.168.1.103)' can't be established.

RSA key fingerprint is 61:e5:be:d1:92:41:b4:22:4e:88:ff:b1:b1:a1:64:bb.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'slave1,192.168.1.103' (RSA) to the list of known hosts.

hadoop@slave1's password:

authorized_keys 100% 1185 1.2KB/s 00:00

[hadoop@slave2 .ssh]$

在三台机器上修改权限

[hadoop@master .ssh]$ chmod 600 authorized_keys

到这里配置完毕，能够直接使用

Ssh slave1 连接不须要提示密码。