java客户端没法上传文件到hdfs

019-07-01 16:45:24,933 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 8020, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 58.211.111.42:63048 Call#3 Retry#0
java.io.IOException: File /a1.txt could only be replicated to 0 nodes instead of minReplication (=1).  There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
    at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1620)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3350)
    at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:678)
    at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.addBlock(AuthorizationProviderProxyClientProtocol.java:213)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:491)
    at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2141)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2137)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1835)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2135)

学些hadoop。遇到这个问题,查找网上好多资料,通常都是说namenode和datanode不一样步致使的,或者防火墙没开50010端口,或者nameNode和datanode没法通讯致使的。java

其实经过命令行都是能够正常操做的,远程调用的时候能够建立目录和文件,可是像文件写内容的时候,就写不进去,报如上错误。node

本地host须要配置好,而后加上下面这句代码apache

configuration = new Configuration();
configuration.set("dfs.client.use.datanode.hostname", "true");

意思大概就是伪分布式hdfs,datanode注册到namenode的ip是本机的127.0.0.1,当远程客户端链接到namenode获得datanode的ip的时候,获得的是127.0.0.1,这天然是链接不上的。这里的意思大概就是强制本地java客户端使用hostname去链接datanode,能够链接成功分布式

防火墙端口50010也是必须打开的,由于数据节点须要使用这个端口ide

参考自连接描述oop

相关文章
相关标签/搜索