Home > other >  Using Java to upload files to the HDFS, error There are 1 datanode (s) running the and 1 node (s) ar
Using Java to upload files to the HDFS, error There are 1 datanode (s) running the and 1 node (s) ar

Time:04-11

Could only be replicated to 0 home nodes minReplication (=1). There are 1 datanode (s) running the and 1 node (s) are excluded in this operation.



The uploaded file size is 0

HDFS configuration:
XML core - site.


Fs. DefaultFS
HDFS://ubt: 9000 & lt;/value>


Hadoop. TMP. Dir
/home/hadoop/hadoop/tmp
A base for other temporary directories. & lt;/description>



XML HDFS - site.



DFS. Replication
1 & lt;/value>


DFS. The namenode. RPC - address
Ubt: 9000 & lt;/value>


DFS. The namenode. RPC bind - host
0.0.0.0 & lt;/value>


DFS. Federation. Nameservices
Mycluster


DFS. Ha. Namenodes. Mycluster
Nn1 & lt;/value>


DFS. The namenode. RPC - address. Mycluster. Nn1 & lt;/name>
Ubt: 9000 & lt;/value>


DFS. Hosts
Nn1 & lt;/value>




Java code

Public static void main (String [] args) throws FileNotFoundException {
Configuration conf=new Configuration();
The conf. Set (" fs. DefaultFS ", "HDFS://ubt: 9000");
The conf. Set (" DFS. Nameservices ", "mycluster");
The conf. Set (" DFS. Ha. Namenodes. Mycluster ", "nn1");
The conf. Set (" DFS. The namenode. RPC - address. Mycluster. Nn1 ", "ubt: 9000");
The conf. Set (" DFS. Client. Failover. Proxy. Provider. Mycluster ",
"Org. Apache hadoop. HDFS. Server. The namenode. Ha. ConfiguredFailoverProxyProvider");
FileSystem fsys=null;
Try {
Fsys=FileSystem. Get (conf);
} the catch (IOException e) {
The log. The error (" HDFS service access failure!" );
}

//read a local file system file, return to the input stream
OutputStream out=null;
InputStream input=new FileInputStream (" ABC. TXT ");
Try {
//create a file on the HDFS, return to the output stream
Out=fsys. Create (new Path ("/ABC. TXT "), the new Progressable () {
Public void progress () {
System. The out. Print (". ");
}
});
The contents of the//tools will be in copy to out
IOUtils. CopyBytes (input and out, 4096, true);
} the catch (IllegalArgumentException e) {
The log. The error (um participant etMessage (), e);
The info (" parameter error ");
} the catch (IOException e) {
The log. The error (um participant etMessage (), e);
The info (" upload error ");
} the finally {
Try {
Input. The close ();
Out. The close ();
} the catch (IOException e) {
The log. The error (um participant etMessage (), e);
The info (" close the file stream error ");
}
}
}


An error is as follows:
The first section of an error:
20:23:32. 903. [the main] the DEBUG org.. Apache hadoop. Util. Shell - Failed to detect a valid hadoop home directory
Java. IO. IOException: HADOOP_HOME or hadoop. Home. The dir are not set.
At org, apache hadoop. Util. Shell. CheckHadoopHome (326) Shell. Java:
At org, apache hadoop. Util. Shell. & lt; Clinit> (351) Shell. Java:
At org, apache hadoop. Util. StringUtils. & lt; Clinit> (StringUtils. Java: 80)
At org, apache hadoop. Fs. $$Key Cache FileSystem. & lt; Init> (2823) FileSystem. Java:
At org, apache hadoop. Fs. $$Key Cache FileSystem. & lt; Init> (2818) FileSystem. Java:
At org, apache hadoop. Fs. $Cache FileSystem. Get (2684) FileSystem. Java:
At org, apache hadoop. Fs. FileSystem. Get (373) FileSystem. Java:
At org, apache hadoop. Fs. FileSystem. Get (172) FileSystem. Java:
At com. Jb. Filesys. Util. HDFSFileSystem. Main (139) HDFSFileSystem. Java:
20:23:32. 921 ERROR [main] org.. Apache hadoop. Util. Shell - Failed to locate the winutils binary in the hadoop binary path
Java. IO. IOException: Could not locate the executable null \ bin \ winutils exe in the Hadoop binaries.
At org, apache hadoop. Util. Shell. GetQualifiedBinPath (379) Shell. Java:
At org, apache hadoop. Util. Shell. GetWinUtilsPath (394) Shell. Java:
At org, apache hadoop. Util. Shell. & lt; Clinit> (387) Shell. Java:
At org, apache hadoop. Util. StringUtils. & lt; Clinit> (StringUtils. Java: 80)
At org, apache hadoop. Fs. $$Key Cache FileSystem. & lt; Init> (2823) FileSystem. Java:
At org, apache hadoop. Fs. $$Key Cache FileSystem. & lt; Init> (2818) FileSystem. Java:
At org, apache hadoop. Fs. $Cache FileSystem. Get (2684) FileSystem. Java:
At org, apache hadoop. Fs. FileSystem. Get (373) FileSystem. Java:
At org, apache hadoop. Fs. FileSystem. Get (172) FileSystem. Java:
nullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnullnull
  • Related