how to access Hadoop -D argument-CodePudding

This is the command:

hadoop jar PopularityLeague.jar PopularityLeague -D league=dataset/league.txt

How can I access the 'league=dataset/league.txt' inside of the Hadoop Java?

I searched this problem, and some one just says I might use configuration to do it. I don't know how exactly I can do it.

CodePudding user response：

Note: I am only answering how you can use -D option To change replication factor below command will be used while storing data to HDFS using -D.

hdfs dfs -Ddfs.replication=1 -put /data/dpq/dummy/temp/GDGA111_1_20180918111340752.csv /data/dpq/temp

Can you try something around similar lines ?

CodePudding user response：

I need to use "dataset/league.txt" inside of java file

If you look at some example mapreduce programs, they might have new Path(args[0]) inside the main method

That command invocation would look like this, not using system properties

hadoop jar PopularityLeague.jar PopularityLeague dataset/league.txt