I want to chown and chmod on hdfs (recursively) but only for the folder.
In Linux we can do it using find
https://superuser.com/questions/91935/how-to-recursively-chmod-all-directories-except-files
https://gist.github.com/bzerangue/3b210b009e8d934b4a68
find /path/to/base/dir -type d -print0 | xargs -0 chmod 755
When I try the same in HDFS
hdfs dfs -find /path/to/base/dir -type d -print0
I get an error
find: Unexpected argument: -type
Hadoop version : Hadoop 3.1.1.7.1.7.0-551
Note: I actually also tried in Hadoop 2.6.0 and got -find: Unknown command
,
so I think I need to use Hadoop version 3 .
Is there any equivalent in Hadoop?
CodePudding user response:
The below should work for both Hadoop 2 and 3.
hdfs dfs -ls -R /path/to/base/dir/ | grep "^d"
As a bonus, one can extend the regex used in grep beyond simple "directories only" (^d
) to further filter the results.