Due to the difference in the number of different partitions quantity size, if the unified set the reduce task number, will cause a lot of small files, some partition is there any way can dynamically set the reduce task number? Or whether a parameter can be a default file size if more than the block size is split files (if you can, just set the reduce task a little bit, make its own cutting file)? Someone you know, thanks

CodePudding user response:

No one reply of cold and cheerless