My side there is A Linux environment, with two large directory, A directory and B directory, for example, there are hundreds of thousands of files, respectively, under the size are more than 100 G,
Now need to copy it to other directories, cannot use shear mv, needs to retain the original directory,
At present is to use the cp - pr/A/C copy, copy time but really difficult to endure; Thought of compression and then copy, but the speed is too slow,
Consult everybody, see if there was a good way to speed up the efficiency of copy, thank you very much,

CodePudding user response:

If you can't use shear, only to copy in this way, both copy and new, estimates are that
Of course, you can open multiple Windows, open multiple processes to copy, such parallel, should be able to slightly reduce the copy of the time

CodePudding user response:

Thank you for your time to communication, this method is not applicable to me, you the Linux system IO should be fixed, dosing process and window parallel, IO, speaking, reading and writing should be can't accelerate,

reference 1st floor kellyseeme response:

if you can't use shear, then only to copy in this way, both copy and new, estimates are that
Of course, you can open multiple Windows, open multiple processes to copy, such parallel, should be able to slightly reduce the time of the copy

CodePudding user response:

Copy files in the directory, I think it takes two aspects:
1, the scanning time consuming a large number of files in the directory
2, and then there is the real copy files time-consuming

CodePudding user response:

I've seen an article with cp when Linux file copy a lot of copy too slow, it is better to use the dd command the whole piece of disk copy,

CodePudding user response:

The directories and directories B divided into directly to the impassability MTD, direct copy the hard disk

CodePudding user response:

Can be used at the time of copy, performance tools and see if it is IO is the bottleneck, or CPU, if is the IO then try to use the dd, if it is CPU, it should be able to process more copies

CodePudding user response:

Should be in the CPU speed bottleneck

CodePudding user response:

Bottleneck is your hard drive

CodePudding user response:

Tar to memory, memory decompression

 tar cf - | (CD target directory; The tar xf -) 

CodePudding user response:

Rsync - a/a/C

CodePudding user response:

Don't ink, old honest practical cp command, and no other good way, so many files and folders you need to create, is necessarily slow, suggest to spend several hundred yuan change the senior point of CPU, memory,