Sometimes we need to transfer files to a different machine. Great tools for this
tasks are of course scp and
rsync, but when we want to transfer
folders with many small files (for example when the folder contains
node_modules …) the transferring takes a lot of time, especially with rsync,
as it does some checks for every file.
The solution would be to compress the files at the source and zip it to one
single file, transfer this and unzip it at the destination. For this task we can
use the beloved
tar with all those easy to remember
The naive approach would be to compress the files to a
.tar.gz file, send it
and decompress it on the destination. This leaves ugly temporary files that
nobody need, but start to clutter the drive.
Using pipes to avoid temporary files
We can leverage the power of unix pipes to avoid those temporary files.
If we would like to send the folder
data to our new server
which we created an entry in
~/.ssh/config) we could use the following:
tar czf - data | ssh newserver "cd /tmp && tar -xvzf - "
This compresses the folder, sends it to
newserver and decompresses it to
/tmp/data. From there we can move the folder, or change initially the path,
where we want to