TL;DR: when using recursive scp, symbolic links aren't preserved and are copied as if they are normal directories. So you have to look for another solution to recursively transfer symlinks over ssh. Use tar over ssh!
scp -r follows symbolic links and copies their targets as normal files and directories, multiplying the amount of data. Without rsync, pipe a tar stream through ssh instead: tar cf - * | ssh user@destination '(cd /path/to/content; tar xf - )' keeps symlinks intact.
Tar over ssh
Imagine the situation where you have to recursively transfer files and directories from station A to station B. While using recursive scp (scp -r), you notice all symbolic links are transferred as normal directories, multiplying the amount of data. When I was in this situation and googled for a solution, I found this blog post: recursive scp and symlinks.
Unfortunately, using rsync was not an option for me, because:
- rsync wasn't available on the system
- the broken file system was mounted read only (RO), making it impossible to
yum install rsync🙁
Luckily I noticed the comment made by "John Mitchell":
Didn't have rsync installed so I used tar over ssh
Hé, that sounds like my situation 🙂 He suggests using tar with ssh instead of recursive scp to transfer symlinks:
# On the source server:
tar cf - * | ssh user@destination '(cd /path/to/content; tar xf - )'The files are packed into a tar stream on stdout, piped through the ssh connection, and unpacked in /path/to/content on the destination server. Symbolic links are stored in the archive as links, so they arrive as symlinks.
Migrating data on Windows? There, junction points can't be copied with Robocopy either, so you have to recreate them on the destination. Find out where they are first: Quickly view all symbolic links, junction points and hard links in a folder using PowerShell.
Use the tar -p parameter if you want to preserve file permissions:
tar cfp - * | ssh user@destination '(cd /path/to/content; tar xfp - )'The blog posts Using tar and ssh to efficiently copy files preserving permissions on meinit.nl (Wayback Machine) and Howto: Use tar Command Through Network Over SSH Session on nixCraft provide more nifty tar over ssh options.
You can exclude hidden files and folders in tar by using --exclude=".*" as the first parameter. For example, with and without the exclude:
tar --exclude=".*" -czvf downloads.tgz Downloads/
Downloads/
Downloads/ISO
tar -czvf downloads.tgz Downloads/
Downloads/
Downloads/.secret
Downloads/ISO
Downloads/.hidden1Want to use ssh, scp and tar from Windows too? Here is how to manually install OpenSSH in Windows Server, and everything else about OpenSSH on Windows: install, keys, tunnels and WSL.