Sysadmins of the North
Don't forget to share this post!

Recursive scp and symlinks

Recursive scp without following symbolic links. TL;DR: when using recursive scp, symbolic links aren’t preserved and are copied as if they are normal directories. So you have to look for another solution to recursively transfer symlinks over ssh. Here is one: Tar over ssh!

Tar over ssh

Imagine the situation where you have to recursive transfer files and directories from station A to station B. While using recursive scp (scp -r), you notice all symbolic links are transferred as normal directories, multiplying the amount of data. When I was in this situation, and googled for a solution, I found this blog post: recursive scp and symlinks.

Unfortunately, using rsync was not an option for me, because: A) rsync wasn’t available on the system, and B) the broken file system was mounted read only (RO), thus making it impossible to yum install rsync.. 🙁

Luckily I noticed the comment made by “John Mitchell”:

Didn’t have rsync installed so I used tar over ssh

Hé, that sounds like my situation 🙂 He suggests using tar with ssh instead of recursive scp to transfer symlinks:

# On the source server:
# cd /path/to/content
tar cf - * | ssh user@destination '(cd /path/to/content; tar xf - )'

The files are compressed and piped into the ssh stream, and uncompressed to /path/to/content on the destination server.

This may interest you:   Bash: Grep through a large number of files - argument list too long

Use the tar -p parameter if you want to preserve file permissions:

tar cfp - * | ssh user@destination '(cd /path/to/content; tar xfp - )'

The blog posts Using tar and ssh to efficiently copy files preserving permissions on meinit.nl and Howto: Use tar Command Through Network Over SSH Session on nixCraft provide more nifty tar over ssh options.

Excluding hidden files / folders in tar

Protip: you can exclude hidden files and folders in tar by using --exclude=".*" as first parameter. For example

tar --exclude=".*" -czvf downloads.tgz Downloads/
Downloads/
Downloads/ISO

tar -czvf downloads.tgz Downloads/
Downloads/
Downloads/.secret
Downloads/ISO
Downloads/.hidden1

buy me a coffee
Buy Me A Coffee

About the Author Jan Reilink

My name is Jan. I am not a hacker, coder, developer, programmer or guru. I am merely a system administrator, doing my daily thing at Vevida in the Netherlands. With over 15 years of experience, my specialties include Windows Server, IIS, Linux (CentOS, Debian), security, PHP, WordPress, websites & optimization. Want to support me and donate? Use this link: https://paypal.me/jreilink.

follow me on:

Leave a Comment:

Skip to content