Encrypt tar.gz file on create

I have a script which creates a nightly backup of several directories.

It is created using tar -czf, and the destination tar.gz file path is on a mounted network directory. The resulting file is about 1.2Gb large.

The network speed is reasonably fast (copying from the network drive to the local occurs at ~28MB/sec).

Now I want to use public-key encryption to encrypt the tar.gz file before transferring it to the network drive, and would like to know what the best way to do so is.

Should I create the file locally first, encrypt it, and then copy over? Or is there a way to "stream" tar output through an encryption process, and write the results directly to the network drive?

5 Answers

Pack your_dir in a encrypted archive your_archive.tgz.gpg (symmetric encryption):

tar -cz your_dir | gpg -c -o your_archive.tgz.gpg

Unpack it:

gpg -d your_archive.tgz.gpg | tar xz

See the docs of GPG for how to use asymmetric instead of symmetric encryption.

The following process encrypts the file on local disk first, and can then be sent over the network (or stored however needed)

First, generate public and private keys (done only once):

openssl genrsa -out key.pem 2048
openssl rsa -in key.pem -out key-public.pem -outform PEM -pubout

Then, at each backup:

1. Generate long random passphrase, save in file

   • echo -n "Tl4R6dnvWXiDeXr1LtpCNkyLG1" > key.txt

2. encrypt file with passphase

   • openssl enc -aes-256-cbc -pass file:key.txt < UNENCRYPTED_FILE > encrypted.dat

3. encrypt passphrase with public key

   • openssl rsautl -encrypt -pubin -inkey key-public.pem < key.txt > enc.key.txt

Then save encrypted.dat AND enc.key.txt where desired.

To decrypt:

1. Decrypt encrypted passphrase with private key

   • openssl rsautl -decrypt -inkey key.pem < enc.key.txt > key.txt

2. Decrypt file

   • openssl enc -aes-256-cbc -d -pass file:key.txt < encrypted.dat > UNENCRYPTED_FILE

This is a lot longer than Florian's anwser, but I decided to use it so I can better understand the process, and not depend on server-dependent GPG configuration variables, etc. I also couldn't find any useful GPG documentation.

I do this with asymmetric key encryption. That means I have a public key (that I can share with anyone I'd like to send me encrypted packages) which allows me to encrypt the package. I also have a private key (that I do not share) which allows me to decrypt the package.

My commands to encrypt the current working directory: the -e to encrypt, the -r to specify a "recipient" or key to use, the -o to specify the output file.

$ tar -cvz . | gpg -e -r ABCD1234 -o backup.tgz.gpg

And to decrypt to the current working directory:

$ gpg -d backup.tgz.gpg | tar -xz

Or to decrypt to a standard tgz file for later unpacking:

$ gpg -o backup.tgz -d backup.tgz.gpg

Of course, this only works if I have already generated a public-private key pair and installed it with gpg. In my case, I have done so using Digital Ocean's guide at . The ABCD1234 in the encryption command refers to one of the public keys installed on my system. This guide also covers how to share your public key and install others' public keys to send and receive encrypted files.

GnuPG is what I've chosen to used for this process and because you've stated that decryption steps where to opaque (I agree in this usage instance) I've also worked on that side of the problem too. Check the projects' Travis-CI build logs for what features currently work as designed and the travis.yml file for following along with the logs' output. Basically you'll need three scripts from this project; the keygen script, decryption helper script, and named pipe listener script. The keygen script & decryption helper script should be used on the device that will preform decryption and the named pipe listener script should be on the device preforming encryption.

The encrypting named pipe listener script accepts strings, file paths, or directory paths once setup and outputs encrypted results in a predictable fashion.

Below are example commands that could be placed into your nightly backup script for encryption and for directories compression+encryption

echo "some messages text" > /path/to/named.pipe
## Message text is ASCII armor encrypted and appended to specified file when script was stated
echo "${HOME}/Documents" > /path/to/named.pipe
## Directories are compressed with tar and encrypted with output to time stamped file in bulk output directory
echo "${HOME}/.bash_history" > /path/to/named.pipe

For decryption you'll want to check out how the build scripts .travis-ci/script_decrypt.sh & .travis-ci/test_search_script_decrypt.sh for how the appended strings of encrypted data are recovered and how bulk files/directories are restored.

Of course with stuff this experimental it's best to not use primary key pares (kinda why the keygen script was written) and it should not be used on important data until you're confident in how to restore it to readable form.

I used the great solution of Florian Diesch above, but ran into this issue:

gpg: problem with the agent: Inappropriate ioctl for device
gpg: error creating passphrase: Operation cancelled
gpg: symmetric encryption of '[stdin]' failed: Operation cancelled

In addition to work with my Ubuntu 20.04, I needed to set for gpg:

$GPG_TTY=$(tty)
$export GPG_TTY

like described here

Then I could proceed with the solution of Florian! Thanks!

A slightly different method which works also well with linux (Ubuntu 20.04) is described here