War story: sources, sinks, archives, and SSH sessions

Sometimes the right way to think about programming problems is as a sequence of transformations between sources and sinks. I present some Go code for streaming a tar archive from a remote host to the local filesystem. The standard library has everything we need and we are just going to put it together. Copy and adapt to your own needs as necessary.

First we need to get some boilerplate out of the way

We are going to mess with files and buffers through SSH and those are the bare minimum set of packages we will need.

Next we need some constants and variables. Some of the constants won’t be constants in an actual production environment but for demonstration and local testing purposes they’re good enough

With the basics out of the way we now have some boilerplate for setting up the SSH connection with public/private key authentication

I have panics but in a production environment you’d probably log the errors and then return the error upstream to the caller to deal with.

Now onto the main method where we set up the SSH session and connect the standard output stream to a local file and then execute the tar command

There you have it. An example of how to pipe a tar archive from a remote host to a local file with Go. Copy and adapt to your own needs as necessary.