🐢 I think I can implement Rsync today. Looking at ssh2-rs, though, I think we can implement file transfer without relying on rsync. It might be easier to implement everything within the code.

🐇 Then you should rename the issue to new ssh.

🐢 Fair. There is also the ssh_rs crate, but it doesn’t have full support for the protocol. Instead, we can have another command, like xvc storage new ssh, that uses libssh2 via the crate mentioned above. It has some limitations with OpenSSH on macOS.

🐇 From the crate’s README, it looks like you can enable the vendored-openssl feature to compile it statically.

🐢 Let’s go ahead then. It’s better to compile it behind a feature flag, though.

🐇 Yup. Rsync can be separate. I think for now you can implement rsync via Exec::cmd and make new ssh a new issue.

🐢 I’ll copy this conversation there.

🐇 Now, let’s start implementing rsync.

🐢 Do we really want to hide it behind a feature flag? It doesn’t bring any extra complexity to generic, for example—just using the commands and returning the errors.

🐇 I think so. If the user doesn’t have rsync on their system, they’ll just get errors. We don’t need to make the implementation optional, but the tests might be.

🐢 OK.

This motivated me to learn Docker and gcloud rather quickly.

I was able to create a new Docker container in a short time. However, as I don’t want to share my Git SSH key publicly, I needed a way to put an SSH private key (~/.ssh/id_rsa) into this new container securely, without leaving any trace.

I tried a few things, but in the end, I decided to do something like:

FROM python:3.5-stretch as intermediate

# add credentials on build
RUN mkdir /root/.ssh/
# To use docker --build-arg, you can uncomment the following two lines and comment out the COPY line below.
# ARG SSH_PRIVATE_KEY
# RUN echo "${SSH_PRIVATE_KEY}" > /root/.ssh/id_rsa
COPY application_sshkey /root/.ssh/id_rsa
RUN chmod 0600 /root/.ssh/id_rsa

# make sure your domain is accepted
RUN touch /root/.ssh/known_hosts
RUN ssh-keyscan bitbucket.org >> /root/.ssh/known_hosts

RUN git clone git@bitbucket.org:username/application /root/application

FROM python:3.5-stretch
COPY --from=intermediate /root/application /root/application

RUN pip3 install -r /root/application/requirements.txt

EXPOSE 9090/tcp

WORKDIR /root/application/

CMD python3 manage.py runserver 0.0.0.0:9090

Here, application_sshkey is a file I created using ssh-keygen and granted read-only access to on Bitbucket.

As you can see, the Dockerfile has two FROM statements. It creates a container to clone the application repository. Then it starts again with a new container and copies the repository to this new container. This way, it is not possible to peek into the private key using the docker history command.

By the way, I included two methods in the Dockerfile because gcloud builds does not accept a --build-arg parameter similar to docker build. I’m sure there are other workarounds for passing secrets to gcloud builds, but instead of digging for them, I found a solution that works for both Docker and Google Cloud Run.

Fortunately, there is a way to use the ~/.ssh/config file to specify different keys for different target URLs.

# Personal account - the default config
Host github.com
   HostName github.com
   User git
   IdentityFile ~/.ssh/id_rsa

# Work Account
Host work.github.com
   HostName github.com
   User git
   IdentityFile ~/.ssh/id_rsa_work

Now you can create another SSH key in ~/.ssh/id_rsa_work and add it to your work account. When you configure a repository, you need to specify git@work.github.com:user/repo.git as the repository address rather than git@github.com:user/repo.git.