TIL I learned how to increase my productivity through a tool that keeps two directories in sync for me.

One of the pains I have as a developer is syncing code edited in a local filesystem to a remote filesystem in a seamless manner. I use nvim for all my development and it’s a pain to re-set up my nvim IDE in every machine I edit code on (at this point, its probably worth it for me to write some automation for it). Instead of setting up my IDE, I tend to follow this workflow:

1. Write all my code and save it to a local filesystem
2. Test it locally as best as I can
3. Once I’m ready to test it on a remote machine, git commit and git push from the local environment and git pull on the remote environment.
4. Run the code in the remote environment
5. Repeat the above steps till I’m ready to merge to master

One way to get around it is to use VS Code and their Remote SSH tool which, in an abstracted sense, allows you to seamlessly edit and save files that are on a remote filesystem. Now, there is no need for the git commit, git push, and then git pull process. However, at Mighty, I typically test things by running the application end to end and oftentimes a feature or bugfix requires changes on both the client and host. I could just edit the code separately with two separate VS Code workspaces and then git commit, git push, and git pull to sync the client and host codebase at the very end, but that’s still a lot of overhead. Sometimes I forget to edit in the correct VS Code workspaces or sometimes there are shared dependencies between the client and the host which lead to painful merge conflicts. In search of a better solution, I decided to dig into what it takes to sync directories from a local to remote machine.

Rsync

I’ve used rsync in the past, so this was the first place I looked. I rsync a directory to a remote machine and everything just worked! Psych. It took forever to rsync because a lot of build, git, cache, and data files were being synced to the remote host. Stuff that I didn’t even need.

I needed to find a way to sync only what was necessary, which led me to look for exclude or filter options with rsync. Indeed rsync has these flags available. This Stackoverflow post showed me how I could use my .gitignore file to exclude certain files. This worked great!

As I was developing, it became pretty tedious to have to rsync every time I made a code change. I wondered if there was a solution to this. A Stackoverflow post mentioned that inotifywait would work, but it is not available on MacOS which is what I used for local development. An alternative to inotifywait for Linux is fswatch in Mac.

This shell command fswatch . will print the name of any file that has changed in the current directory. Betting that someone has attempted this before, I looked for an fswatch and rsync integration for my own use case and found another Stackoverflow post that told me exactly what I needed to do! fswatch -o . | while read f; do rsync; done basically runs the rsync command every time fswatch -o . creates a new line.

Other Tools

Other tools I may try include lsyncd (https://github.com/lsyncd/lsyncd) or build my own! I’ve been wanting to write a project in Rust to learn, and this could be a fun project for me to start out with.