Dotfiles on the SaltStack

Sep 17, 2017 00:00 · 1013 words · 5 minutes read automation configuration dotfiles orchestration saltstack

From dotfiles to arch.

If you’re a software developer, chances are you have a bunch of configuration files you’ve put some time into. Settings for things like editors, or terminals, or whatever else you use to do your work. And if you’re anything like me, you like to take these files wherever you go to make your job as comfortable as possible. For some people they store their config on a USB, and others use Dropbox. For me, it’s a dotfiles repository on Github.

A dotfile is any file that starts with a . for example .tmux.conf. You usually can’t see them in your file browser or with a simple ls, but be sure that they are there.

Having a dotfiles repo on Github allows me to transport and work on my config on whatever system I need to work on. In the past, this repo has been joshleeb/dotfiles. I was a fairly simple setup all my configs put together. And it was tied into the system with two simple scripts setup and teardown.

The setup script creates symbolic links from the ~/dotfiles~ directory to wherever they need to be. And teardown removes them in the case I was working on a temporary system and didn’t want to make a mess. This meant that I could change the file all in the same location and it would all work out nicely.

Yet this didn’t scale that well. It required me to change the setup and teardown scripts each time I added a new config file. The scripts themselves weren’t really up to par either. I still had to install the programs I wanted to use, and manage any dependencies manually. This usually turned into a process of, wanting to use a particular tool, seeing that it didn’t work, and then fixing it on the fly.

So, a little over two weeks ago I started searching for a new solution.

I considered writing my own tool, which I would call Nos. The aim of the tool would be to speed up my setup on a new system, hence the name. Although writing and maintaining the tool would likely use up more time than I would ever save.

If you have ever had thoughts of building your own package manager. Whether that be for a language, or for dotfiles like Nos would’ve been, take a look at “So you want to write a package manager?”. It’s a good read and will give you a good heads up on what you’re getting yourself into.

Next I had the idea of using an orchestration tool. I have worked with some of these tools before. Specifically Puppet, through from my experiences it seemed much too complicated and a bit of an overkill. Then I came across SaltStack.

SaltStack seemed to provide a good amount of flexibility and power without being too complex. And I also came across a few other repos on Github with people’s dotfiles being managed by it too. So I started porting over my dotfiles to use SaltStack in this new repo joshleeb/arch. Let’s dive in then…

The first thing worth mentioning is the bootstrap script which now replaces the previous setup script, but does a few more things:

Download and install curl and saltstack:

...
# Linux SaltStack bootstrap one-liner.
#   -d Don't enable salt-minion autostart.
#   -P Allow pip based installations.
curl -L https://bootstrap.saltstack.com | $SUDO sh -s -- -P -d
...

Setup grains for SaltStack:

...
# Set the user, home-directory, and state root.
$SUDO salt-call --local --config=./ --state-output=changes grains.setvals \
    "{ \"user\": \"$USERNAME\", \"homedir\": \"$HOMEDIR\", \"stateroot\": \"$DIR/states\" }"
...

Run SaltStack to setup all the states have are defined:

...
if [[ ! $1 ]]; then
    $SUDO salt-call \
        --local \
        --config=./ \
        --state_verbose=True \
        --state-output=mixed \
        --log-level=quiet \
        --retcode-passthrough state.highstate
else
    $SUDO salt-call \
        --local \
        --config=./ \
        --state_verbose=True \
        --state-output=mixed \
        --log-level=quiet \
        --retcode-passthrough state.sls $1
fi

This script also does something else that’s pretty nice. If you run ./bootstrap with no arguments, then it will look at states/base/top.sls to determine what to install. But if you provide it with a state name, such as neovim then it will run only that state and it’s dependencies.

Next up we have the minion file. This is a file used by SaltStack to configure the minion. For this config we only need to specify where the file roots are for the top level base states and debian states.

file_roots:
  base:
    - ./states/base
  debian:
    - ./states/debian

And with that we move onto the fun parts: the states. States are units that define groupings of programs and, in this case, config files to setup and install.

We start with the base states. Each directory under base contains a grouping of stuff I want to install, and all the config files associated with that program/tool. Everything in the base state is system agnostic so that’s why we put config files there.

Most of the base states call out to a system specific state file that does the actual installation. So far I’ve only needed to setup debian states.

If you look at the base/neovim state we can see it calling out to the debian/neovim state in the first two lines are:

include:
    - {{ grains.os_family | lower }}: neovim

And then in debian/neovim we install and update the latest version of Neovim:

neovim-install:
  pkg.latest:
    - name: neovim
    - refresh: True

And that’s pretty much it. Some states are a bit more complicated than others, requiring correct ordering or dependencies. There are others too which require installation by scripts, such as golang and node or with specific PPAs like sublime-text. The SaltStack documentation is pretty good so setting this stuff up was pretty straight forward.

Since taking the time to update my dotfiles I haven’t looked back. While this isn’t, and never will be a perfect solution, it’s much better than what I had before. And there’s a lot of work to do, specifically with setting up Alacrity and Rust. But all in all I’m happy with how this transition went.

Would highly recommend :)