Blog Projects
Programming►Helm108 Server

Deploying My Projects Using Git

One of the main requirements of my server was that I would be able to deploy my projects with a git push, the same way that you can with Heroku, Dokku and similar. After a lot of experimenting this was what I ended up with:

  • When the deploy image is built a bare git repo is created for each project in /var/repos.
  • Each repo uses git hooks to install project dependencies, build the site and launch it with pm2.
  • A script is built that pulls each repo into /var/www.
  • Terraform populates various .env files and then runs the script as part of the server deployment.

In the following code samples $repoPath is /var/repos and $sitePath is /var/www.

Initializing a Bare Repository

A bare git repository differs from a regular git repository in that it does not contain the actual files of your project.

1
2
3
4
5
6
exec { "initialize_${title}_repo":
  command => 'git init --bare',
  path    => '/usr/bin',
  cwd     => "${repoPath}/${repoName}",
  user    => 'helm108',
}

Note the git init --bare.

I’ve lost the source that I based this decision on, but the idea was to use a bare repo because it stops the repo from slowly using up harddrive space by tracking the entire git repo’s history unnecessarily.

I then add the project’s origin to the newly-created repo so that it has somewhere to pull from:

1
2
3
4
5
6
7
exec { "add_${title}_remote":
  command => "git remote add origin ${remoteUrl}",
  path    => '/usr/bin',
  cwd     => "${repoPath}/${repoName}",
  user    => 'helm108',
  require => Exec["initialize_${title}_repo"]
}

And finally I copy in my post-receive and post-checkout hooks from .erb templates:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# Copy post-commit hook into repo and make it executable.
file { "${repoPath}/${repoName}/hooks/post-receive":
  require => Exec["initialize_${title}_repo"],
  owner => 'helm108',
  ensure  => file,
  content => template('githook/post-receive.erb'),
  mode => 'ug+x',
}

# Copy post-commit hook into repo and make it executable.
file { "${repoPath}/${repoName}/hooks/post-checkout":
  require => Exec["initialize_${title}_repo"],
  owner => 'helm108',
  ensure  => file,
  content => template('githook/post-checkout.erb'),
  mode => 'ug+x',
}

Performing the Initial Pull on Deploy

For each repo that I add to my site, I add this line to a bash file that gets run on deploy (split into two lines here for legibility):

1
2
cd $repoPath/$title && git fetch origin &&
git --work-tree=${sitePath}/${title} --git-dir=${repoPath}/${title} checkout -f master

This script cd’s to the repo in /var/repos, fetches the origin, and checks out the master branch to the same folder in /var/www.

Speculation: In writing this post I thought to myself ‘does giving a bare repo a working path defeat the point of using a bare repo?’. It’s something I need to verify. I don’t think it does, but if it does actually result in the working file history being tracked like a regular repo than it seems like there’s no way around it anyway.

The final checkout at the end of that line triggers the post-checkout githook on the repo, which contains the following:

1
2
3
4
5
6
7
8
#!/bin/bash
source /etc/profile.d/helm108envvars.sh
cd /var/www/<%= @title %>
echo "Building <%= @title %>."
npm run dist
pm2 restart <%= @title %>
pm2 start server.js --name "<%= @title %>"
pm2 save

The first thing the script does is sources a file that contains some required environment variables. The githook runs in its own bash instance which means that it has no environment variables available to it.

It then cd’s to the project’s directory and runs npm run dist. Each project that goes on Helm108 must have a dist command in its packages.json, and it must install the project’s dependencies and then run a production build.

Next the script starts the project using pm2. The reason that it calls restart and then start is because I didn’t know about the pm2 ecosystem file; this is a configuration file that describes the running state of the project, and running pm2 ecosystem ensures that the project is running according to this file.

This is important because of the post-receive hook. When I push changes to helm108.com it triggers a post-receive hook:

1
2
3
4
#!/bin/bash
echo "Push received."
source /etc/profile.d/helm108envvars.sh
git --work-tree=/var/www/<%= @title %> --git-dir=/var/repos/<%= @title %> checkout -f

As you can see, this ends with checking out the repo which triggers the above post-checkout hook which means that the post-checkout hook has to handle initializing a new project and restarting that project once changes have been received. Running pm2 start server on an already-running project causes pm2 to go “it’s already running” and not do anything, but calling restart on a project that isn’t running yet doesn’t do anything bad, so the final messy solution is:

  • When a checkout happens:
    • Build the project
    • Restart the project
      • If this is a new project, pm2 just complains that there’s no project with that name
      • If this is an existing project, the project restarts and runs with the newly-built code
    • Start the project
      • If this is a new project, pm2 launches the project
      • If this is an existing project, pm2 just complains that this project is already running
    • Save the list of running projects

So yeah, it works

I could have set pm2 to use its watch function and then it would automatically restart after the build command was run, and I should have read the documentation more thoroughly and used the ecosystem file, but I didn’t so here I am! Learn from my mistakes. Please.