Profile

@sgametrio

15 August 2020

Multi-repository deployments with Slack Modals and Gitlab CI

This article assumes the developer is working on a web project where Gitlab CI/CD is used as a way of testing and deploying changes.

Single-repository development workflow

Most common git flows treats master as an always-deployable branch which means that what is in there should be stable enough to be deployed on a staging server. Following the workflow, when you want to introduce a new feature/fix, you need to:

  1. Branch out master in a feature-branch
  2. Implement the desired feature
  3. Pass CI tests on feature-branch
  4. Merge feature-branch back into master

Master branch with feature branches merged in

While running tests before merging avoids most of the errors, sometimes you may introduce bugs into master, which can slow down the development of other people in the team who have just started working on a new feature branching out, unluckily, your bug. Now they have to figure out what's causing the bug without knowing that is not their fault.

This is why the concept of review environments has been introduced: you may want to deploy your feature-branch in a pre-staging server and see your changes live on a website to increase the probability of catching bugs before merging into master.

Ideally review environments are deployed after each significant changes in feature-branch and deployed on a custom subdomain in order to help the manual testing and review of other people too. They don't need to clone and build your branch, they just need to go on http://feature-branch.awesometeam.com, which should reflect, at its best, the production configuration and see if something is broken.

Review environment usual workflow

Multi-repository workflow

Most of the web projects though, requires both back-end and front-end modification to introduce a new feature, which live in separated repositories. Since the repository-scoped nature of Gitlab CI/CD, it's difficult to apply the same logic to review environments: how do I review the changes I have in these two repositories, together? The common situation is that you will have one branch per repo (i.e. feature-branch-back-end and feature-branch-front-end) and you want these branches deployed, talking to each other. In order to achieve the desired environment we need to:

  • Trigger one review deployment per repo
  • Pass custom variables to Gitlab CD so that we can customize the URL configuration
  • (optional) In my use case, we wanted to deploy a tenant-customized web app, which means you have to pass an additional custom variable which define the tenant you are building for.

Passing variables to the CI

Passing custom variables to Gitlab CI/CD is easy and you have different options for it:

  1. If you have a manual job in your pipeline, you can specify custom variables through the Gitlab web UI.
  2. While pushing commits: git push origin branch -o ci.variable="CUSTOM_VARIABLE1=one" -o ci.variable="CUSTOM_VARIABLE2=two" ...
  3. Using the Gitlab API you can trigger pipelines with additional variables

Let's discuss of the downsides of each.

  1. Can't be automated, requires manual action, therefore it is discouraged.
  2. Interesting but it requires you to have some commits to push every time. If you already pushed and you want to deploy, you have to commit-and-push once more.
  3. It is the way to go. Repeatable and easily automatable.

How to use the Gitlab Trigger API

As I briefly introduced, using the Gitlab Trigger API lets you easily trigger new pipelines. Let's see how to do it from a NodeJS script:

let formData = new FormData()
formData.append("ref", ref)
formData.append("token", process.env.GITLAB_TRIGGER_TOKEN)
const response = await fetch(`https://gitlab.com/api/v4/projects/${repo_id}/trigger/pipeline`, {
   method: "POST",
   headers: {
      Authorization: `Bearer ${process.env.GITLAB_API_TOKEN}`
   },
   body: formData
})
const data = await response.json()

Here you can see easily what are the variables and tokens that you need to get in order to call the API. ref is the name of the branch/tag on which we want to run the pipeline. How to retrieve other tokens/values is well documented on the Gitlab Docs so I won't spend much time on it.

The important and useful thing is that when triggering the pipeline we want specific jobs to be executed, therefore we can pass an additional variable which identify that the pipeline has been triggered through the API:

...
formData.append("variables[CP_TRIGGER]", "Triggered from NodeJS.")
...

So that in our .gitlab-ci.yml file we can define a rule to execute a job only if the CP_TRIGGER variable is present:

review:deploy:
   stage: review
   script:
      - echo "Deploying from NodeJS"
      ...
   rules:
      - if: CP_TRIGGER

Given that we want to deploy a full-stack application, living in 2 separate repositories, we need to call the Gitlab Trigger API twice from the same script. Which is doable, but I want to show you how you can integrate Slack into the workflow to have a interactive and user-friendly way to start new deployments.

Easing the interaction with Slack modals

In my use case, I have a multi-tenancy web application, therefore I would like to specify also which customer-ized app I want to deploy.

With Slack modals we can do it easily and in a second. We can build a modal, composed by several dropdowns or input fields, populated based on our needs. Slack modal example screenshot In the above picture there is an example: I decided to automatically list the open Merge Requests, both front and back end, and the list of customer I have. In this case then, I'm deploying a full-stack review environments built on top of the Merge Requests source branches code.

You can request the opening of a custom modal by creating a Slack app and integrating it into your channel. To enrich the dropdowns with custom information you have to build a back-end which call the Gitlab API, retrieving the list of Merge Requests or branches you need.

When pressing the Do it man button the modal send all these options, one common subdomain namespace has to be created and both the project's pipeline triggered with the namespace as a custom parameter!

In order to summarize the whole process, here you are the high-level flow of information (realized with mermaid.js): Flow of information of multi-repo deployments with Gitlab and Slack