How to setup circleci inside a monorepo

How to setup circleci for a monorepo

You might want to setup a ci for your monorepo. In this article we will see how to use Circleci inside a mono-repository. This post assume that you already have knowledge about monorepo and continuous integration process with Circleci.

We will use the version 2.0 of CircleCI API. You can take a look at the API here. In this version they allow you to trigger a workflow from an API call. Now you are able to have different workflow between each project inside your monorepo. The main idea here is to trigger workflow based on boolean parameters that you enable or not.

Note: In this article we will not focus on what we exactly do inside our ci and best practices inside circleci. We will only focus on the way we are triggering workflows. You can take a look at the final result on github here.

Parameters

In your file .circleci/config.yml you can define parameters. They are just variable that you can access inside this file. When you trigger CircleCI for this project you can change the value of theses parameters. You can find more information about that part of the API here

Copy 
1version: 2.1
2
3parameters:
4  # By default we enable the main workflow
5  trigger:
6    type: boolean
7    default: true
8
9  # Add a boolean parameter on each package.
10  core:
11    type: boolean
12    default: false
13
14  components:
15    type: boolean
16    default: false

• We specify the trigger parameter to true. It will be the boolean that trigger or not the main workflow.
• We’ve also added one parameter per package (core and components) by default we set it to false

Now we’ve defined parameters based on the value of theses variable we can enable or not workflow inside our continuous integration process.

Workflows inside the monorepo

Let’s deep dive into the workflow configuration part of .circleci/config.yml. We have three different workflow:

• ci which is the main workflow. It will install dependencies of packages. Moreover it will have the responsibility to trigger other workflow
• core define the workflow of the core package
• components define the workflow of the components package

Copy 
1workflows:
2  version: 2
3
4  # The main workflow will trigger other workflow based on your changes
5  trigger:
6    when: << pipeline.parameters.trigger >>
7    jobs:
8      - trigger-workflows
9      - new_version:
10          requires:
11            - trigger-workflows
12          filters:
13            branches:
14              only:
15                - master
16
17  # Workflows defined for each package.
18  core:
19    when: << pipeline.parameters.core >>
20    jobs:
21      - lint:
22          package_name: core
23
24  components:
25    when: << pipeline.parameters.components >>
26    jobs:
27      - test:
28          package_name: components
29      - lint:
30          package_name: components

You can notice that we have a condition to trigger each of them with the when condition. The condition need to match with the parameter variable defined previously. This is why we have when: << pipeline.parameters.components >> to trigger the components workflow.

Trigger workflow in a monorepo

In this part we will take a look to see how we trigger different workflow on the monorepo. Here is the job trigger-workflows

Copy 
1jobs:
2  trigger-workflows:
3    executor: node
4    steps:
5      - run:
6          name: Trigger workflows
7          command: node ./circleci.js

trigger-workflows only run a javascript file which takes care of trigger workflows. Here is circleci.js

Copy 
1const fetch = require('node-fetch');
2const { exec } = require('child_process');
3
4const modules = ['core', 'components'];
5
6exec('npx lerna changed --all', (error, stdout) => {
7    modules.forEach(name => {
8        if (!stdout.includes(name)) return;
9
10        const params = {
11            parameters: {
12                [name]: true,
13                trigger: false,
14            },
15            branch: process.env.CIRCLE_BRANCH,
16        };
17
18        fetch(
19            `https://circleci.com/api/v2/project/github/${process.env.CIRCLE_PROJECT_USERNAME}/${process.env.CIRCLE_PROJECT_REPONAME}/pipeline`,
20            {
21                body: JSON.stringify(params),
22                headers: {
23                    Authorization: `Basic ${Buffer.from(
24                        process.env.CIRCLE_TOKEN
25                    ).toString('base64')}`,
26                    'Content-Type': 'application/json',
27                },
28                method: 'POST',
29            }
30        )
31            .then(console.log);
32    });
33});

In this file:

• we execute the command npx lerna changed --all to see packages that have changed
• we take the output of that command to see which package change
• for each package that change we trigger the circleci api and tell of execute the workflow that correspond to the name of our package. we also put trigger to false so the base ci will not be executed.

Note:

• CIRCLE_TOKEN is your personal token that allow you to trigger workflow.
• CIRCLE_PROJECT_USERNAME and CIRCLE_PROJECT_REPONAME are injected by circleci so we do not have to care about it

That’s it ! You see how to create different workflow and trigger them into your mono-repository. I’ve added a simple example on github here. The goal of this example is to focus on the way we trigger workflow for a monorepo and not what we are doing inside the workflows. On’ce you see how to make a workflow for a monorepo you can easily adapt that to your continous integration need. This article was insipred by previous post made by Reuven Harrison and Dumitru Deveatii. You can take a look at theses posts that talk about the same subject they are a good resources.

Last note: English is not my native language so if you find mistake inside this article feel free to reach me.