CI System Comparisons

Gitlab CI (MIT)

Project homepage

We have an instance with:

• A runner with an Custom Executor for running jobs with QEMU
• Runners with docker executor (jobs run inside docker containers)
• Some p≡p projects mirrored into it

Pros

• Works well out of the box
• Extensive documentation, and community support
• Gitlab Runner works on MacOS, Linux, Windows, BSD(?)
• Custom Executor allows for flexibility to target additional platforms and architectures via virtualization
• Fairly simple

Cons

• Cannot decouple from full Gitlab forge
• Git is the only VCS natively supported
• “Open Core” model. “Community Edition” is MIT licensed, and maintained in a seperate repo from proprietary “Enterprise Editions”. They sometimes port features from EE to CE.
• Moderate-to-large resource consumption

Buildbot (GPLv2)

Pros

• Extensive documentation, and community support
• Focused on builds. Not full of other features
• Moderate resource consumption
• Very extensible and customizable

Cons

• Not a CI system out of the box, but gives you the tools to create a custom-tailored CI system
• Runs builds in a persistent environment
• Fairly complex
• No full integration with code forges

Sourcehut Builds (AGPLv3)

Project homepage

Pros

• Works well out of the box
• QEMU-based, so possible to target a large variety platforms and architectures
• Modular architecture allows for running independently, or with other Sourchut modules to create a full-fledge forge.
• Native support for Mercurial
• Allows job owners to ssh into failed job environment to debug and access logs
• Moderately simple
• Moderate resource consumption
• Helpful community of hackers

Cons

• New (2~ years)
• Docs are lacking a bit, and small community
• QEMU is the only option for job execution
• Only supports Linux as the host/hypervisor