TL;DR You can now disable DeepSource’s external status check from being sent to GitLab. To still block the merge request due to DeepSource check failures, you can query the Analysis Run API. We’re adding a new command in our CLI to make this easier.
GitLab’s External Checks
Earlier this month, a user complained that their GitLab pipeline succeeds as soon as DeepSource analysis checks succeed and allows them to merge the merge request (MR), even if their main test pipeline is still pending (and could eventually fail). This renders the entire pipeline useless since you’ll potentially end up merging broken code to your default branch.
It turns out that, unlike the pipeline implementations of platforms like GitHub, Azure DevOps Services, and Bitbucket, GitLab short-circuits the pipeline success if the last pipeline succeeds — regardless of the main pipeline’s pending status.
From GitLab Repository → Settings → Merge requests → Merge checks:
DeepSource performs baseline analysis on MRs, so your external check will be blocked only when you’re introducing new code quality or security issues in the MR. So more often than not, DeepSource checks will pass (very quickly at that, think ~3-5 seconds) and cause GitLab to mark the MR ready to merge. ¯\_(ツ)_/¯
Solution
There’s no way for users to configure the pipeline short-circuit behavior on GitLab, so it poses a challenge for our users to effectively adopt DeepSource’s quality gates if they’re using GitLab. Starting today, our GitLab users can disable the external status checks from being sent to GitLab altogether.
You will find this toggle on the Settings → Quality gates tab if you’re using a GitLab repository with DeepSource.
You’ll still be able to see the results of the status checks as comments on the MR (the second toggle in the screenshot above that we do not recommend disabling).
Wait, how do I block MRs now?
Great question. You can retrieve the status of individual checks from our GraphQL API (docs) and add a blocking step in your GitLab pipeline. To simplify this workflow, we’re also adding a new command in our CLI that will get you the status without explicitly calling the API.
If you’re a DeepSource user on GitLab, we hope this helps. VCS providers have their idiosyncrasies in their pipeline implementation, but hey, who’re we to judge?