How to use DeepSource coverage on public repositories

I tried setting up coverage on a public repository, but ran into the problem that GitHub action secrets are not shared with PRs, which prevents CI from sending coverage data to deepsource, and this causes the CI coverage check to fail.

I would be happy for the coverage to only be run on the default branch, and not run on PRs. Is that possible?

Hey John,

Currently, we do not support a configuration where the test coverage analyzer will only run on the default branch. The idea behind this is, we want you to catch coverage issues before your code gets merged into your default branch and fix it as early as possible. If you would like to see this supported by DeepSource, please do open a feature request on our roadmap.

Coming to your issue, I see the problem is due to the pull request being opened from a fork. GitHub doesn’t pass secrets in this case, since it is a security vulnerability.

As a workaround, what you can do is use the pull_request_target event. Since this triggers the workflow on the base of the branch which is safe, the workflow will have access to the secret. But, you won’t be able to get accurate alerts on your pull request from forks, since new code will not be analyzed.

Hope this helps!

I submitted two feature requests. Using pull_request_target event is a terrible workaround, as the energy spent doing the CI is completely wasted, and the results will be misleading. Basically DeepSource doesnt support coverage on Open Source software where the Pull Requests are the “open” part of that.

Since 2020, codecov has supported tokenless code coverage submissions.

Some details about how this is achieved can be seen at security - How can codecov verify a upload request without a token? - Stack Overflow

I don’t know if has a similar feature. I see at Setup for azure-pipelines · Issue #1272 · lemurheavy/coveralls-public · GitHub another approach for how to avoid the token being in the workflow. This might also work for DeepSource.

I believe this might work. We could collate the code coverage on the pull_request event and save it as a workflow artifact, and also have a pull_request_target workflow which waits for the same workflow artifact, and then submits it to DeepSource.