Incident report: Elevated 503s when downloading CLI

Between Sep 13, 1313 hours UTC to Sep 13, 1407 hours UTC, clients were experiencing elevated errors when downloading the CLI. The service which hosts the CLI script was affected, and that affected our landing pages as well. Unfortunately, affected users were met with an NGINX 503 page.

A faulty indentation in one of our Kubernetes ingresses was rolled out to production, where our linters in place weren’t able to identify. This eventually led to the ingress controller not being able to detect endpoints for the service hosting the CLI script and threw 503s to users who attempted to download the CLI during that very instance.

We have listed down corrective actions and improvements to existing processes to impede such incidents from happening again in the future. We truly apologize for the inconvenience caused and assure to maintain uptime to its highest standards.

1 Like