Degraded database performance

Started at

Dashboard & API

Resolved

Our database has recovered and runs are now processing normally again.

Identified

A very large spike in runs caused degraded the database to have high CPU. This made the dashboard very slow to load and reduced the speed that runs were being processed.

We have fixed the issue but it will take a couple of hours to get back to normal run processing times.