Degraded Performance

Incident Report for Pandium

Resolved

Scheduled run runs have caught up, and we have not experienced any more issues with control plane.

Posted Jul 14, 2024 - 17:42 EDT

Monitoring

A fix has been implemented. We are now monitoring recovery.

Posted Jul 14, 2024 - 13:37 EDT

Update

The underlying issue has been identified. Ectd cluster of the control plane is having a fit. Working on mitigation.

Posted Jul 14, 2024 - 12:35 EDT

Identified

Issue is with the control plane of underlying hosting provider. We are working with them on mitigation.

Posted Jul 14, 2024 - 11:53 EDT

Investigating

Underlying Control Plane under high load. Investigating. Run and API are running slow.

Posted Jul 14, 2024 - 03:06 EDT

This incident affected: API and Runs.