Postmortem: Resolving the Internal Error Issue in Flow Runs

Hi everyone,

In the past couple of hours, we encountered an issue where around 4,000 flow runs experienced internal errors. This happened because some servers reached their disk capacity.

The reason for the disk full issue was our frequent deployments, and the images/deployment files were not being deleted from the server.

We’ve cleaned up the servers, and all impacted flows have already been retried. They should now have a normal outcome. Additionally, we’re implementing an automated way to clean up these files after each deployment, and it should be completed today.

We apologize for the inconvenience, and I want to assure you that incidents like these will only contribute to making the Activepieces system even more robust.

Thank you for your understanding! :blush:

6 Likes