Hello everyone,
Activepieces Cloud is experienced service interruptions. The slowness of website began on Tue, 23 Apr at 6:00 am PT and lasted until 9:00 AM.
There was a 15-30 minute downtime between 8:00 - 8:30 AM PT, while we are trying to rollback.
The cause of this issue was a sudden increase in database usage triggered by a specific TEST account. We had granted this account a special role, and each time the automated end-to-end tests ran, it put a strain on the database.
This role had been assigned quite some time ago, and it didn’t pose any problems until we introduced a new feature. This new feature led to a significant slowdown in database queries, ultimately exhausting the connection pool.
Impact:
- Delay in Execution
- Some webhooks and triggers, including instant triggers based on webhooks, might have been dropped when the servers went down during the downtime, during the 15 minutes downtime.
We sincerely apologize for any inconvenience caused.
Mo and the Team.
Thank you,
Mo.