[Resolved] Polling Job being Trigger & Webhook Trigger (10:17 AM PDT and 10:44 AM PDT)

Announcements
1

Hi Everyone,

We’re resetting everyone’s tasks to zero this month due to today’s outages.

Apologies for the issues today. We’ve been pushing fixes to mitigate problems, but this led to a cascading failure.

The outage started this morning when our cloud provider, which we use for storing logs, went down:
:link: Incident details

During this outage, a new version was released with a mistake in the function that determines trigger payloads. This caused all payloads to be incorrectly wrapped inside an array:
:link: More details

Impact:

Flows that ran between 10:17 AM PDT and 10:44 AM PDT were affected as follows:

  • Polling triggers ran at least three times during the outage, each time with an empty [] payload.
  • Webhook triggers produced incorrect outputs, with all payloads being wrapped inside [ ], preventing flows from reading the data properly.
  • If you created a flow during this time, please retest the trigger and remap the data.

Actions Taken

Reset everyone tasks to Zero]
Update our E2E test, we do have a test where it sign in into activepieces using browser and build a flow then test it, we are going to make it test the webhook trigger and it’s data mapping (tracked Modify the Playwright E2E testing to confirm data mapping · Issue #6814 · activepieces/activepieces · GitHub)

I will update this thread soon once the actions are taken.

We appreciate your patience and apologize for the inconvenience! :rocket:

2 Likes
2

We are done resetting all tasks for everyone in this month to zero

We are working on the rest of items.

1 Like
3

Thank you so much for the teams hard work on resolving this!

Resetting usage is a wonderful gesture. :smile:

Hope you all can get some well deserved rest in this weekend!!

1 Like
4

Thank you very much for the transparency, that’s how you deal with things like this :pray: Great that you catch the bug on the update, what steps can you take to make sure this doesn’t happen in the future?

Love :green_heart: