Postmortem: Failure to Store Logs (8:27 AM - 10:00 AM UTC)

Announcements
1

Hello everyone,

We recently experienced technical issues in the Flow Builder. If you attempted to:

  • Create Run
  • View Run
  • Test Step
  • Test Flow

You may have encountered timeouts or infinite loading.

What Happened?

We were using Cloudflare, which experienced issues affecting request deserialization. As a temporary solution, we redirected logs from S3 to another storage source and slowed down the queue to prevent excessive load.

  • Unfortunately, some log files for runs that occurred between 8:27 AM UTC and 10:00 AM UTC were lost, and runs from this period could not be resumed or retried.

Resolution

The issue has now been fully resolved, and logs are once again being stored in S3 as expected.

Prevention

  • We will implement a method to automatically start storing log files for runs to another storage service instead if there is an issue with S3.

Thanks for your patience!

1 Like
2

Thanks for the update!

1 Like
3

Thanks for the update :pray:

So, it is not possible to know what information was transmitted from applications through ActivePieces in those runs that failed? This is actually quite a serious issue, as I have no way to know what information I am missing :see_no_evil:

Love

1 Like
4

We apologize; if it happens during this time, there is no way to retrieve it, as it has been lost. :cry:

5

Now seems task are starting without correct trigger condition.
I have several flow to be start with rss feed new item and they are starting evenif there are no new items on feed

Please check

2 Likes
6

Hi @Tommaso_Florio

We’ve fixed the issue now—could you please check again? Sorry for the trouble, and thanks for your patience!

1 Like
7

No problem from my side, it was just a couple of small runs. Happy you catch the bug and if how you are handling this :green_heart: