Google Sheet Really Crazy Bug

I am using Google Sheet automation on the cloud version.

This is the automation URL:
https://cloud.activepieces.com/projects/vDjIpW9Y14l1tXPosd9uk/flows/ot7lvFP4kPzDNqmW59Upb

Today, I went up in the morning to find out that automation run 700 with people who already ran on the automation.

Instead of only taking new rows, it seems to take them all.
The damage this created might be really big.

What is going on here?

Lately, I keep facing bugs that cause a lot of damage and make me think twice if I want to keep using you.

@FOXKILL_FOXKILL, I assume the flow has a ‘New Row Added’ trigger. Could it be that existing rows were deleted and added again?

cc @abuaboud

When I first started using Activepieces I noticed and repeatedly reported very similar fragility to what’s described here. See for example: ⚡ Updated Row Trigger & Insert Multiple Rows Action for Google Sheets - #5 by HoldYourWaffle, Discord thread #1 and Discord thread #2. The Discord threads describe an “identity column(s)” mechanism that can solve this fragility.

I’m in a similar position. I believe in this project and would love to deploy it more widely, but this kind of fragility and unreliability is a wider problem with this otherwise super high potential product.

Hi, nothing changed from my side.

It happened in about 4 am on my country and i was luky to be on to fix the damage so no new rows as well. So no row were changed or deleted.

Just like that this crazy mess happened.

@abuaboud can you look at this flow logs?

Hi @FOXKILL_FOXKILL

I am taking look, did you add new column or delete any column?

What i see the there is one column different (Column T), but not sure where it’s coming from

I believe I understand the issue now. I noticed that the version being used in Google Sheets is 0.7.7 (around 10 months old). At that time, it was polling for new rows every 5 minutes, but this approach was fragile for many use cases (as pointed out by @HoldYourWaffle back then). For example, adding or removing a row could disrupt the flow.

To resolve this, click on the three dots on the trigger, and replace it with Google Sheets again. This will upgrade it to version > 0.10.0, which works in a different way.

We usually auto upgrade the piece when there is minor changes, but since this was fundamental change we can’t auto upgrade older version.

Thanks,
Mo.

1 Like

I did change things now; I was able to fix that by adding a new column that updates every time the automation runs and marks it as complete.

Now I do a filter check to see if the automation ran in the past, so that way it won’t run twice again.

But as far as I know, when the problem happened, I haven’t changed anything on my side.

@FOXKILL_FOXKILL I would also suggest upgrade Google Piece too

I think you should find a way to notify users that they need to update the element on the automation or allow them to see if there is an update.

At the moment there is no way to know if there is update and most time it doeosn’t update by itself.

It’s a big problem when the main goal for the automation is to be stable and in some cases it doesn’t give me the options of the new updates (like in smtp that you added the html option and i don’t have it in most emails).

How did you implement this? So far I haven’t been able to come up with a solution that isn’t vulnerable to race conditions.

I agree, this is another big reason why I can’t deploy Activepieces more widely yet. The problem is twofold:

  • We really need a system for migrating piece configurations to newer versions, in the vast majority of cases that should be possible to do fully automatically.²
  • For the special cases we need a good workflow for managing these updates. For me that’d include:
    • Flow browser: mark flows that contain outdated pieces.
    • Flow editor: mark outdated pieces.
    • Piece editor: a changelog excerpt/summary, link to relevant documentation if applicable, link to full changelog and an “Update piece” button. Note that pieces can be multiple versions behind.
    • An overview of outdated pieces across all flows.
      • Grouped by piece type + current version, since you’ll often have multiple instances of a migration path across flows.
      • Relevant changelog information, similar to the flow editor.
      • Bulk operation for force-upgrading piece instances, since not every breaking change requires explicit intervention in every situation. (Like the aforementioned Google Sheets trigger upgrade.) An admin can evaluate this with the changelog.
    • (Email) notifications when there’s new updates for pieces in production.
      • Preferably summarized per a configurable interval.

Edit: it’s worth noting that a “breaking change” isn’t necessarily breaking for every instance of that piece. For example, removing an option that a user didn’t use shouldn’t require manual intervention.
Semver therefore isn’t a good indicator, determining that an upgrade requires manual intervention should be the responsibility of the migration script (as in, only require manual intervention if there’s no way to migrate that piece instance’s config such that it has identical behavior).

Edit 2 electric boogaloo: probably not a good idea to upgrade pieces unattended, even if it’s a fully transparent migration. Bugs are inevitable, I’d prefer pressing a couple buttons once a week to flows (silently) breaking in production :stuck_out_tongue:

I just added another column with the title of “finished automation” and if i do automation on new line i put there 1.

When i automate the line i now have filter that check if the value of the column is 1 and if it’s 1 i won’t run the automations on that line.