๐Ÿ”€ Queue Piece for Aggregation / Throttling / Rate Limiting

Hi Activepiecers!

Iโ€™m excited to introduce the new queue feature. A queue provides a way to process data in a First-In-First-Out (FIFO) manner.

You can create queues to be used across different flows, each identified by a unique name.

Actions:

  1. Pull from queue: Specify the queue name and the number of items to retrieve.
  2. Push to queue: Specify the queue name and provide a list of items to add.
  3. Clear queue: Specify the queue name to delete all its items.

Use Case:

  1. :hourglass_flowing_sand: Aggregate and use later: Collect information from various sources and store it for future use. Instead of handling each piece of data immediately, gather it in a queue for easier batch processing later, such as weekly change log announcements.

  2. :bar_chart:: Throttle / Rate Limit: For instance, if you need to process large Google Sheets, set up two flowsโ€”one to push items into the queue and another scheduled to pull a certain number of items from the queue for processing.

queue

Quick tutorial:

1. Push to Queue
Here you need to specify the name of the queue you want to push in, and the list of items.

If you click on test step it wonโ€™t affect your real queue, but will add to a dummy queue isolated from the real one. Same thing applies to the other actions.

2. Pull from Queue

Here you need to specify the queue name and number of items to pull from this queue. It returns an array of those items. If you click on test step it will pull data from the dummy queue not the real one.

3. Clear Queue

Here you need to specify the name of the queue to clear. If you click on test step it will clear the dummy queue.

4 Likes