Hi everyone,
I can notice great interest in scraping / monitoring websites and combining that with LLMs.
If I have enough input on how everyone would like to do the scraping in an automation tool and why they’d like to do it, I could possibly run an internal initiative to make it possible.
So, what’s your website scraping use case that you’d like to automate?
This would be an awesome function, a few options that personally come to mind:
This would be great, I tried some application like visualping and wachete. Some of my use case:
That will be an awesome function.
I am looking forward to seeing this on activepieces.
Hi guys,
Recently I started using Apify, you can try it for free Apify: Full-stack web scraping and data extraction platform works really well and had some great results like searching for companies on Google maps based on location and based on their website scrap their e-mailadres and phone number. I do believe AP should keep their focus on creating a better integration platform to become real competitors of Zapier, Make, Pabbly and so on. So it would might be great to team up and create some great pieces which can scrap “anything”
Not hear to change minds but it think it’s good to not create new software if there is something good available.
Kr Bram
Ohh that’s awesome!! 
PLS tell me if you do develop scraping options!! I’d love it for outreach.
It would be a dream come true to scrape biz emails/leads to people that might be interested in your services! It take so much manual work to find those people and email addresses.
For example, scrape biz emails from google maps, filtered by area, type of business etc.
The extra Bonus would be for all scraped info to be imported into a Excel sheet or CRM platform.
So you automatically scrape and save each new email with details like:
Company role/website/type of business/mo company earning/area.
It would be any marketers dream!!
Read weather data from the BOM (ie the Australian government weather which is very reliable but doesn’t have a traditional API) send a notification when weather changes.
Create simple LLM “agents” who can research using google, go to a web page, read the web page, summarise the web page, draft blog posts based on research
Google a local band or artist, see if they have an instagram, facebook, spotify, or bandcamp, verify the links are actually for the same artist, add those verified links to a database or a promotional social media post.
Am looking on how to perform automated or scheduled web scraping.
For instance, every week new product appears on the AppSumo website, and I want to scrape that data automatically every week and pull it into a Google Sheet, but only the unique data that arrives each week.
I’d like to scrape flight prices, travel advisories, hotel reviews, and new blog posts about living overseas to create a “feed” of fresh information on traveling to specific destinations!
Will be the best addition for me. I like to automate list creation:
I’d like to scrape wedding vendors from google or yelp and add onto my brilliant directory website as a member(user) which ‘‘activepieces supported Brilliant Directories’’
I really need that feature in order to use my activepieces account. Thnx
This will be great for eg. lookiing to automate sale/deal or offer posting website where user submitted sales are posted and once the post goes live, will like activepieces to scrape the webpage and use that image on that post, share on social media and include it as a media file.
This could be great for SEO purposes, especially if we can retain the headline structures and so on.
Hi Guys,
For some of the readers scraping might be new, also the scraping via API, I currently have this set up for client with Frontly, Apify and Active Pieces. If some of you are interested I can create a video how I’ve done the set-up so you can use it too. @here
Let me know.
KR Bram
This would be wonderful!
This would be great. Thanks.
Perfect, will see what I can do, busy year end but maybe I find some time in between.
I would love to simply get a news link from an RSS feed - and get just the main content out of it.
Hi @here,
So I’ve actually found some time this morning to create a video, sorry for the poor quality of sound also this is one of the first times I am recording myself. But I hope you can take that for granted. I hope this will help you understand the use case and how I have set things up to help my client to extend their network with ease by using three different tools:
Here is a link to the video:
Scraping the web with Frontly, Apify and Activepieces.mp4
In case of any questions about this use case or other cases, please let me know.
I’ve also set-up the feature request for APIFY, APIFY: Webscraping so please vote this one up if you think this piece should be there.
KR Bram