Age of Scrapers — Jul 26, 2023

• LLMs and Generative AI are trained on threads and blogs across the internet, which means they have massive scrapers running. No-brainer, because we trust what is written on forums such as Reddit, Twitter, 4chan, and HackerNews more than we trust blogs.
• Workflow software, such as Zapier, n8n, make, and IFTTT, is basically about connecting one app to another based on if-else conditions. For example, a simple workflow could be: when someone signs up on my app, add to Salesforce, notify me on Slack, and add the data to Google Sheets.
• Making all apps expose their endpoints as webhooks is a more sustainable solution, but a workaround for the next few years to connect one app to another will involve accessing the other app's features via scrapers.
• So, a prediction is that scraper tools will appreciate and welcome good investments. Watch out for tools like Bardeen, ZTool.