ScrapeNinja web scraper + Google Sheets for an output: scrape Hackernews and put news items into Google Sheets, executed every 1 hour.Google Sheets + a custom JS for processing + the Telegram bot integration: send new message to me in Telegram, when new row is added to Google Sheet, with a simple custom Javascript code to append a random string to the Google Sheet cell value.I have built 3 basic workflows in all 3 products: I am not a professional reviewer of SaaS and obviously I am not a journalist, I am an indiehacker busy building my own products. Youtube API are already outdated, and that corporate taste in the mouth. I actually don't like Google APIs and Google SDK for most of their products, as well, due to their complexity, huge legacy, when you can't even use StackOverflow properly because all examples of code on SO of e.g. I still hate it, and I still spend too much time on it, going back and forth in Google Cloud UI, setting up all these OAuth consent screens, even after doing it 10 times. One of the biggest pain points when writing my own simple automation scripts in pure Javascript (Node.js) and PHP is always OAuth, specifically Google OAuth, and even Google plain API keys. Low-code platforms were evolving all the time, and I think now it is a pretty exciting time to take another dive and see what can be built without burying myself too deep into coding. Do I even need a low-code solution? I can accomplish everything from scratch in Node.js! Oh, this was painful! During the last ten years, I always felt more comfortable launching npm init or composer init in a new empty folder on my remote server, in VS Code, because I was so fed up with visual website builders. The typical problem back then was the lack of proper code source control and poor tooling for database migration generation: this is when you add a new database field to a content type in a dev environment, using web UI, and you write some PHP code that relies on these fields, you put this code in SVN, which is what we used before GIT, and then you forget to 're-create' these fields on prod env when deploying the code of a feature to production. I mean, anything! I had the opportunity to experience all the benefits and all the pains of this approach. You could literally build anything in Drupal, without leaving your browser. During 2005-2012, I spent hundreds of hours building tens of websites using Drupal CMS, and if you don't know what Drupal is, this is one of the open-source monsters of 'visual website builders'. I have observed the rise of IFTT (winter of 2010), then Zapier, and, during the last 10 years, but this was mostly a glance at this, from the opposite side of a fence. ScrapeNinja is still a one-man product but it is growing successfully and I am definitely enjoying my ride. I am bootstrapping ScrapeNinja, which is a subscription-based API-first SaaS for web scraping. I am very comfortable building & consuming APIs, and a carefully written OpenAPI spec is a perfect piece of documentation for me. I have participated in building quite a few SaaS products, and I am still writing Javascript, SQL, and PHP (when I don't have to manage people). I am a technical lead with over 15 years of web development experience. This journey took longer than anticipated, and this writeup contains a summary of my conclusions and key takeaways. I wanted to do a simple job: extract some website data in JSON format from HTML every hour using ScrapeNinja, apply some simple JS processing, and put everything into Google Sheets. Please check out the examples for MySQL Docker.ĭifferent commands just indicate the database that NocoDB will use internally for metadata storage, but that doesn't influence the ability to connect to a different database type.A few days ago, I took a deep dive into integrating my ScrapeNinja web scrapers into Zapier,, and Integromat () to better understand the market situation among low-code and no-code automation platforms. If you plan to input some special characters, you may need to change the character set and collation yourself when creating the database. Otherwise your data will be lost after recreating the container. To persist data in docker you can mount volume at /usr/app/data/ since 0.10.6.

Zapier airtable github