Scraping 1 million keywords on the Google Search Engine

Posted on in Scraping • Tagged with puppeteer, web scraping, headless chrome, 1 million, queue, architecture • 5 min read

Scraping one million keywords is not a easy task. There are proxy problems, big data problems and reliability issues. In this blog post, the most valuable insights are shared.


Continue reading

The dangers of a poorly planned project

Posted on in Philosophical • Tagged with Architecture, Captcha, Philosophical, Programming, Php, Uncategorized, Wordpress • 9 min read

Preface

Do you like to fiddle around with programming projects in your spare time? And do you sometimes start endeavors ambitiously, but you never actually finish them? Are you fucking tired of stacking unsuccessful projects, doing mediocre work while never being thoroughly satisfied in what you do?

If yes, you may be inclined to listen to some words I have to say over my most recent failed project:

The idea was to create my own little captcha plugin for wordpress. You can learn more about the idea by delving into some of my accompanying investigations in the following blog posts:

Honestly I started this project because back in the time I was using this plugin and I was unsatisfied because for these reason. So this context information hopefully points out some of my motivations to start the project in the first place.

The destiny of every badly planned project

As with many spontaneously started projects in came up with in the past, I first was convinced that it was an awesome idea and subsequently started programming head-first without having a clear path or …


Continue reading