Behavioral Analysis for Bot Detection

Posted on April 11, 2021 in Security • Tagged with Behavioral Analysis, Bot Detection • 4 min read

Behavioral analysis is an interesting approach to detect bots. It surely is not the panacea for bot detection, but it certainly is an useful extension in your bot hunting tool belt.


Continue reading

TCP/IP Fingerprinting for VPN and Proxy Detection

Posted on March 13, 2021 in Security • Tagged with TCP, IP, fingerprinting, Proxy, VPN • 9 min read

TCP/IP fingerprinting is as old as the Internet itself. But this technique seems to have lost it's relevancy in our modern times. However, with the rise of Proxy and VPN Providers, TCP/IP fingerprinting becomes interesting again from a security perspective.


Continue reading

Detecting scraping services

Posted on March 11, 2021 in Scraping • Tagged with detecting, scraping, security, fingerprint • 13 min read

In this blog post I will demonstrate how it is possible to detect several scraping services: luminati.io, ScrapingBee, scraperapi.com, scrapingrobot.com, scrapfly.io.


Continue reading

7 Common Mistakes in Professional Scraping

Posted on March 01, 2021 in Scraping • Tagged with web scraping, crawling, puppeteer, playwright • 13 min read

In this blog post, I am talking about my several year long experience with web scraping and common mistakes I made along the road. The more I dive into web scraping, the more I realize how easy it is to take wrong decisions when scraping a site. For that reason, I compiled a list of seven common mistakes in regard to web scraping.


Continue reading

Why does this Website know that I am sitting on the Toilet?

Posted on February 05, 2021 in Security • Tagged with JavaScript, deviceorientation, devicemotion • 4 min read

Android mobile devices give to any website device orientation and device motion data. This data is quite sensitive in nature and should not be granted to websites without obtaining explicit user consent.


Continue reading

Headful Google Chrome with Xvfb on AWS Lambda Container

Posted on January 23, 2021 in Tutorials • Tagged with AWS Lambda, Xvfb, Docker, Container • 4 min read

The following write-up is an attempt to launch headful Google Chrome with Xvfb on AWS Lambda container.


Continue reading