I am a heavy user of YouTube. I use it to listen to podcasts while cooking or in order to watch the latest documentaries before going to sleep. But lately, the extremely aggressive advertisement of YouTube sparked enough motivation within myself to remove YouTube ads for good. Google overdid it. I have enough.

Cross domain requests with <img> tags are not bound to the same origin policy. I will shed light on several possibilities how malicious web site owners can potentially abuse cross domain request done with <img> and script tags created with JavaScript.

In this article, I demonstrate how to reliably communicate JSON data to a cross domain server after the user is about to end or interrupt the browsing session by either:

• switching the focus to another page
• switching from the browser to another applicaton
• closing the tab
• closing the browser

or any other means of terminating or interrupting the current browsing session. Mobile devices and desktop devices should be equally supported.

Why do I have this very specific requirement?

I am in the process of developing a JavaScript analytics application and I need to record user interactions and send those user interactions from any website to my remote server.

Put differently: I need to record user interactions up until the point where the user leaves the browsing session. The ideal event for this scenario is to attach an event listener to visibilitychange:

document.addEventListener("visibilitychange", function() {
  if (document.visibilityState === 'hidden') {
    localStorage.setItem('triggeredOnPageClose', new Date());
  }
})

This event fires when the user loses focus of the current window and the page visiblity becomes hidden, for example when the user changes the tab.

However, is the above event also triggered when the user closes the page or closes the entire browser? In order …

In this blog article I will introduce my most recent project: The distributed crawling infrastructure which allows to crawl any website with a low-level Http library or a fully fledged chrome browser configured to evade bot detection attempts.

This introduction is divided into three distinct blog articles, because one blog article would be too large to cover this huge topic.

1. (This article) The first part of the series motivates the development of the crawling infrastructure, introduces the architecture of the software and demonstrates how the crawling backend works at a high level.
2. The second part covers the installation of the distributed crawling infrastructure within the AWS cloud infrastructure and tests the freshly deployed stack with a test crawl task.
3. In the third part of this tutorial series, a crawl task with the top 10.000 websites of the world is created. The downloaded Html documents are stored in s3. For the top 10.000 websites, we use the scientific tranco list: A Research-Oriented top sites ranking hardened against manipulation. As a concluding task, we run business logic on the stored Html files. For example, we extract all urls from the Html documents or we run analytics on the <meta> tags …

Quick script that demonstrates how to create s3 buckets in many regions.

This article makes an attempt to understand and predict the consequences of the rapid automation/computerization in the realm of human work. I make them based on my experiences as a software engineer, while I am fully aware that programming is not threatened to be eradicated by automation in the next decades. For that reason, I realize that I am holding a privileged position.

In the second part of the article, I argue why work in general should be voluntary within a welfare state and why the governments main task should be to create a situation where humans dont' have to work in order to obtain the bare essentials such as housing, food and healthcare.

What is work?

So what exactly is work? Work is something that sucks, right? After all, if I could spend my time without having to worry about finances, I'd rather spend my days at a tropic beach drinking some beers and doing exactly nothing.

But for how long? When would my natural urge to be productive kick in? I assume that after a couple of days, I would start a project that consumes my time. But since I don't have to worry about it's financial …