How to dynamically change http/s proxy servers in puppeteer?

Posted on February 14, 2020 in Scraping • Tagged with Puppeteer, Proxies, Issues, Proxy-Authentication • 5 min read

Find the updated blog post here.

Chrome/Puppeteer has a couple of annoying issues when trying to use http/s proxies and socks proxies with the chrome browser controlled by puppeteer. The most pressing issues are the following:

  1. **Dynamically changing proxy servers: ** Once the chrome browser is started, it is not possible to change the proxy configuration any longer. A restart is required to switch proxy configuration.
  2. user/pass proxy authentication: The chrome browser does not support username/password proxy authentication for socks proxies. Puppeteer supports the http proxy authentication via the page.authenticate() function, but it does not have an equivalent for socks proxies.
  3. **Per page proxies: ** Per page proxies are not supported with the chrome browser. The global proxy configuration applies to all pages and windows of a launched chrome process. It seems like the new module tries to solve this issue.

For my purposes, I don't really care about problem 3). I don't need per page proxies anyway, since the crawling software I write runs with one browser tab at the time. However, issue 1) is a mandatory requirement for me and thus needs to be solved.

The reason is, I don't want to restart the browser …


Continue reading

Using http/s and socks4/5 proxies with puppeteer and chrome with squid and danted

Posted on February 12, 2020 in Scraping • Tagged with Puppeteer, Proxy, Danted, Squid, socks4, socks5 • 5 min read

This blogs post demonstrates how puppeteer and the chrome browser can be used with http/s and socks4/5 proxies. For that reason, a proxy server is setup on Ubuntu 18.04 with squid3 and dante.


Continue reading

5 crucial tips how to survive riding a motorbike/scooter in Thailand (2019)

Posted on October 26, 2019 in Traveling • Tagged with Thailand, Scooter, Motorbike, Koh Phangan, Ko Tao, Krabi, Koh Lanta • 3 min read

Thailand is the second most deadly country when it comes to traffic accidents. 80% of all deaths originate from people driving motorbikes. In this blog post, I try to share my experiences in the form of 5 survival tips in a honest way. I drove a scooter on 4 distinct tourist destinations in Thailand without a proper license and wasn't caught in a police checkpoint a single time.


Continue reading

4 reasons why you should NOT travel to Koh Phi Phi

Posted on October 22, 2019 in Traveling • Tagged with Thailand, Koh-Phi-Phi • 5 min read

In this quick blog post I outline why you probably don't want to go to Koh Phi Phi.


Continue reading

Model Based fuzzing of the WPA3 Dragonfly Handshake

Posted on October 19, 2019 in Security • Tagged with WPA3, Dragonfly, SAE, Fuzzing, libFuzzer, Greybox Fuzzing • 8 min read

The results of my Master thesis named Model based fuzzing of the WPA3 Dragonfly handshake will be quickly discussed in this blog post. No severe vulnerabilities were discovered, mostly due to the limited deployment of WPA3 hardware since it's introduction. However, a DoS vulnerability in iwd was found.


Continue reading

Review of the Koh Phangan Full Moon Party in Octobre 2019

Posted on October 15, 2019 in Traveling • Tagged with Thailand, Koh Phangan, Full Moon Party, Jungle Party, Jungle Experience, Goodtime Backpackers • 7 min read

Travel experiences during the days of the full moon party in Koh Phangan. The Jungle Experience Party is a huge scam, the Waterfall Party on Koh Phangan might be even worse.


Continue reading