Rough Measurement for HTTP Client Download Speed

Henrik G. Vogel  / pixelio.de
Ever wonder if your website is slow because of the server or because of the clients?
Do you want to know how fast is your clients' connection to the Internet?
Don't want to use external tracking services, injecting JavaScript etc.?

Why not simply measure how long it takes to deliver the content from your webserver to your users? Apache and nginx both support logging the total processing time of a request with a suitably high precision. That gives the time from starting with first byte received from the client and ending after the last byte sent to the client.

To try out this idea I added %D to the log format for access.log of my Apache server and wrote a little Python script to calculate the transfer speeds. With the help of the apachelog Python module parsing the Apache access.log is really simple. This module takes a log format definition as configuration and automatically breaks down a log line into the corresponding values.

The script can be found together with a little explanation on my GitHub page: github.com/schlomo/apacheclientspeed.

It is rather rough, I am now collecting data to see how useful this information actually is. One thing I can already say: Small files have very erratic results, bigger files yield more trustworthy numbers.

Videos are a good example, the output of this script looks like this:

$ grep mp4 /var/log/apache2/access.log | tail -n 20 | python -m apacheclientspeed
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 1409 KiloByte/s
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 119 KiloByte/s
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 1936 KiloByte/s
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 90 KiloByte/s
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 159 KiloByte/s
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 83 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 2067 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 43226 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 4 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 33491 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 28 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 0 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 0 KiloByte/s
89.204.139.1 GET /c/g2/GANGNAM_320_Rf_28.mp4 HTTP/1.1 0 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 0 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 0 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 0 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 0 KiloByte/s
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 507 KiloByte/s
80.226.1.16 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 46 KiloByte/s
The many requests with 0 KB/s are made by iOS devices. They do a lot of smaller Range requests when streaming a movie over HTTP.

Comments

Post a Comment

Like this content? You could send me something from my Amazon Wishlist. Need commercial support? Contact me for Consulting Services.

Popular posts from this blog

Overriding / Patching Linux System Serial Number

Image
I'm a big fan of test driven development  (TDD) for infrastructure components. I'm currently working on a hardware-related topic where we also use the system serial number as identifier. To create a proper integration test, we need to be able to start a system and set the serial number to a known value. This can easily be done with the help of virtual machines like in VMware or VirtualBox , but I couldn't find a way for changing the system serial number on hardware boxes, cloud VMs (e.g. on Alibaba Cloud) or other Linux system. Problem Analysis I was thinking: Linux is the operating system where I can potentially do everything . So how hard can this be? After some digging around I found out that there are those main sources for the serial number on Linux: /sys/firmware/dmi/tables/DMI contains a binary blob of Desktop Management Interface data provided by the kernel and the dmidecode utility is commonly used to decode...
Read more

A Login Security Architecture Without Passwords

Image
Following up on Lifting the Curse of Static Credentials and Eliminating the Password of Shared Accounts , I have many discussions about why we would benefit from removing password prompts for website logins. Let's dig deeper into the details and show why removing password prompts leads to a  safer security architecture . Update 11.03.2022: Added more details about business vs. consumer websites and additional security suggestions surrounding WebAuthn Problem Space For context, imagine a website that needs to identify online users via their email address. We assume that the website in question is not the primary email system of a user but some other website, e.g. an e-commerce shop system or a collaborative productivity tool. As a User As a user of that website I want to easily sign up for an account have an easy way to login into the account be sure that my account is protected from others or attacks be able to easily recover access to my acco...
Read more

Fixing Chrome Color Printing on Linux with HP Color LaserJet M880

Image
It seems like I can't get a printer that "just works". I recently decided to replace our HP X476 printer with something nicer and bigger, an HP Color LaserJet M880 ( background story & review in German ). And of course there is something that needs fixing: The Chrome browser on Linux wouldn't let us print in color, even though all other applications had no problem to print in color. Even with Chrome color printing was possible, if one used the system printing dialog instead of the built-in Chrome print preview. This strange behavior of course piqued my curiosity. After some digging around I found out that the Chrome browser needs to parse and understand  the printer driver PPD! Chrome tries to find out how to configure color or grayscale printing in order to offer the user the choice. If Chrome can't understand the printer driver then it simply doesn't offer the choice between color and grayscale — and then some printer driver default can ch...
Read more