Schlomo Schapiro

When developing or fine-tuning OpenSSH configurations the testing can be quite tiresome: Change configuration, restart server, run manual tests, repeat. Not to forget the many times when restarting the SSH server does not work and you lock yourself out of your test server. When writing a  Linux Magazin article about SSH key management  I wanted to show how to use OpenSSH PKI in a repeatable way. The result is an automated test suite for OpenSSH configuration : $ ./run_demo.sh    ... lots of info output running through ...   SSH PKI Demo Test Results: Succeeded create-ca-key Succeeded create-host-key Succeeded sign-host-key Succeeded create-user-root-key Succeeded sign-user-root-key Succeeded create-user-unpriv-key Succeeded sign-user-unpriv-key Succeeded test-trusting-known-hosts-via-cert-and-login-with-password Succeeded test-that-hostname-in-cert-must-match-target-host Succeeded test-login-with-root-key-trusted-by-cert Succeeded test...

When I bought a new Chromebook Acer C720  last week I got confirmation that times are changing: It has only an HDMI connector, no more VGA. Luckily, at ImmobilienScout24 we are also adapting and last month our big projector got an upgrade to Full HD with 16:9 Wide Screen. And you can now connect the computer through HDMI, too. Since me myself so much got used to creating presentations in 4:3 I took the opportunity to remind myself and everybody else why it really pays to pay attention to this little detail. Video is in German with English subtitles.

A colleague, Eric Grehm, raised an interesting challenge: How to maintain his personal work environment (VIM settings, .bashrc ...) on all servers? The first thought was putting this somehow into our software distribution, but we quickly realized that this would trigger needless updates on hundreds of servers. The benefit would be that the personal work environment is already on every server upon first access. The next idea is to switch from a pre-installed personal environment to an on-demand solution where the personal environment is transferred each time a remote connection (over SSH) is established. A simple implementation would just to a scp before the ssh, but that entails two connections which takes more time and might also bother the user with a double password request. Side-channel data transfer An alternative is to piggyback the file transfer onto the regular SSH connection so that the personal environment is transferred in a side channel: On the client creat...

Henrik G. Vogel  / pixelio.de Ever wonder if your website is slow because of the server or because of the clients? Do you want to know how fast is your clients' connection to the Internet? Don't want to use external tracking services, injecting JavaScript etc.? Why not simply measure how long it takes to deliver the content from your webserver to your users? Apache and nginx both support logging the total processing time of a request with a suitably high precision. That gives the time from starting with first byte received from the client and ending after the last byte sent to the client. To try out this idea I added %D to the log format for access.log of my Apache server and wrote a little Python script to calculate the transfer speeds. With the help of the apachelog Python module parsing the Apache access.log is really simple. This module takes a log format definition as configuration and automatically breaks down a log line into the corresponding values. ...

Do you ever get tired of typing sudo apt-get update && apt-get install <package> just to install one package that you added to your DEB repo ? I do and I decided to do something about it. What I really miss is the intelligence of yum which simply updates its repo caches if they are too old. apt-install  ( github.com/schlomo/apt-install ) is the next best thing. It is a simply Python script that updates the cache and installs the packages given as command line arguments. And it shows a nice GUI with a progress bar: Turns out that the parts are all there and part of aptdaemon . The only part missing was putting them together into this little script: Please note that I actually completely don't understand how to write async code. I'll be happy about all feedback with better implementations.

While working on the new Recorder (see also last posting) I suddenly faced several challanges with the resulting video files: Many short chunks (50MB each, about 30-60 seconds) need to be merged Extract the actual talk from a longer recording, e.g. the recorder was on for one hour but the talk was only half an hour Convert the video into another container format because Adobe Premiere does not like AVI files Create video thumbnails Convert videos to be compatible with HTML5 <video> playback Turns out that avconv (or ffmpeg ) is the swiss army knife for all of these tasks! I am not qualified to say which is better, for my purposes the avconv that ships with Ubuntu is good enough. The examples given here work with both. When I write avconv I mean both tools. Since I don't want to degrade the video quality through repeated decode/encode steps I always use  -codec copy after the input file to simply copy over the audio and video data without reencoding it. C...

Sometimes less is more. The most simple way to protect my private web space on my web server is this: <Location />     Order Deny,Allow     Deny from All     Allow from home.schapiro.org </Location> But what to do if home.schapiro.org changes the IP every 24 hours and if the reverse DNS entry (PTR) is something like  p5DAE56B9.dip0.t-ipconnect.de ? When my computer at home connects to the web server the source IP address is used for a reverse DNS lookup. This lookup returns the above mentioned provider-assigned name and not home.schapiro.org ,  the web server will never be able to identify this IP as belonging to my home router. The solution is to write the IP↔Name mapping for my dynamic IPs into /etc/hosts . That way a reverse lookup on the IP will actually yield the information from /etc/hosts and not ask the DNS system. Since I don't want to do this manually every time my IP changes, I automate it with this script. I...

At the office I got a 3 channel digital Audio/Video Recorder to conveniently record our talks without much human effort. The device has an analog video input for the video camera (standard resolution) and a digital video input (Full HD) and an audio input. Epiphan VGADVI Recorder These 3 inputs will be merged into a single side-by-side video where you can see the speaker next to his computer output. The video can be even larger than Full HD, for example 2688x1200 (a 768 pixels wide SD image next to a 1920 pixels wide HD image): The device is far from cheap (list price is 1840 € + VAT) and can really do a lot. For example, it can create H.264 movies with a bitrate of up to 9 Mbit. It can also upload the videos to a CIFS share, but sadly that works only at a transfer speed of about 4 Mbit! So how could I transfer the videos at really high quality settings (9 Mbit) to the CIFS share? Waiting 2 hours to transfer the videos of a 1 hour talk is no option. Linux and Open S...

Update (2019):  https://github.com/schlomo/rpi-image-creator  is the new home of the code. Recently we got a whole bunch of Raspberry Pi systems at work - the cheapest platform for building Dashboards . Everybody loves those little cute boxes - but nobody wants to deal with the setup and maintenance. The kiosk-browser package is of course also the base of our Pi-based setup. But how does it get onto the Pi? The solution is a Bash script: rpi-image-creator  available on the ImmobilienScout24 GitHub project . It automates the setup of a Pi with Raspbian by downloading a Raspbian image, customizing it a bit and writing it to a SD card. The reason to write my own script where the following features: It creates the file systems on the SD card instead of dumping a raw image onto the SD card. That way the partitions are all aligned properly and have the right size from the beginning. No later resizing needed It removes all the stuff that Raspbian runs at the first boo...

While writing some test scripts that use rsh (see below about why) to run commands on a remote test server I noticed that rsh and ssh have a significant difference: ssh reports the remote exit code and rsh does not .  As a result all my tests did not test anything, the error condition was never triggered: My solution is this rsh wrapper: The reason for using rsh instead of ssh is very simple: In a fully automated environment it provides the same level of security as ssh without the added trouble of maintaining it: I need to make sure that ssh continues to work after all the SSH host keys change (e.g. after I reinstall the target). Also, to allow unattended operation I must deploy the SSH private keys so that in the end others could also extract them and use them. In the end I would be using IP/hostname restrictions on the SSH server side to restrict the use of the private key. With rsh I don't need to worry about deploying or maintaining keys and just configure th...

Yesterday I was at the Berlin DevOps meetup and we had a very nice fishbowl about Test Driven Infrastructure (TDI). I used my Lightning Talk from the PyCon in Köln as an introduction to the topic, but quickly realized that the term does not fully explain itself. Test Driven Development  in itself is not a new thing, maybe it is not yet common to apply it to platform operations. As an old Ops guy I had a lot to learn when I started to work at ImmobilienScout24 , which is a real software development company. The bottom line is really simple: Untested = Broken My idea of TDI is to apply most of the basic ideas of TDD also to the development process  of the software that runs our platform. Again, the same thing as the developers do with their code already for a long time. Let's just say that we start to test all  code that goes on a server, no matter who wrote it or what it actually does. Some specific examples that we did in the last month: A service that ...

Recently I needed to prepare a USB thumb drive with several Ubuntu installations. A little research quickly yielded many setup instructions, for example like the one from Pendrivesystem.com .  I was really surprised at how well this works and wanted to understand it better. In essence all recipes rely on GNU GRUB in version 2 and the loopback feature that it contains and on the OS's ability to work off an ISO image. The loopback command  mounts a CD or HDD image that contains the kernel and initrd from an ISO image. As a result one can put several ISO images on the boot media without the need to extract them. The OS then also mounts the ISO image and uses that instead of a CD/DVD drive. So here is my version of the recipe, the USB thumb drive is in /dev/sdc  in my examples: 1. partition & format device I prefer to partition the device with parted because it aligns the partition at 1MB so that it leaves enough space for GRUB to embed itself into the first ...

For our team monitors that use the Kiosk Browser on a Raspberry Pi I am building a Raspbian wheezy image. One requirement is that the system will configure the hostname from DHCP so that we can use the same image for all devices. Sadly, setting the hostname from DHCP is not  trivial in Debian, here is the result of my research into the topic. I found 2 things to be essential and learned both of them by analysing the dhclient script. 1. Set the hostname to localhost The first thing to do is to set the system hostname to localhost:     # echo localhost >/etc/hostname 2. Workaround for broken dhclient-script The dhclient-script has (IMHO) a bug: If there is an old DHCP lease with a hostname in the lease database (e.g. in  /var/lib/dhcp/dhclient.eth0.leases ), then the script will not  set the hostname from DHCP even if the system name is still localhost. To circumvent this bug simply create a dhclient enter hook to unset the old host n...

Today I was walking a bit on the way home. After the third silent taxi (Toyota Prius Hybrid) passed by I started to think about what would be the ideal electric car for my use case. We are a large household and won't do with the standard 4 or 5 seat cars that are now beeing offered as electric or hybrid cars. Since the car makers don't offer what I need I am posting my idea here. This is an approximation of our current family car: It is nice, fairly large, seats 8 people comfortably and even has some extra luggage space in the back. We use it for short trips in the city to go shopping, drive the kids around and do the occasional family sunday trip to the surrounding country side. When we go travelling, we usually need a larger trunk so we have this add-on trunk: This is a trailer which is about half as long as the car and of the same height and width. It fits everything we need even for a long vacation. Or all our bikes for a shorter one. Or the insanely large...

I was again at the  PyConDE , this year in Cologne. As before, the conference was a mix between different types of talks. It seems like this year we had less people attending compared to last year, at least the crowd in the main hall looked much smaller (see photo). Many talsk where really interesting, but the lighning talks where the real highlight with lots of funny, useful or astonishing talks.b The conference included a beginners programming competition for school students (13-21), the 2 winners showed their project in the opening keynotes. The project had to do something with Blender and Python and one of the winners (a 13-year old boy!) presented a generated animation of a Skat game with a very solid software design. All that after just 9 months of learning Python was really stunning. Andreas Schreiber  most certainly gave up a lot of personal information in his very practical talk about the " Internet of Things ", in which he showed how to connect vario...