Happy Birthday, Debian!


Debian 25 years Thank you! Image

Image taken from https://bits.debian.org/2018/08/debian-is-25.html, published under the MIT License (see: https://www.debian.org/license)

On Aug 16th, 1993 the 20 year old Ian Murdock released a message on comp.os.linux.development (that’s a Usenet Newsgroup – Usenet was the way to connect to and interact on the Internet before there was the World Wide Web that you are using now (that’s why you have to put www infront of most websites), and Newsgroups where the forums/message boards of the Usenet) announcing his new Linux distribution called Debian (named after his then girlfriend/later wife Debra and himself Ian).

Before distributions, Linux only consisted of the kernel itself that was in source code – so you had to download that onto your host system, compile it, and by hand from your host system set up your machine with all the components needed to run it as a Linux system. This was really complicated, and so distributions arose, allowing the user to simply install the system from a bootable device:

Debian will contain a installation procedure that doesn’t need to be  babysat; simply install the basedisk, copy the distribution disks to the harddrive, answer some question about what packages you want or don’t want installed, and let the machine install the release while you do more interesting things

For the elder generations, 25 years might not seem as much. But watching back, this actually makes Debian one of the oldest Linux distributions and probably the oldest one that is still actively used; in comparison, the Linux kernel itself is just 2 years older! Praised for its stability it is the choice of system administrators.

Debian’s package management system is – as far as I could research it – the oldest and therefore first one, and it is still actively used and ported to different systems even today: .dep packages were ported to the UNIX System V via OpenSolaris, a port to BSD via the UNIX-like FreeBSD, which also macOS is derived from, and speaking of Apple – Fink brought Debian’s package management system to macOS and Cydia to iOS. According to Murdock himself, package management is “the single biggest advancement Linux has brought to the industry”. He himself later worked for Docker, which is – if you will – in a way an even bigger package managing software in the sense that it not only packages the software itself but also the entire running system.

It also is the basis of countless derivates, from which the most famous or important are probably Knoppix, Grml, Kali Linux, Raspbian and of course the most important of all: Ubuntu, that itself is the basis of countless additional derivates (the Wikipedia has a great graph showing all Linux distributions and how they a related to each other – Debian makes up for more than a third of that space). And Distrowatch’s Top 10 Distributions list (which actually contains 11 distros) lists 3 Debian-based distros, including Debian itself.

So it’s more than fair to say, that Debian had a great impact on the Linux world, helped shape it to what it is today. I myself therefore wish the Debian system and it’s developing team a very happy birthday, and hope that it will see the next 25 years with same prosperity!

And if you are puzzled as to why it is I talk about Debian – well I myself used that distribution for a year in my early Linux days. I started out with S.u.S.E. Linux 6.3 at arround 1998-2000. I got fed up with it after they introduced YaST2 for SuSE 8.0 and switched to Debian 3.0 aka Woody in 2004. However, we weren’t meant to be for each other. It was incredible outdated, which made me really laugh at this passage of the Newsgroup post:

Debian will contain the most up-to-date of everything.

While other distributions of that times where enjoying Kernel version 2.6, Debian came with 2.2 in the stable branch. It was also missing a lot of software – which at that time where Linux software was still a rare thing, meant a lot, especially in the everyday personal desktop PC context. After mixing stable, testing and unstable packages and dreading every update, because it meant another weekend trying to repair the system, I once more switched – to Gentoo.

However, I was always fond of Debian, their goals and their Manifesto. Debian has always been my goto distribution for running servers. And lately – well, in April I switched from Gentoo to Ubuntu as my main driver. For mainly two reasons:

  1. With the last months and years Gentoo has become more and more unstable. While in my beginning years I could simply run emerge -avuND world && shutdown, go to bed and have an up-to-date system the next morning, nowadays most times this fails and I need to spend hours and days fixing it. That was my main reason to leave Debian long time ago – now it’s the reason for me leaving a distribution that has been my main driver for 14 years. That’s hard, yes. But also somewhat exciting.
  2. I need certain software that unfortunately is just developed for Ubuntu and does not work under other Linux distributions – either at all, or only with limitations. I am really frustrated about that, because developing software only for one certain distribution is not at all the Linux way and shouldn’t be rewarded – however if you need the software…. This list of software includes (without being complete):
      • ROS: Supported platforms are Ubuntu and sometimes Debian. On Gentoo installation was tricky but possible, however some packages weren’t ported. The maintainer is fast to react, but as I really needed one package really fast, the switch was inevitable.
      • Rock: Like ROS but with real-time support from Orocos, developed by the DFKI. Like ROS it’s Ubuntu only, even if the website says something different. I tried the installation under Gentoo, Arch and macOS and blatantly failed. During my time at the DFKI I did not meet anyone using anything else than Ubuntu, and even on Ubuntu installation was buggy as hell.
      • Unreal Engine: While under Linux you always have to build it from source, and it is not Ubuntu-only (and provides installation pointers for CentOS, Fedora, Arch and Mint), UE4 runs more smothly on Ubuntu and is easier to build than it was under Gentoo.

While Ubuntu wouldn’t have been my first choice I have to say that I am pretty happy with it. It’s different for sure and I still have a number of smaller issues, but we get along. On the longrun however I might also be looking into Debian once again.

PS: On a rather sad note – in 2015 for publicly unknown reasons the founder and inventor of Debian (an American who was actually born in Konstanz, Germany!) killed himself on rather mysterious circumstances after having something that could be called a nervous breakdown at the age of 42, leaving behind three children. May he rest in peace. Your legacy will live on!

Advertisements

Automount a specific USB drive


Actually this is a straight forward thing, however since it has been a while I had to google it myself, and was astonished about how many non-working solutions I found, besides solutions that simply mount every USB according to their label. So here is the straight forward solution to mount a specific USB device to a specific location on your Unix hierarchical file system, using udev. It assumes that you have a running version of udev, and the udev tools. If not, please consult the distribution specific documentation on the Linux distribution of your choice. This might include recompiling your Kernel, as udev will need the following settings:

General setup --->
[*] Configure standard kernel features (expert users) --->
[ ] Enable deprecated sysfs features to support old userspace tools
[*] Enable signalfd() system call
Enable the block layer --->
[*] Block layer SG support v4
Networking support --->
Networking options --->
<*> Unix domain sockets
Device Drivers --->
Generic Driver Options --->
() path to uevent helper
[*] Maintain a devtmpfs filesystem to mount at /dev
< > ATA/ATAPI/MFM/RLL support (DEPRECATED) --->
File systems --->
[*] Inotify support for userspace
Pseudo filesystems --->
[*] /proc file system support
[*] sysfs file system support

As for Gentoo Linux, the other things you will want to do, is to add “udev” to your USE-flags (by adding it into your /etc/portage/make.conf), get udev installed (calling emerge -avuD sys-fs/udev), and add udev to your sysinit runlevel (rc-update add udev sysinit).

Now to the fun part. First of all you need to get some information about the device you are interested in. There are a number of ways, like using udev monitor, etc. Most of them, to me however, are too messy. If you have no idea about your device and still need to figure things out, blkid -o list will show you a nice table of all devices, their device file, file system type, label, mount point and UUID – everything you need. For me, I know I have a stick with the label “Public” on an OS X with file system type exFat, and now I inserted it into a dual boot Linux with a number of partitions:

ancalagon ~ # blkid -o list
device                            fs_type      label         mount point                           UUID
---------------------------------------------------------------------------------------------------------------------------------------
/dev/sdb1                         vfat                       /boot                                 58FB-332D
/dev/sdb2                         swap                       [SWAP]                                73db158f-0e19-4d17-8c88-a8b0c1dff1f3
/dev/sdb3                         ext4                       /home                                 355af6d8-6f03-4a98-9a45-edafc3ccedde
/dev/sdb4                         ext4                       /                                     63be67f3-5c7c-48ea-a8b3-58dff9da1737
/dev/sda1                         ntfs         Wiederherstellung (not mounted)                     562065062064EF05
/dev/sda2                         vfat                       (not mounted)                         6265-B138
/dev/sda4                         ntfs                       (not mounted)                         6C58731C5872E46C
/dev/sdc1                         exfat        Private       /media/private                        56B6-CE90
/dev/sdd1                         exfat        Public        (not mounted)                         56BE-6477
/dev/sda3                                                    (not mounted)

If you want more information, with the device file you can get it with:

ancalagon ~ # udevadm info /dev/sdd1

I want the stick to be mounted at /media/public, so I need to create a rule file; on Gentoo it lies under /etc/udev/rules.d/90-local-usb.rules. Actually the name is totally arbitrary, except for the number at the beginning, and the extension that always has to be .rules. The number should be something high, because we want udev to first run all other rules (e.g. the ones that assign the device to a device file) before running ours. 90 is a good value for that.

So in my case, this is what I added:

SUBSYSTEMS=="usb", ENV{ID_FS_UUID}=="56BE-6477", ACTION=="add", RUN+="/usr/bin/logger --tag udev Mounting public", RUN+="/bin/mount -o umask=0077,nosuid,uid=1000,gid=1001 '%E{DEVNAME}' /media/public"

We need to provide the system or subsystem, which for an USB device is usb. The UUID comes from blkid and identifies the device. The action triggers when to run the command. In our case, when a new USB device is added and it has the UUID we want. And finally the mount command. I’ve added another command such that there is a log entry but thta is no need. And as I want it to be accessible as user, I added uid and gid accordingly. If you need to find out your user and group id, just run:

ancalagon ~ # id -u 
ancalagon ~ # id -g 

And that’s it. If you want to see if the rule triggers, just run

ancalagon ~ # tail -f /var/log/everything/current

It should output:

[udev] Mounting public

somewhere. And you can <em>simulate</em> the USB event with udevadm, by triggering the rule you just wrote (although this is rather interesting for more general rules that should fit more than just one device). This is how it’s done:

ancalagon ~ # udevadm trigger --action="add" --property-match=ID_FS_UUID="56BE-6477"

researchr on Linux (Addemdum to Computer-aided Scientific Workflow)


In my last blog entry, I presented the extemporary, yet neat solution for an academic workflow, which was unfortunately limited to Mac. Though I own a MacBook myself, I am also very passionate about Linux and like to find solutions that are plattform-independent.

I already suspected, that given setup should be easily portable to Linux, only the Open Source tools Skim and BibDesk would take some deeper programming, as they depend on the Cocoa Library. But one could find alternatives:

So the presented workflow should be more or less reproducible on a Linux system. Get a feel for why this workflow is – in my opinion – ingenious, and then try porting it to Linux. The community of scientists using Linux will most definitely appreciate it.

One way to replace Skim and BibDesk would be to turn to an integrated solution, such as the Cross-Platform solution Mendeley, which uses the Qt framework and is therefor available for Linux, Mac and Windows. It is similar to Papers, but in my opinion, Mendeley seems to be much leaner. It also offers some social-network features, that other reference management systems lack.

The Ph.D. student Bodong Chen (who incidentally also studies at the University of Toronto) tried it, and seems to have succeeded. On his researchr-Wiki he gives some pointers on how it’s done (and it seems, like he also tried to do so on Windows, but as I suspected, there seems to be no success).

So, to all you Linux-Heads out there, here’s a solution for you, too. Try it out, make it better, document it, and give me a link, if you do 😉

Non-breakable Space Problems


On OS X and apparently also some Unix-based machines you may have encountered a strange behaviour (in my experience especially on Laptop keyboards). On the shell it sometimes might just happen that you pipe something and get an error message. E.g.

pygospa@lalaith ~ % ls -ahl | grep .log
zsh: command not found:  grep

Retyping it may then bring you:

pygospa@lalaith ~ % ls -ahl | grep .log
-rw-r--r--    1 pygospa  staff    54B 15 Nov 11:55 tmux-client-8140.log
-rw-r--r--    1 pygospa  staff    24K 15 Nov 11:55 tmux-server-8142.log

So, what just happened? To me this actually happens so seldom that I never even bothered to try understanding. Therefore I often encountered another problem wich started annoying me. When editing a TeX-File, every now and then I got this error message:

! Package inputenc Error: Unicode char \u8:  not set up for use with LaTeX.
l.193 \end{align*}

I discovered that rewriting (not copying!) the line does some good. So it had to be some non-printing control character, and after searching for “OS X, Latex \u8 error” (I initially assumed that it must be an OS X thing, as I never encountered it in LaTeX on my Linux days – I now assume that it’s rather a Keyboard thing) I found a quick and dirty solution to it, adding this line:

\DeclareUnicodeCharacter{00A0}{ }

So the non-visible Unicode sequence was overwritten with a space. And thus I was never bothered again when editing LaTeX.

But now, while writing some RSpecs for a Ruby on Rails project I am working on it occurred again, and in RSpec there’s no quick and dirty solution that can be written into some preamble to fix an input error.

Continue reading

Ein recht objektiver Vergleich von Betriebssystemen


Christian Schneider ist zwar oft auch dafür bekannt, ein wenig zu Flamen – aber er hat durchaus auch ein gewisses Know-How um eine Aussage über gute Betriebssystem treffen zu können. Einen sehr schönen Artikel hat Christian gerade veröffentlicht, und stellt dabei heraus, warum Linux nicht unbedingt die beste Wahl ist, und gerade Debian alles andere als das Non-Plus-Ultra ist. Ich finde ihn sehr lesenswert, weshalb ich hier mal einen “Just Found on the Internet” Eintrag veröffentlichen musste 🙂

[Linux] “0.107 No irq handler for vector” aka “Problems with my PCIe Port”


As you may have already guessed, I was currently building a new Kernel for my PC (a md8818), and while doing so I encountered a strange problem, which I didn’t quite understand.

Of course, first thing to do is to ask Google, and I found out that I wasn’t the only one having that trouble, but it also seems like no one really had a solution to it (except for suppressing the symptoms). It took me 11 Kernel builds, until I finally eliminated the problem, so to help you not go through all this mess yourself, I’d like to present the problem plus solution.

Continue reading

[Linux] Before building a Kernel


This entry is about a little something I wrote down on what you want to do, before building a Kernel. It is aimed for users with some Linux experience that want to build their first Kernel (and fail). It’s not about the building process at all, but about the information process that needs to take place even before building the Kernel!

Continue reading