[Linux] Before building a Kernel


This entry is about a little something I wrote down on what you want to do, before building a Kernel. It is aimed for users with some Linux experience that want to build their first Kernel (and fail). It’s not about the building process at all, but about the information process that needs to take place even before building the Kernel!


Content

  1. Introduction
  2. Starting situation
  3. Some brief words on the Kernel
  4. “Hi, I’m $user” – “Nice to meet you, I’m your CPU”
  5. What to do with all those information?
  6. Some other quite useful tips with the Kernel configuration (not quite related)

Introduction

I am writing this entry, because every now and then I encounter people that want to build a Kernel, and that are new to building one. These people normally don’t know where to start, so they look in the internet and find out how to download the actual stable sources, either at The Linux Kernel Archives, or they get something as follows:

gentoo root # emerge -u gentoo-sources

(Source: Gentoo)

root@debian:~# apt-get install linux-source-version
root@debian:~# tar jxf /usr/src/linux-source-version.tar.bz2

(Source: Debian)

[root@redhat ~]# rpm –Uvh kernel-headers-version.rpm kernel-source-version.rpm
[root@redhat ~]# rpm –ivh kernel-version.rpm kernel-ibcs-version.rpm

(Source: Red Hat)

The Debian Kernel upgrade should work fine with all Debian based distribution, such as grml, Ubuntu, Knoppix…). Same counts for the Red Hat way to all distributions that work with the Redhat Packet Management System (rpm). For more information on the different Kernel upgrade processes, you could have a look at the Handbook that came with your distribution, or Google – but in my opinion (and this is constantly proven by the distributions I used), you are most probably better of using the official Kernel sources, as any distribution just applies one patch after another, and in the end you are stuck up with a overly patched Kernel, which features you’d most likely do not need, but which in most cases just cause you to have funny problems.

Anyhow, the normal procedure now would be, that the manual just tells you to choose your hardware (sometimes you get a tip like looking into dmesg(1) or using lspci(8)), if you’re lucky you get some general options explained, but normally that’s it. Now any beginner just glances through the huge number of options, doesn’t know, what to choose, and ends up in any IRC channel, and in worst case gets a “Kernel building is nothing for beginners!”.

With this entry I want to make a change by telling you how to make it better. And this of course starts far before even downloading your Kernel. And this process is called: “Get to know your hardware”.


Starting situation

Now, the reasons why you want to build your own Kernel may be various. Most probably you already have a running generic Kernel, and you just have some hardware that is not working. You could also want to update your Kernel for some new features that you might want to make use of. Or you have an installation “form scratch”. Whatever the reason may be – I don’t want to discuss whether or not it makes sense, or whether or not you should. You can find that else where.

In most of the situations you already have a Kernel to fall back to. Anyhow I recommend for the next steps to use a “Live Distribution”, because they have ideal hardware detection routines at boot up. Any lsmod(8) or dmesg of a fully functional system is of better use than the ones of a system that has a faulty Kernel. But anyhow that is not a must do – but if you want to and wonder which distribution to take, I’d recommend grml. grml is a neat, light weight, slim distribution, that offers every system administrative tool you could think of, but nothing more. And for our purpose we don’t need any, let’s say Knoppix that offers 3D X Window effects using compiz and beryl. This of course is nice for presenting purposes, but does not meet our needs…


Some brief words on the Kernel

… in case you don’t know what exactly a Kernel is. Now when we use a computer, we actually use all different parts of hardware, that do things differently and where we don’t know, what they are capable of. And even the rest of the hardware that this piece is equipped to, doesn’t know what the piece is capable of. Actually no one of the pieces by themselves know, how any of the pieces work. Or to put it in other words: Hardware is nothing but a bunch of metal, plastic, rubber, and other worthless stuff, that by itself can’t do anything (except for consuming space and power).

To make this stuff work, we need the OS, which in our case is Linux, but which might also be Windows, Solaris, OS/2, etc. These OS not only make your hardware run – they also somewhat work as a layer, for your software to run on. Now any programmer that would want to write a program to print something would have to include information about all the printers there are if it wasn’t for the OS. This would not only lead to pretty unhandy work for the programmer himself, it also would lead to big programs and (as every program would need the same information over and over again) to waste of space due to redundant code.
Now actually the way that it is done is a different one: The programmer only makes it program to ask the OS to please print this data. And all the rest is handled by the OS itself.
Now the OS of course needs to know the hardware, and here’s where the Kernel enters the stage. The Kernel integrates all information there is about the hardware, and thus provides a layer to the OS and the applications, offering them the services the printer has, without telling anyone how this printer actually works. To know this the Kernel of course has to integrate little programs that exactly know how to access the hardware – these are the drivers.

To put it simple, the Kernel is just a bunch of put together hardware drivers (plus some information on how to handle different processes, etc.), separating the hardware from the software (there’s a bunch of further information with also a lot of neat visuals on this topic at Wikipedia).

Now for the Kernel to know which hardware there is… well there’s you, who’s actually willing to compile a Kernel with all the information needed. It’s as simple as that – the actual compiling is done by itself – you don’t have to know anything about that – all you do, is doing the configuration for the Kernel before it’s build. And that is telling him, what hardware he needs to be able to handle!


“Hi, I’m $user” – “Nice to meet you, I’m your CPU”

Do you know your hardware? I mean really know? All of this itzy bitzy parts? By heart? – Don’t worry, me neither. Nor do most of the people who need to build a Kernel. But anyway, why should you? That’s what tools are for. Now on Windows you have the hardware manager, and if you need more information… well buy software! (I can recommend Sandra by SiSoftware). On Linux of course we just have that software pre-installed (or it’s just a few clicks away). To proceed I’ll just define some of the pretty neat tools there are, and show you how you can use them, and what they’ll be telling you:

dmesg
Now the first thing to start with is dmesg. dmesg is an abbreviation for display message, and it is a tool that just reads out and displays the message buffer of the Kernel. Now the Kernel keeps a message buffer because it writes a lot of output on the screens (if it is allowed to), and sometimes you don’t see it, or there’re so much more messages following that you could not possibly read it that fast, e.g. at the boot process. Now these messages are exactly what we are interested in. When Linux is booting, the Kernel just get’s hold of all of it’s device drivers, that are compiled into the Kernel, and probes the hardware with them. E.g. you’ve compiled your Kernel with two different printer drivers the Kernel will just use the first one, probe it, and either it get’s some feedback, or it just skips it and takes the next one. Now if the hardware is found, there’s a bunch of information provided, such as what hardware was found. Now you see why using a faulty Kernel may affect our results here (e.g. no printer driver was selected at all).
Of course there’s also a lot of other messages included – whenever there’s a problem detected, or a new device (e.g. a USB pen drive) is detected, some messages are generated and therefore also accessible with dmesg.

Here are some snippets of information that dmesg might provide you with. Just see how much you could learn with that information alone – but there’s more to come 😉

telperion logs # dmesg
[    0.003487] CPU: L1 I cache: 32K, L1 D cache: 32K
[    0.003490] CPU: L2 cache: 2048K
(…)
[    0.061120] CPU0: Intel(R) Core(TM)2 CPU          6300  @ 1.86GHz stepping 06
 (…)
[    0.154768] CPU1: Intel(R) Core(TM)2 CPU          6300  @ 1.86GHz stepping 06
(…)
[    0.156695] Brought up 2 CPUs
[    0.156698] Total of 2 processors activated (7450.66 BogoMIPS).
(…)
[   35.866796] usbcore: registered new interface driver ati_remote
[   35.866801] ati_remote: 2.2.1:ATI/X10 RF USB Remote Control

That’s just really little information. But it’s enough to demonstrate the capabilities of dmesg. Here we’ll see, that we’re working with, an Intel Core 2 Processor that has two cores with the model number 6300, having 1.86GHz, as well as the L1 and L2 Cache values (lines 2-10). Now a little googling with that information will result in this processor being a Intel Core 2 Duo E6300 Conroe showing you all the capabilities the CPU has.
We even find devices on the USB bus, in this case our computer has a remote control receiver (line 12), so you could use any ATI/X10 compatible (line 13), e.g. an ATI Remote Wonder by AMD.

lspci
lspci is part of the package pciutils, which probably needs to be installed on your system (given that you don’t use grml). The cool thing about lspci is, that it probes the PCI bus, and therefore even finds hardware for which no drivers are installed in the Kernel.
A typical output could look like this:

telperion logs # lspci
(…)
00:0f.1 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 07)
(…)
00:12.0 Ethernet controller: VIA Technologies, Inc. VT6102 [Rhine-II] (rev 7c)
(…)
07:09.0 Multimedia controller: Philips Semiconductors SAA7131/SAA7133/SAA7135 Video Broadcast Decoder (rev d1)

Also here we get a lot of information. We now know the chip set of the IDE controller (line 3), or what network interface card is installed (a VIA Rhine) (line 5). Or that we could use the Remote control we found earlier with the TV tuner card by Philips (line 7).

Now, knowing the hardware parts of course is nice, but maybe we’ll want a little more information. E.g. when you look for the TV tuner card in the internet, you will surely find out that it’s a tuner chip. But maybe you really want to know, what card it is, that you use.
Just use lspci with the verbose switch:

telperion logs # lspci -v
07:09.0 Multimedia controller: Philips Semiconductors SAA7131/SAA7133/SAA7135 Video Broadcast Decoder (rev d1)
        Subsystem: Creatix Polymedia GmbH Device 000d
        (…)
        Kernel driver in use: saa7134
        Kernel modules: saa7134

Besides now knowing that it is manufactured by Creatix (line 3), which then of course together with the chip details let you find out that it is a Creatix CTX948 being able to look up the data sheet, or specific help with that device, etc, it even shows us the module that is used – useful information when we build our own Kernel! (lines 5-6)

lsusb
lsusb(8) is what lspci is for the PCI bus – a tool of the usbutils package, that allows you to probe your USB devices. Just let’s do it:

telperion logs # lsusb
Bus 001 Device 005: ID 148f:2573 Ralink Technology, Corp.
Bus 001 Device 006: ID 0bc7:0006 X10 Wireless Technology, Inc.
Bus 001 Device 004: ID 0424:2228 Standard Microsystems Corp. 9-in-2 Card Reader
(…)

Of course you’ll find hardware that was probed for while booting. But you’ll also find hardware that wasn’t, like our Card Reader (line 4), or this Ralink device (line 2). Let’s just see, what that is:

telperion logs # lsusb -v
Bus 001 Device 005: ID 148f:2573 Ralink Technology, Corp.
Device Descriptor:
(…)
iProduct                2 802.11 bg WLAN
(…)

Cool, it’s a wireless LAN interface! (line 5)

lsmod
lsmod(8) will list us all modules that are currently loaded. You could of course also access this information through the /proc pseudo file system at /proc/modules – but let’s just say, lsmod presents them more… nicely. This again gives us ideal information about what to put into our own Kernel that we’ll build – but of course this again only works if your current Kernel isn’t faulty. If you have hardware that the Kernel doesn’t have the modules for, they won’t show up here. So best is to make sure, to use a good generic Kernel as the one in the Live Distributions.
hwinfo
The last tool I want to present, and now this is a burner, is hwinfo(8). If you installed it, you can call it with the help switch to get an overview of what hwinfo is capable of doing:

telperion logs # hwinfo --help
Usage: hwinfo [options]
Probe for hardware.
--short just a short listing
--log logfile write info to logfile
--debug level set debuglevel
--version show libhd version
--dump-db n dump hardware data base, 0: external, 1: internal
--hw_item probe for hw_item
hw_item is one of:
cdrom, floppy, disk, network, gfxcard, framebuffer, monitor, camera,
mouse, joystick, keyboard, chipcard, sound, isdn, modem, storage-ctrl,
netcard, printer, tv, dvb, scanner, braille, sys, bios, cpu, partition,
usb-ctrl, usb, pci, isapnp, ide, scsi, bridge, hub, memory, smp, pppoe,
pcmcia, pcmcia-ctrl, wlan, zip, dsl, all, reallyall

The meaning of that is quite simple. You can have a log file for output that’s just hwinfo relevant, you can also see that information if you just called hwinfo. The most interesting option though is using hwinfo with the hw_item switches (line 9). What does that mean? Just replace hw_item with any of the given items (lines 11-15) and you’ll see. E.g.:

telperion logs # hwinfo --netcard
17: PCI 12.0: 0200 Ethernet controller
(…)
Model: "VIA VT6102 [Rhine-II]"
(…)
  Driver: "via-rhine"
  Driver Modules: "via_rhine"
  Device File: eth0
(…)
Link detected: yes
(…)
    Driver Status: via-rhine is active
    Driver Activation Cmd: "modprobe via-rhine"

I again stripped off all the non-interesting parts. So what do we see here? For starters it lists all network devices (also the wireless LAN device, but I’ve stripped that off). But more, we do not only see the model (line 4), we’ll also see the driver, that it uses – even if that driver is not installed or active (lines 12-13)! The “Activation Cmd” shows us, how to activate the driver (that’s interesting if the module has a different name, or if there are parameters to include). And we even see, to which file this device is bound to (line 8). As we talk, it’s even has an active network connection (line 10). On some devices you’ll even get speed information, for wireless LAN devices you’ll see encryption modes, authentication modes, etc. Of course you’ll also get the MAC address (stripped that). Quite cool stuff.

We’ll try another one:

telperion logs # hwinfo --framebuffer
02: None 00.0: 11001 VESA Framebuffer
  Memory Size: 256 MB
  Memory Range: 0xc0000000-0xcfffffff (rw)
  Mode 0x0300: 640x400 (+640), 8 bits
Mode 0x0301: 640x480 (+640), 8 bits
(…)
Mode 0x031b: 1280x1024 (+5120), 24 bits
(…)

Now this is also quite interesting. This’ll just show you the modes that you can use your frame buffer with. Even better, it offers you the correct hex value that you’ll need to provide your boot loader with, e.g my preferred mode is 1280×1024@24bit (line 8), so this is what you’ll find in my grub.conf:

kernel /boot/kernel-2.6.30r6-20091006-01 root=/dev/sda3 vga=0x031B

Just some of the nice things hwinfo offers. Play around with it to figure out more yourselves 😉

Now you see, we’ve already gained a lot of information on a computer that we (okey, you, I knew beforehand 😉 ) didn’t know. This PC could have been a total stranger to us. And all of this without even once looking inside the PC (the times when you needed to open and crawl into your PC to check out your hardware are most definitely over!)


What to do with all those information?

Now actually this is a lot of information that you’ll get, and you can’t possibly be thinking of reading all this and memorize it for your Kernel. It’s also not possible to just select some part’s as we do not yet know, which parts will be interesting to us. So here’s what I suggest. First of all you make a directory for all your hardware information. I’ll call it “logs”. Now in this logs-directory goes everything you’ll do. You may choose to have just one text file – I somehow prefer to have more than one. Now just probe your hardware, and do as much as detailed as possible:

telperion logs # dmesg > dmesg.txt
telperion logs # cat /proc/cpuinfo > cpuinfo.txt
telperion logs # lsmod > lsmod.txt
telperion logs # lspci –v > lspci.txt
telperion logs # lsusb –v > lsusb.txt
telperion logs # hwinfo --tv > hwinfo_tv.txt
(…)

Save all your logs (don’t forget that – on a Live CD they are lost as soon as you switch the machine off. So mount your home partition, or use an USB flash drive, or whatever (you’ll figure something out 😉 )).
Now here comes the deal. You build your Kernel, as you would, and whenever you encounter something that you don’t know, just probe your logs for it.
E.g. you build your Kernel, and you encounter something like this (help text on the Kernel config menu for the option [ ] Enable MPS table:

CONFIG_X86_MPPARSE:
For old smp systems that do not have proper acpi support. Newer systems
(esp with 64bit cpus) with acpi support, MADT and DSDT will override it

Now what to do? Find out if you have acpi, MADT and DSDT (we don’t probe for acpi, as I know I’ve got that):

 telperion logs # cat * | grep MADT
[    0.000000] Using ACPI (MADT) for SMP configuration information
telperion logs # cat * | grep DSDT
[    0.000000] ACPI: DSDT 3FEE3180, 6072 (r1 MEDION AWRDACPI     1000 MSFT  100000E)
[    0.161097] ACPI: EC: Look up EC in DSDT

Et voilà. This machine supports MADT and DSDT, so saying yes to MPS table is not necessary (but wouldn’t do any harm either).
The important thing, and I guess you’d already got that, is that you don’t have to be searching for all the information anymore. You just have it at your fingertips. With cat(1) and the wildcard (*) in your logs directory you just produce an output using all the log files you have. Handing that stream of data over to grep(1) which just get’s what you want, you’ll simply find what you need in no time. Just keep in mind that grep is case sensitive, so you’d most probably want to use the -i flag, if you don’t want to end up like this:

 telperion logs # cat * | grep madt
telperion logs # cat * | grep Madt
telperion logs # cat * | grep MADT
[    0.000000] Using ACPI (MADT) for SMP configuration information

telperion logs # cat * | grep -i madt
[    0.000000] Using ACPI (MADT) for SMP configuration information

But anyhow, that is much faster as now probing /proc/cpuinfo or googling with nothing but your processor name.

And if you have to much output, just pipe all that stuff to your favorite pager.

telperion logs # cat * | grep ACPI | vimpager

(Of course, you’ll need to have installed vimpager, as I have already described. If you don’t have it and/or don’t want it, you can also use something like more(1) or less(1), which should be available on any Linux. Oh, and if grep is just not showing the line you needed (as e.g. it is the next one), just use it like this:

telperion logs # cat * | grep -n3 test

This’ll show you three additional lines before and after the line in wich the word was found.


Some other quite useful tips with the Kernel configuration (not quite related)

Another thing that comes in quite handy is the search functionality of make menuconfig. In any menu just press / and it’ll open a search field allowing you to add whatever you want. This comes in handy when you search for a certain configuration item, say a module. E.g. if you’d followed my entry until now, you know that I’d need a via_rhine module for my network device.

A search for via_rhine shows me:

 Symbol: VIA_RHINE [=y]
Prompt: VIA Rhine support
   Defined at drivers/net/Kconfig:1741
   Depends on: NETDEVICES && NET_ETHERNET && NET_PCI && PCI      
   Location:
     -> Device Drivers
       -> Network device support (NETDEVICES [=y])
         -> Ethernet (10 or 100Mbit) (NET_ETHERNET [=y])
   Selects: CRC32 && MII

Now this information does not only show me, where to look for it (the option is called [] VIA Rhine support, and found under Device Drivers -> Network device support –> Ethernet (10 or 100Mbit) (lines 2 and 5-8). If we don’t see it, we’d know, that most probably one of the options this driver depends on, wasn’t activated, like e.g. NET_PCI (line 4). Now if you don’t know where to find that, just search again. Makes live pretty easy sometimes 😉

And if you’re stuck with not finding an option though you have activated everything asked for, and though you are most certain, that this option should be available, just check out the option at the general setup menu called

[ ] Prompt for development and/or incomplete code/drivers

Check it, if unchecked, and maybe then you’re driver will show up, just like magic.

Hope this helps you out. If it does, I’d be glad if you’d drop a comment 🙂

Please comment. I really enjoy your thoughts!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s