Making entries in the /proc virtual file system

As a first step we present we are going to the whole solution of the exercise, then we will „celebrate the source code”.

Creating the module in VirtualBox.

The solution is presented using the virtual machine Batfai_Prog1.ova.

Exercise: making entries in the /proc

Write a kernel module that prints debug information about processes into a file under /proc.

This example already appeared in the original Paternoster [PP] (pp. 177). Further background information can be found in the books [LDD] and [KMGUIDE] and especially in the following articles

  1. Now we have made the module. Its source code called sbejegyzes.c can be found in the PROP_peldak/sajat_bejegyzes_modul directory of the virtualized system.

    #include <linux/init.h>
                    #include <linux/module.h> #include
                    <linux/kernel.h> #include <linux/sched.h> #include
                    <linux/list.h> #include <linux/proc_fs.h> #include
                    <linux/stat.h> #include <linux/seq_file.h>
                    MODULE_DESCRIPTION ("Ez a sajat bejegyzes (PROP konyv) kernel
                    modul"); MODULE_AUTHOR ("Bátfai Norbert (nbatfai@gmail.com)");
                    MODULE_LICENSE ("GPL"); static int taszk_lista (struct
                    seq_file *m, void *v) { int i = 0; struct task_struct *task;
                    struct list_head *p; // a kernel/sched/core.c-ben latott modon
                    iratjuk ki // a betut (szemben a fs/proc/array.c-ban
                    lathatoval) // stat_nam[task->state ? __ffs(task->state)
                    + 1 : 0] static const char stat_nam[] =
                    TASK_STATE_TO_CHAR_STR; seq_puts (m, "sajat taszk lista
                    (negyedik kernel modul, PROP konyv)\n"); seq_printf (m, "%-9s
                    %-16s %-6s %-12s %-6s %-3s\n", "#", "CMD", "PID", "FLAGS",
                    "ST1", "ST2"); list_for_each (p, current->tasks.next) {
                    task = list_entry (p, struct task_struct, tasks); seq_printf
                    (m, "%-9i %-16s %-6i %-12u %-6li %-3c\n", ++i, task->comm,
                    task->pid, task->flags, task->state,
                    stat_nam[task->state ? __ffs (task->state) + 1 : 0]); }
                    return 0; } static int sajat_open (struct inode *inode, struct
                    file *file) { return single_open (file, taszk_lista, NULL); }
                    static struct file_operations sajat_fajl_muveletek = { .owner
                    = THIS_MODULE, .open = sajat_open, .read = seq_read, .llseek =
                    seq_lseek, .release = single_release }; static struct
                    proc_dir_entry *sajat_proc; static int sbejegyzes_init_module
                    (void) { struct proc_dir_entry *sajat_proc_fajl; if
                    (!(sajat_proc = proc_mkdir ("sajat", NULL))) {
                    remove_proc_entry ("sajat", NULL); printk (KERN_NOTICE
                    "/proc/sajat/ letrehozas sikertelen\n"); return -1; } printk
                    (KERN_NOTICE "/proc/sajat/ letrehozva\n"); if
                    ((sajat_proc_fajl = create_proc_entry ("taszk_stat", S_IFREG |
                    S_IRUGO, sajat_proc))) { sajat_proc_fajl->proc_fops =
                    &sajat_fajl_muveletek; printk (KERN_NOTICE
                    "/proc/sajat/taszk_stat letrehozva\n"); return 0; } else {
                    remove_proc_entry ("taszk_stat", sajat_proc);
                    remove_proc_entry ("sajat", NULL); printk (KERN_NOTICE
                    "/proc/sajat/taszk_stat letrehozas sikertelen\n"); return -1;
                    } } static void sbejegyzes_exit_module (void) {
                    remove_proc_entry ("taszk_stat", sajat_proc); printk
                    (KERN_NOTICE "/proc/sajat/taszk_stat torolve\n");
                    remove_proc_entry ("sajat", NULL); printk (KERN_NOTICE
                    "/proc/sajat torolve\n"); } module_init
                    (sbejegyzes_init_module); module_exit
                    (sbejegyzes_exit_module);
  2. The module has been created into the kernel object file sbejegyzes.ko using the command make.

    [norbert@matrica sajat_bejegyzes__modul]$ make make -C
                    /lib/modules/`uname -r`/build M=`pwd` modules make[1]:
                    Entering directory `/usr/src/kernels/2.6.42.12-1.fc15.x86_64'
                    CC [M]
                    /home/norbert/kernelfa/sajat_bejegyzes__modul/sbejegyzes.o
                    Building modules, stage 2. MODPOST 1 modules CC
                    /home/norbert/kernelfa/sajat_bejegyzes__modul/sbejegyzes.mod.o
                    LD [M]
                    /home/norbert/kernelfa/sajat_bejegyzes__modul/sbejegyzes.ko
                    make[1]: Leaving directory
                    `/usr/src/kernels/2.6.42.12-1.fc15.x86_64'

    We assume that the Makefile is in the same directory as sbejegyzes.c. The reader can be familiar with Makefiles for kernel modules from the Documentation/kbuild/modules.txt of the kernel source tree.

    obj-m += sbejegyzes.o all: make -C /lib/modules/`uname
                    -r`/build M=`pwd` modules clean: make -C /lib/modules/`uname
                    -r`/build M=`pwd` clean rm *~
  3. The created module can be loaded as root using the command insmod sbejegyzes.ko

    [root@matrica sajat_bejegyzes__modul]# insmod
                    sbejegyzes.ko
  4. We can see your entries under the /proc/sajat. Let's try it. Enter the command more /proc/sajat/taszk_stat

    [root@matrica sajat_bejegyzes__modul]# more
                    /proc/sajat/taszk_stat sajat taszk lista (negyedik kernel
                    modul, PROP konyv) # CMD PID FLAGS ST1 ST2 1 systemd 1 4202752
                    1 S 2 kthreadd 2 2149613632 1 S 3 ksoftirqd/0 3 2216722496 1 S
                    4 migration/0 6 2216722496 1 S 5 watchdog/0 7 2216722752 1 S 6
                    migration/1 8 2216722496 1 S 7 ksoftirqd/1 10 2216722496 1 S 8
                    watchdog/1 12 2216722752 1 S 9 migration/2 13 2216722496 1 S
                    10 ksoftirqd/2 15 2216722496 1 S 11 watchdog/2 16 2216722752 1
                    S 12 migration/3 17 2216722496 1 S 13 ksoftirqd/3 19
                    2216722496 1 S 14 watchdog/3 20 2216722752 1 S 15 cpuset 21
                    2216722496 1 S 16 khelper 22 2216722496 1 S 17 kdevtmpfs 23
                    2149613888 1 S 18 netns 24 2216722496 1 S 19 sync_supers 25
                    2149613632 1 S 20 bdi-default 26 2157969472 1 S 21 kintegrityd
                    27 2216722496 1 S 22 kblockd 28 2216722496 1 S 23 ata_sff 29
                    2216722496 1 S 24 khubd 30 2149580864 1 S ...

Example 3.1. The contents of the PCB

The Linux PCB is defined in the struct task_struct structure in the include/linux/sched.h source file. Investigate the elements of the PCB and add further columns to /proc/sajat/taszk_stat.


„Celebrating” the source code

The essential part of the work of the module is done by taszk_lista function, in which the variable i counts the processes (PCBs) in the system and a struct task_struct * pointer task is used to iterate through the processes. To be more precise a struct list_head * pointer p is used because the next and back links of the circular doubly linked list of PCBs are organised around of the structure struct list_head as shown in the next figure.

Figure 3.11. The circular doubly linked list of PCBs in Linux.

The circular doubly linked list of PCBs in Linux.


Handling linked list in the kernel

The macros in linux/list.h can be used to manage the linked list of PCBs. The link member of the PCB list is a list_head structure member that is defined in linux/types.h.

struct list_head { struct list_head
              *next, *prev; };

the usage of these link members were shown in the previous figure. These elements are part of the PCB as it can be seen in the next code snippet cited from the code of PCB from linux/sched.h.

struct task_struct { volatile long
              state; /* -1 unrunnable, 0 runnable, >0 stopped */ ... struct
              list_head tasks;

We have used the

list_for_each(pos,
            head)

macro defined in linux/list.h:

 /** * list_for_each -
            iterate over a list * @pos: the &struct list_head to use as a
            loop cursor. * @head: the head for your list. */ #define
            list_for_each(pos, head) \ for (pos = (head)->next; pos !=
            (head); pos = pos->next) 

The

list_for_each(pos, head)

iterates a struct list_head * pointer pos that is passed as the first parameter through the list from the element head that is passed as the second parameter.

The current macro

It should be noted that in the code snippet

 list_for_each (p,
              current->tasks.next) { 

we have used the current macro that points to the process that is being executed. What is the current macro exactly? Its definition can be found in asm-generic/current.h

 #define get_current()
              (current_thread_info()->task) #define current get_current()
              

The current_thread_info function is defined in (arch/x86/include/)asm/thread_info.h.

 #ifndef __ASSEMBLY__ /* how to get the
              current stack pointer from C */ register unsigned long
              current_stack_pointer asm("esp") __used; /* how to get the
              thread information struct from C */ static inline struct
              thread_info *current_thread_info(void) { return (struct
              thread_info *) (current_stack_pointer & ~(THREAD_SIZE - 1));
              } #else /* !__ASSEMBLY__ */ /* how to get the thread information
              struct from ASM */ #define GET_THREAD_INFO(reg) \ movl
              $-THREAD_SIZE, reg; \ andl %esp, reg 

in the kernel, the thread_union union is on the bottom of the memory area of a process. If we filled the entire stack, then the stack pointer would point to the bottom of the stack and the memory area of thread_union would be overwritten. This union is 8 KB, so we can easily calculate the address of struct thread_info of the running process. The stack pointer's low 25 bits must be set to zero. It can be seen in the following code snippet from the (arch/x86/include/)asm/thread_info.h: current_stack_pointer & ~(THREAD_SIZE - 1), where the value of THREAD_SIZE is equal 8 KB

 #define THREAD_SIZE 8192
              //(2*PAGE_SIZE) 

as can be seen in the file arch/xtensa/include/)asm/thread_info.h. The address of the thread_union and the address of the thread_info are the same:

 union thread_union { struct
              thread_info thread_info; unsigned long
              stack[THREAD_SIZE/sizeof(long)]; }; 

and the task member of the struct thread_info points to PCB of the process.

 struct thread_info { struct task_struct
              *task; /* main task structure */ 

See also the exercise at the end of this section.

Returning to the code snippet of the module to be developed, the list_for_each macro iterates through the list of PCBs, and accesses the PCBs using the

list_entry(ptr, type, member)

macro that defined in the linux/list.h header.

 /** * list_entry - get the
            struct for this entry * @ptr: the &struct list_head pointer. *
            @type: the type of the struct this is embedded in. * @member: the
            name of the list_struct within the struct. */
            

The printed debug information also includes a status character indicating the state of the given process. These possible characters are collected in the following macro that is defined in linux/sched.h.

            #define TASK_STATE_TO_CHAR_STR "RSDTtZXxKW" 

 static int taszk_lista (struct
            seq_file *m, void *v) { int i = 0; struct task_struct *task;
            struct list_head *p; // a kernel/sched/core.c-ben latott modon
            iratjuk ki // a betut (szemben a fs/proc/array.c-ban lathatoval)
            // stat_nam[task->state ? __ffs(task->state) + 1 : 0] static
            const char stat_nam[] = TASK_STATE_TO_CHAR_STR; seq_puts (m,
            "sajat taszk lista (negyedik kernel modul, PROP konyv)\n");
            seq_printf (m, "%-9s %-16s %-6s %-12s %-6s %-3s\n", "#", "CMD",
            "PID", "FLAGS", "ST1", "ST2"); list_for_each (p,
            current->tasks.next) { task = list_entry (p, struct
            task_struct, tasks); seq_printf (m, "%-9i %-16s %-6i %-12u %-6li
            %-3c\n", ++i, task->comm, task->pid, task->flags,
            task->state, stat_nam[task->state ? __ffs (task->state) +
            1 : 0]); } return 0; } 

Making entries under the /proc is done by using the next two functions defined in linux/proc_fs.h

extern struct proc_dir_entry
            *create_proc_entry(const char *name, umode_t mode, struct
            proc_dir_entry *parent); extern struct proc_dir_entry
            *proc_mkdir(const char *,struct proc_dir_entry
            *);

of which the first is the more interesting one

if ((sajat_proc_fajl =
            create_proc_entry ("taszk_stat", S_IFREG | S_IRUGO,
            sajat_proc)))

For example, according the linux/stat.h, the S_IRUGO

 #define
            S_IRUGO (S_IRUSR|S_IRGRP|S_IROTH) 

gives read permission to user (USR), group (GRP) and other (OTH).

 # ls -l /proc/sajat/taszk_stat -r--r--r--. 1 root root 0 Aug 10
            17:42 /proc/sajat/taszk_stat 

Example 3.2. Playing the game with the current macro

Modify the third example module (that can be found in the virtualized system) to print the address pointed by the current macro.

#include <linux/init.h>
              #include <linux/module.h> #include <linux/kernel.h>
              #include <linux/sched.h> #include <linux/list.h>
              #include <linux/thread_info.h> MODULE_DESCRIPTION ("Ez a
              harmadik kernel modulom - modositasa"); MODULE_AUTHOR ("Bátfai
              Norbert (nbatfai@gmail.com)"); MODULE_LICENSE ("GPL"); static
              int current_and_sp (void) { struct task_struct *task; struct
              list_head *p; struct thread_info *ti; register unsigned long esp
              asm ("esp"); ti = (struct thread_info *) (esp & ~(sizeof
              (union thread_union) - 1)); list_for_each (p,
              current->tasks.next) { task = list_entry (p, struct
              task_struct, tasks); printk (KERN_NOTICE "nORBi a kernelben:
              %p\n", ti->task); printk (KERN_NOTICE "nORBi a kernelben,
              current: %p\n", current); printk (KERN_NOTICE "%s %i %u %li\n",
              task->comm, task->pid, task->flags, task->state); }
              return 0; } static int harmadik_init_module (void) { return
              current_and_sp (); } static void harmadik_exit_module (void) {
              current_and_sp (); } module_init (harmadik_init_module);
              module_exit (harmadik_exit_module);

After compiling and loading/removing the module we can see the following in the kernel logs.

[ 8379.261436] nORBi a kernelben: f0f49920 [
              8379.261438] nORBi a kernelben, current: f0f49920 [ 8379.261440]
              systemd 1 4202752 1 [ 8379.261441] nORBi a kernelben: f0f49920 [
              8379.261442] nORBi a kernelben, current: f0f49920 [ 8379.261443]
              kthreadd 2 2129984 1 [ 8379.261444] nORBi a kernelben: f0f49920
              [ 8379.261444] nORBi a kernelben, current: f0f49920 [
              8379.261445] ksoftirqd/0 3 69238848 1 [ 8379.261446] nORBi a
              kernelben: f0f49920 . . . [ 8379.261848] nORBi a kernelben:
              f0f49920 [ 8379.261849] nORBi a kernelben, current: f0f49920 [
              8379.261850] kworker/0:1 2486 69238880 1 [ 8379.261851] nORBi a
              kernelben: f0f49920 [ 8379.261852] nORBi a kernelben, current:
              f0f49920 [ 8379.261853] flush-253:1 3846 10485824 1 [
              8379.261853] nORBi a kernelben: f0f49920 [ 8379.261854] nORBi a
              kernelben, current: f0f49920 [ 8379.261855] kworker/1:1 6477
              69238880 1 [ 8379.261856] nORBi a kernelben: f0f49920 [
              8379.261857] nORBi a kernelben, current: f0f49920 [ 8379.261858]
              insmod 6913 4202752 0 [ 8393.314942] nORBi a kernelben: f0d28000
              [ 8393.314945] nORBi a kernelben, current: f0d28000 [
              8393.314947] systemd 1 4202752 1 [ 8393.314948] nORBi a
              kernelben: f0d28000 [ 8393.314949] nORBi a kernelben, current:
              f0d28000 [ 8393.314951] kthreadd 2 2129984 1 [ 8393.314952]
              nORBi a kernelben: f0d28000 [ 8393.314953] nORBi a kernelben,
              current: f0d28000 [ 8393.314955] ksoftirqd/0 3 69238848 1 [
              8393.314956] nORBi a kernelben: f0d28000 . . . [ 8393.315565]
              nORBi a kernelben: f0d28000 [ 8393.315566] nORBi a kernelben,
              current: f0d28000 [ 8393.315567] bash 1733 4202496 1 [
              8393.315569] nORBi a kernelben: f0d28000 [ 8393.315570] nORBi a
              kernelben, current: f0d28000 [ 8393.315571] kworker/0:1 2486
              69238880 1 [ 8393.315572] nORBi a kernelben: f0d28000 [
              8393.315573] nORBi a kernelben, current: f0d28000 [ 8393.315574]
              flush-253:1 3846 10485824 1 [ 8393.315575] nORBi a kernelben:
              f0d28000 [ 8393.315576] nORBi a kernelben, current: f0d28000 [
              8393.315578] kworker/1:1 6477 69238880 1 [ 8393.315579] nORBi a
              kernelben: f0d28000 [ 8393.315580] nORBi a kernelben, current:
              f0d28000 [ 8393.315582] rmmod 6915 4202752 0

Example 3.3. Drawing a memory map

Draw a sketch of the memory to show how to compute the value of the current macro. See also the Figure 3.2 of the book [ULK], but use the number of the previous exercise.