Friday, October 27, 2017

Oracle Linux - digging into the bioset Kernel working

when you are looking into the processes that are running on your Oracle Linux machine you might notice a process, or processes named bioset. In our example we are running an Oracle Linux 6 machine in a vagrant box, and for informational purposes, we are using it to run dockerd to serve Docker containers for development purposes.

As you can see in the below example, we find a number of bioset processes running on our Oracle Linux machine;

[root@docker ~]# ps -ef|grep bioset
root        30     2  0 12:04 ?        00:00:00 [bioset]
root       299     2  0 12:04 ?        00:00:00 [bioset]
root       302     2  0 12:04 ?        00:00:00 [bioset]
root      4136     2  0 14:20 ?        00:00:00 [bioset]
root      4138     2  0 14:20 ?        00:00:00 [bioset]
root      4140     2  0 14:20 ?        00:00:00 [bioset]
root      4379  3470  0 14:35 pts/0    00:00:00 grep bioset
[root@docker ~]# 

Finding bioset
To find out what bioset is and where it is being spawned from you can look at your processes in a number of ways. You can use the below command which provides a nice tree view of all the processes based upon the ps command or you can use the standard pstree command.

ps axfo pid,euser,egroup,args

using the above command we get the below output (cut down to the relevant part only);

As you can see bioset is a mentioned a number of times, and always as a "child" from kthreadd. Where kthreadd is within Oracle Linux (Linux in general) the kernel thread daemon and by doing so it is one of the backbone components of the Linux kernel which makes sure your kernel threads are handled in the right manner.

root@docker ~]# ps axfo pid,euser,egroup,args
  PID EUSER    EGROUP   COMMAND
    2 root     root     [kthreadd]
    3 root     root      \_ [ksoftirqd/0]
    5 root     root      \_ [kworker/0:0H]
    6 root     root      \_ [kworker/u4:0]
    7 root     root      \_ [rcu_sched]
    8 root     root      \_ [rcu_bh]
    9 root     root      \_ [rcuos/0]
   10 root     root      \_ [rcuob/0]
   11 root     root      \_ [migration/0]
   12 root     root      \_ [watchdog/0]
   13 root     root      \_ [watchdog/1]
   14 root     root      \_ [migration/1]
   15 root     root      \_ [ksoftirqd/1]
   16 root     root      \_ [kworker/1:0]
   17 root     root      \_ [kworker/1:0H]
   18 root     root      \_ [rcuos/1]
   19 root     root      \_ [rcuob/1]
   20 root     root      \_ [khelper]
   21 root     root      \_ [kdevtmpfs]
   22 root     root      \_ [netns]
   23 root     root      \_ [perf]
   24 root     root      \_ [khungtaskd]
   25 root     root      \_ [writeback]
   26 root     root      \_ [ksmd]
   27 root     root      \_ [khugepaged]
   28 root     root      \_ [crypto]
   29 root     root      \_ [kintegrityd]
   30 root     root      \_ [bioset]
   31 root     root      \_ [kblockd]
   32 root     root      \_ [ata_sff]
   33 root     root      \_ [md]
   34 root     root      \_ [kworker/0:1]
   36 root     root      \_ [kswapd0]
   37 root     root      \_ [fsnotify_mark]
   49 root     root      \_ [kthrotld]
   50 root     root      \_ [acpi_thermal_pm]
   51 root     root      \_ [kworker/u4:1]
   52 root     root      \_ [kworker/1:1]
   53 root     root      \_ [kpsmoused]
   54 root     root      \_ [bcache]
   55 root     root      \_ [deferwq]
   56 root     root      \_ [kworker/0:2]
  200 root     root      \_ [scsi_eh_0]
  201 root     root      \_ [scsi_tmf_0]
  202 root     root      \_ [scsi_eh_1]
  203 root     root      \_ [scsi_tmf_1]
  233 root     root      \_ [scsi_eh_2]
  234 root     root      \_ [scsi_tmf_2]
  298 root     root      \_ [kdmflush]
  299 root     root      \_ [bioset]
  301 root     root      \_ [kdmflush]
  302 root     root      \_ [bioset]
  368 root     root      \_ [kworker/0:1H]
  369 root     root      \_ [jbd2/dm-1-8]
  370 root     root      \_ [ext4-rsv-conver]
  626 root     root      \_ [iprt-VBoxWQueue]
  627 root     root      \_ [ttm_swap]
  738 root     root      \_ [jbd2/sda1-8]
  739 root     root      \_ [ext4-rsv-conver]
  827 root     root      \_ [kauditd]
  889 root     root      \_ [ipv6_addrconf]
 3744 root     root      \_ [kworker/1:1H]
 4122 root     root      \_ [loop0]
 4123 root     root      \_ [loop1]
 4126 root     root      \_ [kdmflush]
 4133 root     root      \_ [dm_bufio_cache]
 4136 root     root      \_ [bioset]
 4137 root     root      \_ [kcopyd]
 4138 root     root      \_ [bioset]
 4139 root     root      \_ [dm-thin]
 4140 root     root      \_ [bioset]

However, now we know where bioset is commong from and the parent is the kernel thread daemon we still do not know what bioset is doing.

What is bioset
As we now known bioset is part of the Linux kernel, it is used to Block Input Output instructions. This means it is an integrated part of the inner workings of the kernel when operations to a block device need to be done. bioset finds its origin in the struct bios_set located in bio.h within the Linux kernel code. The bio_set struct is shown below and can also be found in the kernel Github code repository.

/*
 * bio_set is used to allow other portions of the IO system to
 * allocate their own private memory pools for bio and iovec structures.
 * These memory pools in turn all allocate from the bio_slab
 * and the bvec_slabs[].
 */
#define BIO_POOL_SIZE 2

struct bio_set {
 struct kmem_cache *bio_slab;
 unsigned int front_pad;

 mempool_t *bio_pool;
 mempool_t *bvec_pool;
#if defined(CONFIG_BLK_DEV_INTEGRITY)
 mempool_t *bio_integrity_pool;
 mempool_t *bvec_integrity_pool;
#endif

 /*
  * Deadlock avoidance for stacking block drivers: see comments in
  * bio_alloc_bioset() for details
  */
 spinlock_t  rescue_lock;
 struct bio_list  rescue_list;
 struct work_struct rescue_work;
 struct workqueue_struct *rescue_workqueue;
};

If you explore the Linux kernel code you will find that within /block/bio.c will be include /linux/bio.h . The Linux block I/O subsystem includes bioset and this is a vital part of how BIOs are handled within the kernel.

Below images shows the Linux storage stack for kernel version 4.0 where you can see the flow of BIOs. BIOs are send from LIO (Linux I/O) or the VFS to the block I/O layer who in return will communicate with the I/O scheduler who will schedule the I/O operation to be send on the actual block device.



In conclusion
A lot of posts are being written that would indicate that bioset would be something that could be removed or would be malicious for the system, as outlined in the above post bioset is however a vital part of the Linux Kernel and an integrated part of the I/O subsystem for handling I/O requests in combination with block devices.

No comments: