Chapter 1. Introduction

This document describes the SCSI subsystem as the Linux kernel enters the 2.4 production series.

An external view of the SCSI subsystem is the main theme. Material is included to help the system administration of the Linux SCSI subsystem. There are also brief descriptions of ioctl()s and interfaces that may be relevant to those writing applications that use this subsystem. However internal data structures and design issues are not addressed [see reference W2]. To unclutter the presentation, compile options and system calls (including ioctl()s) have been placed in Appendix E. Although not strictly part of the SCSI subsystem, there is also a description of raw devices in Chapter 11.

For those who have no interest in the SCSI subsystem and just want to get their ATAPI cd writer going, see Section 9.2.4. It may also be useful to browse Chapter 2.

This document follows on from one written five years ago by Drew Eckhardt called the SCSI-HOWTO [see reference W7]. That document described the SCSI subsystem in Linux kernel 1.2 and 1.3 series. It is still available from the Linux Documentation Project [LDP, see reference W8] in its "unmaintained" section. Both documents have roughly similar structures although Drew's document has a lot of information on the adapter drivers.

This document can be found in electronic form at www.tldp.org/HOWTO/SCSI-2.4-HOWTO. The home site and perhaps the most up to date version of this document can be found at www.torque.net/scsi/SCSI-2.4-HOWTO (this is the multi-page html version). At that location this document is rendered in txt, pdf, ps, a single (long) page of html as well as multi-page html. For example, a pdf version is at www.torque.net/scsi/SCSI-2.4-HOWTO.pdf).

This document was last altered on 24th August 2004.

Chapter 2. Architectural Overview

The SCSI subsystem has a 3 level architecture with the "upper" level being closest to the user/kernel interface while the "lower" level is closest to the hardware. The upper level drivers are commonly known by a terse two letter abbreviation (e.g. "sd" for SCSI disk driver). The names of the corresponding module drivers which, for historical reasons, sometimes differ from the built in driver names are shown in braces in the following diagram.

The upper level supports the user-kernel interface. In the case of sd and sr this is a block device interface while for st and sg this is a character device interface. Any operation using the SCSI subsystem (e.g. reading a sector from a disk) involves one driver at each of the 3 levels (e.g. sd, SCSI mid level and aic7xxx drivers).

As can be seen from the diagram, the SCSI mid level is common to all operations. The SCSI mid level defines internal interfaces and provides common services to the upper and lower level drivers. Ioctls provided by the mid level are available to the file descriptors belonging to any of the 4 upper level drivers.

The most common operation on a block device is to "mount" a file system. For a sd device typically a partition is mounted (e.g. mount -t ext2 /dev/sda6 /home). For a sr device usually the whole device is mounted (e.g. mount -t iso9660 /dev/sr0 /mnt/cdrom). The dd command can be used to read or write from block devices. In this case the block size argument ("bs") needs to be set to the block size of the device (e.g. 512 bytes for most disks) or an integral multiple of that device block size (e.g. 8192 bytes). A recent addition to the block subsystem allows a device (or partition) to be mounted more than once, at different mount points.

Sd is a member of the generic disk family, as is the hd device from the IDE subsystem. Apart from mounting sd devices, the fdisk command is available to view or modify a disk's partition table. Although the hdparm command is primarily intended for ATA disks (also known as IDE or EIDE disks), some options work on SCSI disks.

Sr is a member of the CD-ROM subsystem. Apart from mounting file systems (e.g. iso9660), audio CDs can also be read. The latter action does not involve mounting a file system but typically by invoking some ioctls. General purpose Linux commands such as dd cannot be used on audio CDs.

St is a char device for reading and writing tapes. Typically the mt command is used to perform data transfers and other control functions.

Sg is a SCSI command pass through device that uses a char device interface. General purpose Linux commands should not be used on sg devices. Applications such as SANE (for scanners), cdrecord and cdrdao (for cd writers) and cdparanoia (for reading audio CDs digitally) use sg.

Chapter 3. Names and Addresses

This section covers the various naming schemes that exist in Linux and the SCSI worlds and how they interact.

Chapter 4. Kernel Configuration

The Linux kernel configuration is usually found in the kernel source in the file: /usr/src/linux/.config . It is not recommended to edit this file directly but to use one of these configuration options:

• make config - starts a character based questions and answer session

• make menuconfig - starts a terminal-oriented configuration tool (using ncurses)

• make xconfig - starts a X based configuration tool

The descriptions of these selections that is displayed by the associated help button can be found in the flat ASCII file: /usr/src/linux/Documentation/Configure.help

Ultimately these configuration tools edit the .config file. An option will either indicate some driver is built into the kernel ("=y") or will be built as a module ("=m") or is not selected. The unselected state can either be indicated by a line starting with "#" (e.g. "# CONFIG_SCSI is not set") or by the absence of the relevant line from the .config file.

The 3 states of the main selection option for the SCSI subsystem (which actually selects the SCSI mid level driver) follow. Only one of these should appear in an actual .config file:

CONFIG_SCSI=y CONFIG_SCSI=m # CONFIG_SCSI is not set

Some other common SCSI configuration options are:

CONFIG_BLK_DEV_SD [disk (sd) driver] CONFIG_SD_EXTRA_DEVS [extra slots for disks added later] CONFIG_BLK_DEV_SR [SCSI cdrom (sr) driver] CONFIG_BLK_DEV_SR_VENDOR [allow vendor specific cdrom commands] CONFIG_SR_EXTRA_DEVS [extra slots for cdroms added later] CONFIG_CHR_DEV_ST [tape (st) driver] CONFIG_CHR_DEV_OSST [OnSteam tape (osst) driver] CONFIG_CHR_DEV_SG [SCSI generic (sg) driver] CONFIG_DEBUG_QUEUES [for debugging multiple queues] CONFIG_SCSI_MULTI_LUN [allow probes above lun 0] CONFIG_SCSI_CONSTANTS [symbolic decode of SCSI errors] CONFIG_SCSI_LOGGING [allow logging to be runtime selected] CONFIG_SCSI_<ll_driver> [numerous lower level adapter drivers] CONFIG_SCSI_DEBUG [lower level driver for debugging] CONFIG_SCSI_PPA [older parallel port zip drives] CONFIG_SCSI_IMM [newer parallel port zip drives] CONFIG_BLK_DEV_IDESCSI [ide-scsi pseudo adapter] CONFIG_I2O_SCSI [scsi command set over i2o bus] CONFIG_SCSI_PCMCIA [for SCSI HBAs on PCMCIA bus] CONFIG_USB_STORAGE [usb "mass storage" type] CONFIG_MAGIC_SYSRQ [Alt+SysRq+S for emergency sync] [Alt+SyrRq+U for emergency remount ro]

If the root file system is on a SCSI disk then it makes sense to build into the kernel the SCSI mid level, the sd driver and the host adapter driver that the disk is connected to. In most cases it is usually safe to build the sr, st and sg drivers as modules so that they are loaded as required. If a device like a scanner is on a separate adapter then its driver may well be built as a module. In this case, that adapter driver will need to be loaded before the scanner will be recognized.

Linux distributions have many of the SCSI subsystem drivers built as modules since building all of them in would lead to a very large kernel that would exceed the capabilities of the boot loader. This leads to a "chicken and the egg" problem in which the SCSI drivers are needed to load the root file system and vice versa. The 2 phase load used by the initrd device addresses this problem (see Chapter 6 for more details).

Chapter 5. Boot Parameters

On a PC the motherboard's BIOS together with the SCSI BIOS provided by most SCSI host adapters takes care of the problem of loading the boot loader's image from a SCSI disk into memory and executing it. This may require some settings to be changed in the motherboard's BIOS. When more than one SCSI adapter is involved, the SCSI BIOS settings may need to change to indicate which one contains the disk with the boot image. The boot image make also come from an ATA (IDE) disk, a bootable CD-ROM or a floppy.

Both lilo and grub are commonly used boot loaders with Linux. Their configuration files are in /etc/lilo.conf and /etc/grub.conf [3] respectively. One difference is that after changing lilo's configuration the lilo command must be executed for the changes to take effect (and there is no equivalent requirement for grub). See their "man" pages for usage information. An excellent paper on lilo and the Linux bootup sequence can be found ftp://icaftp.epfl.ch/pub/people/almesber/booting/bootinglinux-0.ps.gz. For further information on grub see www.gnu.org/software/grub.

Some boot parameters related to the SCSI subsystem:

single [enter single user mode] <n> [enter run level <n> {0..6}] root=/dev/sda6 [*] root=/dev/scsi/host0/bus0/target0/lun0/part6 [*] root=/dev/sd/c0b0t0u0p6 [*] devfs=mount [overrides CONFIG_DEVFS_MOUNT=n] devfs=nomount [overrides CONFIG_DEVFS_MOUNT=y] init=<command> [executes <command> rather than init] quiet [reduce output to console during boot] debug [increase output to console during boot] nmi_watchdog=0 [turn off NMI watchdog on a SMP machine] max_scsi_luns=1 [limits SCSI bus scans to lun==0] scsi_allow_ghost_devices=<n>

* When devfs is in use the initial read-only mount of the root partition can be done via the old /dev/sd<a><n> notation or the new devfs notation (and two of these are shown). The joint "root=/dev/sda6 single" may be useful when disk or adapter changes have broken the kernel boot load.

The "root=" argument may also be a hex number. For example, if the root partition is on /dev/sda3 then "root=803" is appropriate. The last two digits are the minor device number discussed in an earlier section.

The default argument to the "init" parameter is /sbin/init (see man (8) init). If files such as /etc/fstab have incorrect entries, it may be useful to drop directly into a shell with "init=/bin/bash". However if shared libraries files or their paths are inappropriate this may also fail. That leaves "init=/sbin/sash" which is a statically linked shell with many useful commands (for repairing a system) built in (see man (8) sash).

When Linux fails to boot after reporting a message like:

VFS: Cannot open root device 08:02

then the kernel expected to find a root partition on device /dev/sda2 and did not. The numbers in the error message are major and minor device numbers (in hex) [see Section 3.2 for the mapping to device names]. In such situations the "root" boot option can be useful (also the rdev command can be used to modify where the boot image looks for the root partition).

Lilo's configuration file /etc/lilo.conf can take the "root=" option in two ways. The normal way is a line like: 'root=/dev/sda2'. In this case /dev/sda2 is converted into major and minor numbers based on the state of the system when the lilo command is executed. This can be a nuisance, especially if hardware is going to be re-arranged. The other way is a line of the form: 'append="root=/dev/sda2"' In this case the /dev/sda2 is passed through to the kernel the next time it is started. This is the same as giving the "root=/dev/sda2" string at the kernel boot time prompt. It is interpreted by the kernel at startup (once the HBAs and their attached devices have been recognized) and thus is more flexible.

Chapter 6. Modules and their Parameters

There are many SCSI related modules. The mid and upper level modules are listed below:

• scsi_mod.o

• sd_mod.o

• sr_mod.o

• st.o [osst.o]

• sg.o

Notice that the first 3 have "_mod" appended to their normal driver names. Lower level drivers tend to use the name (or an abbreviation) of the HBA's manufacturer (e.g. advansys) plus optionally the chip number of the major controller chip (e.g. sym53c8xx for symbios controllers based on the NCR 53c8?? family of chips).

All SCSI modules depend on the mid level. This means if the SCSI mid level is not built into the kernel and if scsi_mod.o has not already been loaded then a command like modprobe st will cause the scsi_mod.o module to be loaded. There could well be other dependencies, for example modprobe sr_mod will also cause the cdrom module to be loaded if it hasn't been already. Also if the SCSI mid level is a module, then all other SCSI subsystem drivers must be modules (this is enforced by the kernel build configuration tools).

Modules can be loaded with the modprobe <module_name> command which will try to load any modules that the nominated <module_name> depends on. Also <module_name> does not need the trailing ".o" extension which is assumed if not given. The insmod <module_name> command will also try and load <module_name> but without first loading modules it depends on. Rules for how modules can cause other modules to be loaded (with appropriate parameters appended) are usually placed in the file /etc/modules.conf. [Note that in earlier Linux kernels this file was often called /etc/conf.modules.] For further information about the format of this file try man modules.conf.

Any module can have its allowable command line parameters queried with this command: modinfo -p <module_name>.

When upper level drivers are initialized and if there are no hosts active then the mid level will attempt to load a module called "scsi_hostadapter". An "alias" can then be used to associate "scsi_hostadapter" with the actual name of the lower level (adapter) driver. For example, a line like "alias scsi_hostadapter aic7xxx" in the /etc/modules.conf file would cause the aic7xxx module to be loaded (if there were no lower level drivers already active). [4]

There is a special relationship between the module parameter "scsi_hostadapter" and the initrd file system. For more information see man initrd and man mkinitrd. [5]

Chapter 7. Proc pseudo file system

The proc pseudo file system provides some useful information about the SCSI subsystem. The kernel configuration option that selects "proc_fs" is CONFIG_PROC_FS and in almost all cases it should be selected. SCSI specific information is found under the directory /proc/scsi. Probably the most commonly accessed entry is cat /proc/scsi/scsi which lists the attached SCSI devices. See Section 8.3 for more details.

The lower level drivers are allocated proc_fs entries of the form:

/proc/scsi/<driver_name>/<scsi_adapter_number>

where the <driver_name> is something like "aic7xxx" or "BusLogic". The <scsi_adapter_number> (also known as the host number) is the same number that was discussed in