ZFS root (builtin)

From SlackWiki
Revision as of 10:15, 23 August 2012 by Foobarz (talk | contribs)
Jump to navigation Jump to search

The first "ZFS root" wiki explains the details of running ZFS at your root filesystem by using a fully modular generic kernel approach. Now, I can share how it can be done with the SPL and ZFS modules built into the kernel. This procedure is just an example and can use some fine tuning, but here goes:

The steps below are to create kernel with SPL and ZFS modules builtin. This kernel will be installed as an alternatve kernel to boot in lilo, and it will have a separate installation from the fully modular and working system. This will allow testing the builtin kernel while able to boot back onto a working modular ZFS system. We start this procedure assuming you are on a working fully modular ZFS install as in the "ZFS root" wiki.

zfs set mountpoint=legacy zfs-root
# use legacy so zfs will not expect zfs mount, but instead expect standard mount for this fs
# edit /etc/rc.d/rc.S and rc.6 to use regular "mount" commands, remove "zfs" commands
# edit rc.6 and remove or comment out zfs export command
zpool set bootfs=zfs-root zfs-root
# this may help, but not really sure

mkdir /boot/initramfs-source
# this will hold some files for rootfs inside kernel

cd ~
mkdir src
cd src
tar xvzf /mnt/cdrom/slackware64/k/kernel-source-*.txz
mv usr/src/linux-3.2.27 /usr/src/linux-3.2.27b
rm -r install
cd /usr/src/linux-3.2.27b
make menuconfig
  General setup->Local version - append to kernel release = b
  General setup->Default hostname                         = slackzfs
  General setup->Initramfs source files(s)                = /boot/initramfs-source
  #  make usre you made this directory or the kernel build fails
  Device Drivers->SCSI device support->SCSI low-level drivers-> <*> SYM53C8XX Version 2 SCSI support
  #  for qemu if=scsi -option-rom 8xx_64.rom,bootindex=1  hard disks
  # buildin any hard drive controllers etc that you need
  File systems -> <*> The Extended 4 (ext4) filesystem
  # /boot may use this ext4 fs
make prepare scripts
# this make command is what the spl and zfs copy-builtin scripts expect to be done before they are run

cd ~/src
tar xvzf ~/spl-0.6.0-rc10.tar.gz
mkdir install
cd spl-0.6.0-rc10
./configure \
    --prefix=/ \
    --libdir=/lib64 \
    --includedir=/usr/include \
    --datarootdir=/usr/share \
    --enable-linux-builtin=yes \
    --with-linux=/usr/src/linux-3.2.27b \
    --with-linux-obj=/usr/src/linux-3.2.27b
wget https://raw.github.com/zfsonlinux/spl/master/copy-builtin
chmod +x copy-builtin
./copy-builtin /usr/src/linux-3.2.27b
make
make install DESTDIR=~/src/install
cd ~/src/install
makepkg ../spl-0.6.0rc10_3.2.27b-x86_64-1root.txz
cd ..
rm -r install

tar xvzf ~/zfs-0.6.0-rc10.tar.gz
mkdir install
cd spl-0.6.0-rc10
./configure \
    --prefix=/ \
    --libdir=/lib64 \
    --includedir=/usr/include \
    --datarootdir=/usr/share \
    --enable-linux-builtin=yes \
    --with-linux=/usr/src/linux-3.2.27b \
    --with-linux-obj=/usr/src/linux-3.2.27b \
    --with-spl=/root/src/spl-0.6.0-rc10
wget https://raw.github.com/zfsonlinux/zfs/master/copy-builtin
chmod +x copy-builtin
./copy-builtin /usr/src/linux-3.2.27b
make
make install DESTDIR=~/src/install
cd ~/src/install
makepkg ../zfs-0.6.0rc10_3.2.27b-x86_64-1root.txz
cd ..
rm -r install

### move zfs and spl modules inside kernel source to be at end of drivers:
### the order builtin modules init is the order they link into the kernel
### and we need zfs to init after all hard drive controllers
### zfs is more like a device driver layer over the lower-level hba drivers

cd /usr/src/linux-3.2.27b
mkdir drivers/zfsonlinux
mv spl drivers/zfsonlinux
vi Kconfig
  # remove references to spl
vi Makefile
  # remove references to spl

cd /usr/src/linux-3.2.27b/fs
mv zfs ../drivers/zfsonlinux
vi Kconfig
  # remove references to zfs
vi Makefile
  # remove references to zfs

cd /usr/src/linux-3.2.27b/drivers
vi Kconfig
  )# add line at end of menu, before "endmenu":
  )source "drivers/zfsonlinux/Kconfig"
  )endmenu

cd /usr/src/linux-3.2.27b/drivers
vi Makefile
  )# add line at very end of file:
  )obj-$(CONFIG_ZFSONLINUX) += zfsonlinux/

cd /usr/src/linux-3.2.27b/drivers/zfsonlinux
cat > Kconfig <<"EOF"
menuconfig ZFSONLINUX
	tristate "ZFSonLinux support"

if ZFSONLINUX

source "drivers/zfsonlinux/spl/Kconifg"

source "drivers/zfsonlinux/zfs/Kconifg"

endif
EOF

cd /usr/src/linux-3.2.27b/zfsonlinux
cat > Makefile <<"EOF"
obj-$(CONFIG_SPL) += spl/
obj-$(CONFIG_ZFS) += zfs/
EOF

### move complete

cd /usr/src/linux-3.2.27b
make menuconfig
   Device Drivers ->
    <*> ZFSonLinux support ->
      <*> Solaris Porting Layer (SPL)
      <*>     ZFS


### prepare contents of initramfs
cd /boot/initramfs-source

# make standard directories
mkdir -p proc dev sys mnt bin sbin etc/zfs

# zfs seems to want mtab present, even if empty
touch etc/mtab

# if zpool.cache file can be read by zfs at module init, it imports the pools in the cache
cp /etc/zfs/zpool.cache-initrd etc/zfs/zpool.cache

# make initial console device node; otherwise, there is a problem at startup:
# "Warning: unable to open an initial console" and you'd have problems with console and tty login
# making these nodes from within /init is not early enough to avoid problem
mknod dev/console c 5 1   # system console

# make memory device kmsg to "printk" kernel messages
# we can write to this file to send out messages
mknod dev/kmsg    c 1 11  # lines printed to kmsg enter kernel messages buffer

# recommended loop0 device to mount fs images
mknod dev/loop0   b 7 0

# make initial virtual terminal devices
mknod dev/tty     c 5 0   # current tty
mknod dev/tty0    c 4 0   # current virtual term
mknod dev/tty1    c 4 1   # login virtual term 1 (F1)

# make alternative console=ttyS0 standard 8250/16550 UART serial port devices
#   useful with kernel parameter console=ttyS0
#          with qemu -nographic -option-rom sgabios.bin,bootindex=0
mknod dev/ttyS0   c 4 64  # COM1
mknod dev/ttyS1   c 4 65  # COM2
mknod dev/ttyS2   c 4 66  # COM3
mknod dev/ttyS3   c 4 67  $ COM4

# this should be enough device nodes initially
# once devtmpfs mounts over /dev, a lot more is available in it by default

cat > /init <<"EOF" <some initial program PID=1 for the system to start on> EOF
# an initramfs-source/init program is required to mount and boot a root linux system
# the common way to make /init is using busybox with files like this:
#    /bin/busybox
#    /bin/ash -> busybox
#    /bin/sh -> busybox
#    # run busybox to see what other "applets" can be symlinked to it
#    # when busybox is run as a different symlinked applet name, it runs the applet
#    /init
#    # init is ash script, see slackware's /boot/initrd-tree/init for example
#    Inside this script:
#      1) parse kernel parameters passed into it and set variables based on them
#      2) start udevd and trigger rules for block devices to setup initial /dev devices
#      3) load kernel modules and keyboard map
#      4) run mdadm, cryptsetup, lvm, zpool/zfs to setup more devices
#         (any udev rules for them should do more /dev setup in the background)
#      5) mount the root filesystem read-only at /mnt
#      6) stat /mnt/sbin/init, and if not executable run a rescue shell /bin/sh
#      7) shutdown udevd so it can be restarted by root system when booted
#      8) mount --move {proc,sys,run,dev} to under /mnt
#      9) run: exec switch_root /mnt /sbin/init $RUNLEVEL
#    The slackware mkinitrd package is fully configured busybox installation
#      for an extern file initramfs initrd.gz image outside the kernel.
#    It could be changed to be inside the kernel easily by just using initrd-tree
#      as target of INITRAMFS_SOURCE in the kernel config and removing initrd in lilo.

# Because the zfs modules are builtin, we really don't need busybox and the whole
# mkinitrd package. All we need to do is mount the zfs that is already imported
# because the zpool.cache is present in rootfs when the zfs module initializes, and
# then we do the switch root procedure to boot up on the mounted system.
# udevd rules will still be run when the root system starts udevd.

# So, let's make a simple /init c program that will be all that we really need:
cat > init.c <<"EOF"
/* software product name: foobarz-init.c
 * suggested binary name: /init (in initramfs-source, rootfs)
 * license              : BSD
 * license text:
Copyright (c) 2012, foobarz
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
    * Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
    * Neither the name of the <organization> nor the
      names of its contributors may be used to endorse or promote products
      derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL <COPYRIGHT HOLDER> BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

#define FOOBARZ_INIT_VERSION "1.0.0"
#define _BSD_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/mount.h>
#include <sysexits.h>
#include <errno.h>
#include <string.h>

/* Your initramfs-source should contain the following
 * cd /boot/initramfs-source
 * mkdir -p proc dev sys mnt bin sbin etc/zfs
 * touch etc/mtab
 * cp /etc/zfs/zpool.cache-initrd etc/zfs/zpool.cache
 * mknod dev/console c 5 1   # system console
 * mknod dev/kmsg    c 1 11  # lines printed to kmsg enter kernel messages buffer
 * mknod dev/loop0   b 7 0
 * mknod dev/tty     c 5 0   # current tty
 * mknod dev/tty0    c 4 0   # current virtual term
 * mknod dev/tty1    c 4 1   # login virtual term 1 (F1)
 * mknod dev/ttyS0   c 4 64  # COM1
 * mknod dev/ttyS1   c 4 65  # COM2
 * mknod dev/ttyS2   c 4 66  # COM3
 * mknod dev/ttyS3   c 4 67  $ COM4
 */

int main(int argc, char* argv[]) {
 /***** variables
  *
  */
 int i;
 FILE* kmsg = NULL;
 int   fd = 0; /* file descriptor */
 unsigned long mountflags;

 /* kernel command line */
 off_t cmdline_size;
 char* cmdline; /* to be malloc 4096B */
 char* cmdline_end;
 char* temp_end;
 int usage = EX_OK;

 /* use to hold contents of a misc /proc/<file> */
 char* miscproc_buff; /* to be malloc 4096B */
 off_t miscproc_size;

 /* note about environ, argv, and kernel cmdline for init:
  *   environ is not defined for init
  *   only argv[0] is set for init
  *   kernel command line parameters are accessed
  *   at /proc/cmdline
  */

 /* kernel parameters expected to be name=value */
 /* do not use quotes or spaces in parameters   */
 /* you can add more params somwhere after root= */
 struct nv { char* n; char* v; char* v_end; int req; };
 struct nv param[] = {
   { "root=", NULL, NULL, 1 },      /* required; root device to boot */
   { "rootfs=", NULL, NULL, 1 },    /* required; root filesystem type */
   { "mountopt=", NULL, NULL, 0 },  /* optional; mount option ro or rw; default ro */
   { "init=", NULL, NULL, 0 },      /* optional; alt. init prgm; default /sbin/init */
   { "runlevel=", NULL, NULL, 0 },  /* optional; runlevel 0123456Ss; default 3 multiuser */
   { "console=", NULL, NULL, 0 }          /* optional; alt. console; e.g. ttyS0 */
 };
 enum { iroot, irootfs, imountopt, iinit, irunlevel, iconsole, ilastparam };

 /***** program
  *
  */
 kmsg = fopen("/dev/kmsg", "w");
 fprintf(kmsg, "foobarz-init, version %s: booting initramfs.\n", FOOBARZ_INIT_VERSION);

 cmdline = (char*) malloc(4096);
 miscproc_buff = (char*) malloc(4096);
 if( (cmdline == NULL) || (miscproc_buff == NULL) ) {
   fprintf(kmsg, "Unable to allocate buffer memory: malloc: %s\n", strerror(errno));
   return EX_UNAVAILABLE;
 }

 /* mount proc before dev since some devices
  * symlink into proc */
 fprintf(kmsg, "Attempting mount proc /proc\n");
 if( mount("proc", "/proc", "proc", 0, NULL) != 0 ) {
  fprintf(kmsg, "time to panic: mount: %s\n", strerror(errno));
  return EX_UNAVAILABLE;
 } else {
  fprintf(kmsg, "Mount proc successful.\n");
 }

 /* mount devtmpfs as expected by an init program
  * and maybe required to mount zfs */
 fprintf(kmsg, "Attempting mount devtmpfs /dev\n");
 if( mount("devtmpfs", "/dev", "devtmpfs", 0, NULL) != 0 ) {
  fprintf(kmsg, "time to panic: mount: %s\n", strerror(errno));
  return EX_UNAVAILABLE;
 } else {
  fprintf(kmsg, "Mount devtmpfs successful.\n");
 }
 /* it is assumed that all required devices are in devtmpfs
  * and if not, there will be boot problems
  *
  * if you need a more complex device setup, then you might need
  * udevd and run it with a larger full initrd package from your distro
  */

 /* process kernel command line */
 fd = open("/proc/cmdline", O_RDONLY);
 if( fd == -1 ) {
   fprintf(kmsg, "Cannot open /proc/cmdline: %s\n", strerror(errno));
   return EX_UNAVAILABLE;
 }
 /* note, on /proc fs:
  *       lseek likely always returns error
  *       stat likely always returns st_size = 0
  *   so determining size of /proc file means just reading it;
  *   you have to read /proc files according to their documented
  *   maximum sizes; this is probably for performance reasons */
 cmdline_size = read(fd, cmdline, 4095);
 if( cmdline_size == -1 ) {
   fprintf(kmsg, "Failed to read /proc/cmdline: %s\n", strerror(errno));
   return EX_UNAVAILABLE;
 }
 close(fd);

 /* cmdline may be newline + null terminated, but make it null + null */
 cmdline[cmdline_size] = '\0';
 if( cmdline[cmdline_size-1] == '\n' ) {
   cmdline[cmdline_size-1] = '\0';
   cmdline_size--;
   cmdline_end = cmdline + cmdline_size;
 }
 fprintf(kmsg, "Kernel cmdline size: %i\n", cmdline_size);
 fprintf(kmsg, "Kernel cmdline: \"%s\"\n", cmdline);

 for( i=iroot; i<ilastparam; i++ ) {
   param[i].v = strstr(cmdline, param[i].n);
   if( param[i].v != NULL ) {
     while( *(param[i].v) != '=' ) param[i].v++;
     param[i].v++;
     temp_end = param[i].v;
     while( !( (*temp_end == ' ') ||
               (*temp_end == '\n') ||
               (temp_end == cmdline_end)
	     ) ) temp_end++;
     if( temp_end == param[i].v ) {
       fprintf(kmsg, "Kernel parameter %s: value missing.\n", param[i].n);
     } else param[i].v_end = temp_end;
   }
 }

 /* set defaults on non-required params, except for console */
 if( param[imountopt].v == NULL ) param[imountopt].v = "ro";
 if( param[iinit].v == NULL )     param[iinit].v = "/sbin/init";
 if( param[irunlevel].v == NULL ) param[irunlevel].v = "3";

 for( i=iroot; i<ilastparam; i++ ) {
   if( param[i].v_end != NULL ) *(param[i].v_end) = '\0';
   fprintf(kmsg, "Using %s \"%s\"\n", param[i].n, param[i].v);
 }

 for( i=iroot; i<ilastparam; i++ ) {
   if( (param[i].req == 1) && (param[i].v == NULL) ) {
     printf("Error: missing required kernel parameter: %s\n", param[i].n);
     usage = EX_USAGE;
   }
 }
 if( usage != EX_OK ) {
   printf("Aborting boot process. Missing required kernel parameter(s).\n");
   return usage;
 }
 /* generic nv pair kernel cmdline processing finished
  *  now, examine specific params for defaults and correctness
  */

 /* param[iroot]: nothing to do, if user put bad device in then we fail */

 /* param[irootfs]: can be checked against /proc/filesystems: */
 /* required, so should have value */
 fd = open("/proc/filesystems", O_RDONLY);
 if( fd == -1 ) {
   fprintf(kmsg, "Cannot open /proc/filesystems: %s\n", strerror(errno));
   return EX_UNAVAILABLE;
 }
 miscproc_size = read(fd, miscproc_buff, 4095);
 if( miscproc_size == -1 ) {
   fprintf(kmsg, "Failed to read /proc/filesystems: %s\n", strerror(errno));
   return EX_UNAVAILABLE;
 }
 close(fd);
 if( strstr(miscproc_buff, param[irootfs].v) == NULL ) {
   fprintf(kmsg, "%s \"%s\": filesystem type not available.\n",
	   param[irootfs].n, param[irootfs].v);
   return EX_UNAVAILABLE;
 }

 if( strcmp(param[irootfs].v, "zfs") == 0 ) {
   if( access("/etc/zfs/zpool.cache", F_OK) == 0 )
     fprintf(kmsg, "%s: /etc/zfs/zpool.cache is present in initramfs.\n", param[irootfs].v);
   else
     fprintf(kmsg, "%s: /etc/zfs/zpool.cache not present in initramfs.\n", param[irootfs].v);
   if( access("/etc/hostid", F_OK) == 0 )
     fprintf(kmsg, "%s: /etc/hostid is present in initramfs.\n", param[irootfs].v);
   else
     fprintf(kmsg, "%s: /etc/hostid not present in initramfs.\n", param[irootfs].v);
 }


 if(      strcmp(param[imountopt].v, "ro") == 0 ) mountflags = MS_RDONLY;
 else if( strcmp(param[imountopt].v, "rw") == 0 ) mountflags = 0;
 else {
   fprintf(kmsg, "% \"%\": invalid parameter value;\n"
		 "   use \"ro\" or \"rw\" (no quotes or spaces);\n"
		 "   defaulting to \"ro\".\n",
		 param[imountopt].n, param[imountopt].v
	  );
   mountflags = MS_RDONLY;
 }

 fprintf(kmsg, "Attempting %s %s mount %s /mnt.\n",
	 param[irootfs].v, param[imountopt].v, param[iroot].v);
 if( mount(param[iroot].v, "/mnt", param[irootfs].v, mountflags, NULL) != 0 ) {
  fprintf(kmsg, "time to panic: mount: %s\n", strerror(errno));
  return EX_UNAVAILABLE;
 }
 fprintf(kmsg, "%s mounted successfully.\n", param[iroot].v);

 fprintf(kmsg, "Attempting mount --move /dev /mnt/dev\n");
 if( mount("/dev", "/mnt/dev", NULL, MS_MOVE, NULL) != 0 ) {
  fprintf(kmsg, "time to panic: mount: %s\n", strerror(errno));
  return EX_UNAVAILABLE;
 }

 fprintf(kmsg, "Attempting mount --move /proc /mnt/proc\n");
 if( mount("/proc", "/mnt/proc", NULL, MS_MOVE, NULL) != 0 ) {
  fprintf(kmsg, "time to panic: mount: %s\n", strerror(errno));
  return EX_UNAVAILABLE;
 }

 fprintf(kmsg, "Beginning switch root procedure.\n");
 fprintf(kmsg, "Attempting chdir from / to /mnt\n");
 if( chdir("/mnt") != 0 ) {
  fprintf(kmsg, "time to panic: chdir: %s\n", strerror(errno));
  return EX_UNAVAILABLE;
 }

 fprintf(kmsg, "Attempting mount-move . to /\n");
 if( mount(".", "/", NULL, MS_MOVE, NULL) != 0 ) {
  fprintf(kmsg, "time to panic: mount: %s\n", strerror(errno));
  return EX_UNAVAILABLE;
 }

 fprintf(kmsg, "Attempting chroot to .\n");
 if( chroot(".") != 0 ) {
  fprintf(kmsg, "time to panic: chroot: %s\n", strerror(errno));
  return EX_UNAVAILABLE;
 }

 fprintf(kmsg, "Attempting chdir to /\n");
 if( chdir("/") != 0 ) {
  fprintf(kmsg, "time to panic: chdir: %s\n", strerror(errno));
  return EX_UNAVAILABLE;
 }
 fprintf(kmsg, "Completed switch root procedure.\n");

 /* check for "console=" kernel parameter and switch
  *  stdin, stdout, and stderr to named console device
  */
 if( param[iconsole].v != NULL ) {
   fprintf(kmsg, "Console redirection to device %s requested.\n", param[iconsole].v);
   fprintf(kmsg, "Opening stdin, stdout, and stderr on %s.\n", param[iconsole].v);
   close(0);
   /* expect only basename of device (e.g., ttyS0), so chdir /dev */
   chdir("/dev");
   open(param[iconsole].v, O_RDWR);
   dup2(0, 1);
   dup2(0, 2);
   chdir("/");
 }

 fprintf(kmsg, "Execing: \"%s %s\" to boot mounted root system.\n",
	 param[iinit].v, param[irunlevel].v);

 /* free resources held to this point */
 fclose(kmsg);
 free(cmdline);
 free(miscproc_buff);

 if( execl(param[iinit].v, param[irunlevel].v, (char *) NULL ) != 0 ) {
  kmsg = fopen("/dev/kmsg", "w");
  fprintf(kmsg, "time to panic: execl: %s\n", strerror(errno));
  fclose(kmsg);
  return EX_UNAVAILABLE;
 }
}

EOF
## end of rootfs /init

gcc --static init.c -o init
strip init

## build and install kernel
cd /usr/src/linux-3.2.27b
make -j8
make -j8 modules_install
cp arch/x86/boot/bzImage /boot/vm3.2.27b
cp System.map /boot/System.map-vm3.2.27b
# add new lilo menu entry for vm3.2.27b kernel
vi /etc/lilo.conf
  image = /boot/vm3.2.27b
   label = vm3.2.27b
   addappend = " spl.spl_hostid=0x007f0100 zfs.spa_config_path=/etc/zfs/zpool.cache root=zfs-root ro rootfstype=zfs rootwait "
lilo
reboot
# note: root=, rootfstype, ro, rootwait are "legacy" kernel boot options and are not really being used here
#       so, they probably can be removed but are not hurting

# if you have errors using zfs and zpool commands on booted system with builtin modules, then
# upgrade/switch to the 0.6.0rc10_3.2.27b builds, or you can make a custom package with the binaries
# renamed like zpoolb, zfsb etc for builtin

# if you have boot problem, you might want to run in qemu's -nographic console mode:
qemu-kvm <all regular options> -nographic -option-rom sgabios.bin,bootindex=0
# wait for the lilo prompt, it takes time to show up
boot: vm3.2.27b console=ttyS0
# ctrl-a h    for help
# ctrl-a c    for (QEMU) console
# BECAREFUL not to start qemu twice on the same ZFS guest
#  two simultaneous running qemu on the same ZFS will corrupt the pool
#  then it will NOT recover... you lose your whole installation!
# BECAREFUL when using -nographic ttyS0 that you are not doing commands on the HOST!
# only use -nographic to see kernel problems then quit it
# Doing this is only useful until the kernel panics because additional console switching needs
# to be done still. If it does boot, then ctrl-a c, and quit (sorry) and reboot normal
# The console switching part missing in init would look similar to this:
#
#if (console) {
#		close(0);
#		xopen(console, O_RDWR);
#		xdup2(0, 1);
#		xdup2(0, 2);
#	}

# to use if=scsi, linux module sym53c8xx, may need to use the qemu option:
#       -option-rom 8xx_64.rom,bootindex=1
# download it here:
# http://www.lsi.com/downloads/Public/Host%20Bus%20Adapters/Host%20Bus%20Adapters%20Common%20Files/lsi_bios.zip
# extract 8xx_64.rom to /usr/share/qemu/

Then, at the lilo prompt, use kernel parameter: console=ttyS0     (do NOT say /dev/ttS0)
There is a delay before you see LILO so be patient. You should see full messages of kernel.
The purpose of sgabios is that it is the "serial graphics adapter" video BIOS that sends all its text mode I/O over ttyS0.
The -nographic option redirects the guest's ttyS0 to your console.
The purpose of 8xx_64.rom is the SCSI interface firmware, and it boots after the SGABIOS so that everything will be visible.
For QEMU help, press ctrl-a then hit h.

Good luck!