Gentoo Diskless HOWTO for building an SSI image. Used in conjunction with the and Installing Gentoo as a Beowulf Head node, you might just end up with a Beowulf cluster in your hands!
This document only gets you to the point of having a system image which is bootable via NFS and gives you a functional console. Adding apps is up to you. If you want to make an X Terminal SSI, check out the references, I haven't had the time to MueCow Gentoo yet.
Contents [hide]
Reference articles I try to keep these in order of importance/relevance.
More references are inserted in the relevant sections. HOWTO Diskless X terminal with read-only root This link is of main interest (Wilhelm Meier's web site on the subject) Especially the "Preparation of the client-root" section Gentoo Diskless Clients Installation, Configuration by Wilhelm Meier, Markus Muller,Zweibrucken Project MueKow Ubuntu's ThinClientIntegration Linux Boot Present limitation This procedure is to give you a base system INIT scripts which should and shouldn't run on boot time still need to be identified (ie: do we really need to run checkfs when booting an SSI system).
Update, use baselayout 1.12.*, it's much nicer on these processes Kyron 15:22, 1 June 2006 (EDT)
I am presently concentrating on building this environment on locally compatible ARCHes (ie: building the root for a PIII on a P4, Athlon-XP on amd64 Opteron...).
Even so, you will have to use the following command to chroot into your node's tree (if your on an amd64 and the nodes are older generation AMD procs (x86)): linux32 chroot ./bin/bash instead of chroot . /bin/bash My true interest is to use this approach for building a "dynamic" cluster. So I won't spend too much time on the SSI as a workstation aspect although I will work on "MueKow"ing Gentoo at one point ;P The approach explored here is a custom kernel for custom hardware. This implies that I won't get into the complicated, yet flexible, approach which requires and INItial Ram Disk(initrd).
We should use dev-util/cpuinfo-collection to identify and select the proper arch-optimised SSI Getting to a basic Gentoo system. Presently, the approach is to CHROOT into the environment. Conversations on the #gentoo-embedded IRC channel from freenode.net points to using a simple script modified for each environment. There are drawbacks to using such a script and this is not the approach tested/used here. Nonetheless, here is an example of such a script: File: PIII_SSI.sh <pre class="brush:py;">
-fomit-frame-pointer" CXXFLAGS="$CXXFLAGS ${CFLAGS}" LDFLAGS="$LDFLAGS -L${ROOT}/lib -L${ROOT}/usr/lib" CHOST=i686-pc-linux-gnu
softfloat-linux-uclibc/lib/pkgconfig/ USE="-* netboot sse sse2 blas mpi zlib nls userlocales pbs minimal $USE" FEATURES=" $FEATURES -sandbox" export PKG_CONFIG_PATH ARCH ACCEPT_KEYWORDS CFLAGS CXXFLAGS LDFLAGS ROOT CBUILD CHOST CTARGET CC CXX PKGDIR USE FEATURES emerge -av $ </pre>
Stage 3 Install Skip to the stage 3 section of the Gentoo Installation Handbook. We essentially follow the same procedure as a regular Gentoo install. We will use <pre class="brush:py;"> /tftproot/$ARCH_OF_NODE/ </pre>
as the install dir in our case (where $ARCH_OF_NODE is the actual arch specific for your node, we can have more than one of course).
Code: Quick copy/paste of a typical STAGE 3 root preparation export ARCH_OF_NODE=AthlonXP export GENTOO_INSTALLDIR=/tftproot/$ARCH_OF_NODE tar -xpjf
stage3 -*.tar.bz2 -C $GENTOO_INSTALLDIR wget
http://gentoo.osuosl.org/snapshots/portage-latest.tar.bz2 wget
http://gentoo.osuosl.org/snapshots/portage-latest.tar.bz2.md5sum
we want to keep the portage trees separated, distfiles can be shared though...
<pre class="brush:py;"> tar -xpjf portage-latest.tar.bz2 -C $GENTOO_INSTALLDIR/usr/ cp /etc/resolv.conf $GENTOO_INSTALLDIR/etc/ mount -t proc none $GENTOO_INSTALLDIR/proc/ mount -o bind /sys $GENTOO_INSTALLDIR/sys/ </pre>
<pre class="brush:py;">
--bind /usr/portage/distfiles/ $GENTOO_INSTALLDIR/usr/portage/distfiles/
$GENTOO_INSTALLDIR/etc/locales.build
</pre>
env-update emerge sync Client kernel configuration Apart from personnalised hardware support which you must select for your own hardware, the following options must be compiled into your kernel. Don't forget that you will want your network device driver(s) to be BUILT_IN (a * and not an M for module, more than one can be selected).
Linux Kernel Configuration: Device Drivers ---> Networking support ---> Networking options --->
Once you have downloaded the package, I recommend you untarr it into the chroot environment and build the module from this environment. Note that you cannot just change into the unionfs source directory and type make unless your chroot environment is runnign the exact same kernel version as the system (server) your are building the image from. At the top of the Makefile you will find: # this should point to where your kernel headers are KVERS=$(shell uname -r) You will want to change the KVERS=$(shell uname -r) line with the kernel version you are using for your nodes.
Here is an example: KVERS="2.6.16-gentoo-r7" You will also need to add the following entry in the same Makefile: EXTRACFLAGS=-DUNIONFS_UNSUPPORTED Once you have set these two variables in th Makefile, you should be able to compile the unionfs module with: make ...and installing the module with: make install Since we use modprobe in our stateless.sh script to load this module at startup, depmod has to be run manually. Again, if the chroot's kernel is not exactly the same version as the server system your have to tweak the tool's argument with (for example):
depmod -v 2.6.16-gentoo-r7 And, since I really love em, here is a "one-liner" that automagically does all this: make clean
http://nfs.sourceforge.net/ Description: NFS client and server daemons License: GPL-2 Client side files to add/modify REMEMBER, all of this is done in the client's CHROOT environment! Modify /etc/make.conf Set the CFLAGS and USE flags as you see fit for your environment and hardware (you should have already done this if you started with a Stage1 system). Nothing is preventing you from building more than one SSI ;)
Here is an example for stripped down nodes (Beowulf cluster oriented). File: /etc/make.conf USE="-* sse sse2 blas mpi zlib nls userlocales pbs minimal unicode" CFLAGS="-O2 -mcpu=pentium3 -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CXXFLAGS="${CFLAGS}" MAKEOPTS="-j4" /boot/stateless.sh You can download this version of the script here. This script is the one called instead of the usual init script (which is called at the end). This script sets up unionfs and sets per-node variables such as hostname. It's a regular BASH script, to you can do as you please with it ;). Note:Append init=/boot/stateless.sh softlevel=unionfs to your kernel command line and place this script in your node /boot directory if you haven't already done so. Don't forget to make stateless.sh executable and to add the mount directories chmod a+x /boot/stateless.sh mkdir -p /mnt/unionfs/var mkdir -p /mnt/unionfs/etc File: stateless.sh #!/bin/bash MODPROBE=/sbin/modprobe IFCONFIG=/sbin/ifconfig MYHOST=$(/sbin/dhcpcd -H; /bin/hostname)
<pre class="brush:py;">
we'll use the DHCP assigned hostname NODE_NAME="Node" ahostname(){ if [ -z ${MYHOST} ]; then echo "DHCP didn't tell me my name. Generating my own hostname..." MYHOST="${NODE_NAME}$IFCONFIG eth0 | awk '/inet addr/ {print
$2}' | tr -t . ' ' | awk '{print $4}'" else echo DHCP told me my hostname is ${MYHOST}... echo "Setting domainname to DHCP's settings" /sbin/dhcpcd -D fi echo "STATELESS: Setting Hostname to $MYHOST" echo "HOSTNAME="$MYHOST"" > /etc/conf.d/hostname /bin/hostname "$MYHOST" } getparams() { local cmdline=$(dmesg | grep '^Kernel command line' | sed 's/^Kernel command line://g') for pp in $cmdline; do echo $pp | grep '^softlevel=' >/dev/null 2>&1 if [ $? -eq 0 ]; then echo $pp | sed 's/softlevel=//g' return 0 fi done echo "" return 1 } isset() { for p in $(getparams | tr ',' ' '); do if [ "$p" == "$1" ]; then return 0 fi done return 1 } aunionfs() { isset unionfs if [ $? -eq 0 ]; then echo "STATELESS: Loading module unionfs ..." $MODPROBE unionfs while [ "$1" != "" ]; do echo "STATELESS: Mounting tmpfs $1 ..." mount -n -t tmpfs -o defaults none /mnt/unionfs/$1 echo "STATELESS: Mounting $1 unionfs ..." mount -n -t unionfs -o dirs=/mnt/unionfs/$1=rw:/$1=ro none /$1 shift done else echo "STATELESS: Not using unionfs as requested ..." fi } aunionfs etc var ahostname exec /sbin/init /etc/conf.d/local. </pre>
stop Since we are stateless, we can speed up things a little bit. Note that this forces the node to ALWAYS shutdown (even if you press CTRL-ALT-DEL)! File: /etc/conf.d/local.stop
<pre class="brush:py;">
</pre> here.
<pre class="brush:py;"> /sbin/poweroff -f /etc/fstab This is obviously implementation dependant but here is an example. Note that the dump/pass parameters here can have some importance. File: /etc/fstab (without comments) </pre>
<pre class="brush:py;">
192.168.1.1:/home /home nfs rw,defaults,hard,intr,rsize=8192 0 2 </pre>
<pre class="brush:py;">
tmpfs nodev,nosuid,noexec 0 0 Important server-side files /tftpboot/pxelinux.cfg/default This file is crucial.
</pre>
It's here that you define many parameters that could change the loaded SSI as well as the kernel being used. Refer to How do I Configure SYSLINUX?
File: <pre class="brush:py;"> /tftpboot/pxelinux.cfg/default prompt 1 default ltsp timeout 50 say Press F1 for boot profiles, default is ltsp in 5 seconds... F1 BootProfiles label ltsp kernel lts/2.6.9-ltsp-3/bzImage-2.6.9-ltsp-3 append init=/linuxrc rw root=/dev/ram0 initrd=lts/2.6.9-ltsp-3/initrd-2.6.9-ltsp-3.gz label Gentoo kernel AthlonXP/boot/vmlinuz append root=/dev/nfs nfsroot=192.168.1.2:/tftproot/AthlonXP,hard,intr,rw udev gentoo=nodevfs ip=dhcp init=/boot/stateless.sh softlevel=unionfs label Windows localboot 0 /tftpboot/BootProfiles </pre>
This file is not essential but is quite useful in a multi-boot environment.
<pre class="brush:py;"> File: /tftpboot/BootProfiles Type ltsp for regular LTSP bootup (default after 5 seconds) Type Gentoo for the AthlonXP environment (GENTOO SSI boot dev env) Type Windows to play worms :) </pre>
<pre class="brush:py;"> /etc/exports And finally, the NFS exports: File: /etc/exports /tftproot/AthlonXP 192.168.1.0/255.255.255.0(ro,sync,no_all_squash,no_root_squash) /home 192.168.1.0/255.255.255.0(rw,async,no_root_squash) /ScratchPad/NodeExport 192.168.1.0/255.255.255.0(rw,async,no_root_squash) </pre>
runlevel scripts This is not mandatory, we could just use the default runlevel. But here, in the kernel command line, we specify the "softlevel" to be unifonfs. This implies that you create the folder /etc/runlevels/unionfs and you add to this runlevel all apps you want fired up under these conditions. <pre class="brush:py;"> For example: mkdir -p /etc/runlevels/unionfs rc-update add local unionfs We'll be using NFS mounts: rc-update add portmap unionfs rc-update add nfs unionfs rc-update add netmount unionfs Some other useful services for I in acpid gmond sshd; do rc-update add $I unionfs; </pre>