Disk-Less nodes are a useful way of bring up machines on a network. Applications include workstations or cluster nodes. This short paper outlines how to get such a system up and running. Caveats on doing this with Intel Ether Express Pro hardware are covered in detail.
The disk in one of machines broke so I decided it was time to try disk-less again. Initially I was going to try and boot of a floppy disk, but I soon discovered that the Intel Ether-Express Pro 100 has an on-board DHCP and TFTP server. Here is my journey.
The vmlinuz-2.4.13.mosix was configured with the following to allow an nfs-root to be established:
Networking Options
TCP/IP networking (CONFIG_INET)
IP: kernel level autoconfiguration (CONFIG_IP_PNP)
IP: DHCP support (CONFIG_IP_PNP_DHCP)
File Systems
/dev file system support (CONFIG_DEVFS_FS)
Automatically mount at boot (CONFIG_DEVFS_MOUNT)
Network File Systems
NFS file system support (CONFIG_NFS_FS)
Root file system on NFS (CONFIG_ROOT_NFS)
In addition the following option was used to allow tmp to be kept in virtual memory, rather than on an nfs mount.
File Systems Virtual memory file system support (CONFIG_TMPFS)
On the NFS server a directory was populated with a root file system for the disk-less machine. In the end I made a separate copy of / and a shared copy of /usr and /home. This suited my needs at the time but is by no means the best solution.
As we will be using tmpfs the following symlinks were established: /tmp -> /dev/shm /var/tmp -> /dev/shm
The after this the fstab on the disk-less node was as follows.
proc /proc proc defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 /dev/fd0 /floppy auto defaults,user,noauto 0 0 /dev/cdrom /cdrom iso9660 defaults,ro,user,noauto 0 0 nfs-server:/export/nfs/loretta / nfs defaults 0 0 nfs-server:/usr /usr nfs defaults,ro 0 0 nfs-server:/home /home nfs defaults 0 0
As devfs is being used some modification to the system configuration is required. In particular I modified the /etc/inittab lines for virtual terminals to the following:
1:2345:respawn:/sbin/getty 38400 vc/1 2:23:respawn:/sbin/getty 38400 vc/2 3:23:respawn:/sbin/getty 38400 vc/3 4:23:respawn:/sbin/getty 38400 vc/4 5:23:respawn:/sbin/getty 38400 vc/5 6:23:respawn:/sbin/getty 38400 vc/6
Setting up DHCP was straight forward, a static entry as follows was inserted into the dhcp.conf entry on the DHCP server.
host loretta {
filename "/tftpboot/pxelinux.0";
next-server 192.168.7.14;
hardware ethernet 00:90:27:3E:EA:60;
fixed-address 192.168.7.15;
}
Where 192.168.7.14 is the IP address of the TFTP server.
Setting up the TFTP server was quite painful as in their infinite wisdom Intel insist on Lan Desk shipping with a broken TFTP server. In short you need to disable MTU discovery for the TFTP server and set the daemon to disable blksize[1]. The former can be achieved by running the following on the TFTP server:
echo 1 > /proc/sys/net/ipv4/ip_no_pmtu_disc
Alternatively add the following line to /etc/sysctl.config and then run sysctl -p. This will ensure that this option is set on reboot.
net.ipv4.ip_no_pmtu_disc=1
To disable blksize you need a TFTP daemon that allows you to do this. I used tftp-hpa[2], I believe atftp[3] will allow you to do the same. If your TFTP server runs windows try tftp32[4] The /etc/inetd.conf entry for tftp-hpa with blksize disabled is as follows:
tftp dgram udp wait nobody /usr/sbin/tcpd /usr/sbin/in.tftpd -r blksize /tftpboot
Instead of the tftp server supplying a linux Kernel, Pxelinux[5], which is part of Syslinux[6] is used. This is done for two reasons. Firstly, Lan Desk's TFTP client has a limit on the size of file that it is able to download. I am not sure of the size of the limit but it seems to be significantly smaller than a kernel image. Fortunately Pxelinux is small enough for Lan Desk to be able to download. Secondly, using Pxelinux allows us to pass boot-time options to the kernel.
To do this I populated the /tftpboot directory as follows:
vmlinuz-2.4.13.mosix pxelinux.cfg/default pxelinux.0
Where:
Pxelinux searches for configuration files by taking the hexadecimal representation of the host's IP address and removing lest significant hex digits until a match is found. If no match is found then default is used. [6]
For example 192.168.0.15 is C0A8000F.
Thus the following configuration files will be requested. The first one found will be used.
pxelinux.cfg/C0A8000F pxelinux.cfg/C0A8000 pxelinux.cfg/C0A800 pxelinux.cfg/C0A80 pxelinux.cfg/C0A8 pxelinux.cfg/C0A pxelinux.cfg/C0 pxelinux.cfg/C pxelinux.cfg/default
This configuration file was used to specify the name of the image, the image to be used, and nfs root information needed. ipappend 1 tells syslinux to set the IP address of the node passed to the kernel to that supplied to Lan Desk by DHCP. The configuration file is as follows:
label linux kernel vmlinuz-2.4.13.mosix append nfsroot=192.168.7.14:/export/nfs/loretta ipappend 1
Getting TFTP up and running proved to be much more painful than it should have been due to bugs and limitations in Lan Desk. Thanks Intel for wasting my time. I'll be sure to return the favour in the future.
In the end it all works quite well. The disk-less node boots up over a 100Mb Ethernet link quite quickly as there are no disk checks. Provided you aren't doing disk-intensive work performance shouldn't be an issue.
The majority of these references are written by, or supplied by H. Peter Anvin.
Copyright © 1995-2024 Simon Horman
Copyright © 2010-2024 Horms Solutions Ltd.
All rights reserved unless otherwise noted