Guruplug meets NetBSD step 4: A closer look at our rootfs creation

In the second step I started from a distribution build in build.sh this was an unprivileged build and I used this build to populate my rootfs by passing through a tarball. Initially this has worked but it exhibited some fundamental problems which are side-effects of my hackish approach there.

An unprivileged build creates the rootfs in desdir (obj/destdir.evbarm) and stores the permissions in a file called METALOG:

edb@lapedb:~/netbsd/src$ head -n 10 obj/destdir.evbarm/METALOG
. type=dir uname=root gname=wheel mode=0755
./.cshrc type=file uname=root gname=wheel mode=0555 size=1297 sha256=0f09961bfa5543fa7da2cf8836935d6459d3ae3180795b33e2d04c81536c10a9
./.profile type=file uname=root gname=wheel mode=0555 size=849 sha256=55bacdbebf6e16e4cebd76e48ead3875615fb9236f622dfded41843a2a4d8b14
./altroot type=dir uname=root gname=wheel mode=0755
./bin type=dir uname=root gname=wheel mode=0755
./bin/[ type=file uname=root gname=wheel mode=0555 size=12902 sha256=8af5ca258dbc6ff97b586ce0b9c7fae5f692d35e45db01b97844e123ccc649b3
./bin/cat type=file uname=root gname=wheel mode=0555 size=12546 time=1312632230.0 sha256=ad8ec94d65ca37c69aeeeddf88aa99e4cb2e318bfc23ef7b0d42417df9f29d53
./bin/chgrp type=link mode=0755 link=/sbin/chown
./bin/chio type=file uname=root gname=wheel mode=0555 size=16052 time=1312632230.0 sha256=579bf34f922708e10b2196edfa77f666e865748b41662b380d19c2ba64f51d5d
./bin/chmod type=file uname=root gname=wheel mode=0555 size=9906 time=1312632231.0 sha256=76aacc5979b482c04796b0c844b74d40d9f9d475f178e90871f0e783c1df63b8
...

In order to create a proper rootfilesystem the information stored in METALOG should also be applied otherwise just copying destdir over will look funny, (permission wise that is).


For some local testing I generated the binary installation sets (which will also contains the faulty permissions), to get to know which ones are there:

edb@lapedb:~/netbsd/src$ ./build.sh -j 2 -m evbarm -U -u sets 
...
edb@lapedb:~/netbsd/src$ ls -hal obj/releasedir/evbarm/binary/sets/
total 128M
drwxr-xr-x 2 edb edb 4.0K Aug  7 11:39 .
drwxr-xr-x 3 edb edb 4.0K Aug  7 11:37 ..
-rw-r--r-- 1 edb edb  453 Aug  7 11:39 MD5
-rw-r--r-- 1 edb edb 1.4K Aug  7 11:39 SHA512
-rw-r--r-- 1 edb edb  35M Aug  7 11:38 base.tgz
-rw-r--r-- 1 edb edb  56M Aug  7 11:39 comp.tgz
-rw-r--r-- 1 edb edb 396K Aug  7 11:38 etc.tgz
-rw-r--r-- 1 edb edb 3.2M Aug  7 11:38 games.tgz
-rw-r--r-- 1 edb edb  13M Aug  7 11:39 man.tgz
-rw-r--r-- 1 edb edb 3.4M Aug  7 11:39 misc.tgz
-rw-r--r-- 1 edb edb 1.2M Aug  7 11:39 modules.tgz
-rw-r--r-- 1 edb edb  13M Aug  7 11:39 tests.tgz
-rw-r--r-- 1 edb edb 3.4M Aug  7 11:39 text.tgz

And then I installed the rootfs to a local directory:

edb@lapedb:~/netbsd/src$ export DESTDIR=/home/edb/netbsd/src/obj/destdir.evbarm 
edb@lapedb:~/netbsd/src$ export RELEASEDIR=/home/edb/netbsd/src/obj/releasedir 
edb@lapedb:~/netbsd/src$ export INSTALLSETS="base comp etc games man misc modules tests text"  ./build.sh -j 2 -m evbarm -E  -U -u install=/tmp/blah

Here I repeat the DESTDIR and RELEASEDIR variables, since the -E (expert) flag of build.sh somehow removes these and then an UNPRIVED installation will take place in /tmp/blah, this will typically be a copy of DESTDIR (with the permissions set to a wrong value), but later on we can omit certain installation sets (games, modules, tests come to mind). But a next step is actually applying the permissions stored in the METALOG file, this can be done by using the mtree(8) command. On my host system
(Linux) it is not available, but it is luckily/obviously part of the toolchain:

edb@lapedb:~/netbsd/src$ sudo obj/tooldir.Linux-2.6.39-2-686-pae-i686/bin/nbmtree -f obj/destdir.evbarm/METALOG -p /tmp/blah -N /tmp/blah/etc/ -u

Some comments:

• sudo: since some binaries will need to end up with setuid privileges etc this needs to be done with elevated privileges.
• -f: Reference to the METALOG file
• -p: Path containing the root directory
• -N: Use the user databases (passwd and friends) from this location, obviously from out target, this because my host does not per see need to have all users/groups referenced (e.g. wheel group isn't available on Linux, this while the METALOG references users and groups by name instead of by id
• -u: Update the fields

Okay, so this would allow us to repopulate the partition on the USB stick we are using, however nothing forbids us to apply these settings on our running filesystem (post-mortem that is). So let's just give that a try.

GURUPLUGGER# mtree -f /METALOG -p / -u
...

And you will see it will fix the permissions of all files. (Along with all other anomalies in the filesystem, e.g. in the previous step we touched rc.conf and mtree will show that the hash has changed).