In the technical article Why are unlink("/") And rmdir("/") Failing? which I wrote back in 2021, I imagined a scenario where all the hard links to the root directory are removed from a file system. I had always wanted to actually achieve that somewhere since then.
I chose to experiment with FreeBSD kernel and UFS, since this kernel is the one I'm most familiar with. I planned to load a kernel image with Multiboot support and an UFS RAM disk for use as root, using qemu-system-i386(1), just like what I already done for that experiment in 2019. I will also need to prepare a statically linked ls(1) program at '/ls', to list and stat(2) the root directory.
The first issue, without entering the crazy part of removing '.' and '..' links, was removing the '/dev' directory.
In kFreeBSD, a devfs(5) is mounted automatically on '/dev/', as a part of the progress of mounting '/' (actually, this devfs was initially mounted at '/', then moved to '/dev/' after mounting the final root file system; see kernel function vfs_mountroot). Attempt to forcibly unmount '/dev/' from the local console resulted in the console to freeze up completely.
After some testings in another virtual machine, I realized that the kernel should still function without '/dev', at least for stuffs that don't need any device node to function, such as pipes and sockets. I just need to configure the IP network, start an inetd-like service to run sh(1), with redirected stdin, stdout and stderr from the accepted TCP socket. The process 1 in my RAM disk system is '/bin/sh'; I also need to exec it to something else, preferably a command like 'while true; do wait; done', otherwise it will exit upon unmounting of '/dev/' (due to losing its stdin, '/dev/console'), kFreeBSD will then complain 'Going nowhere without my init!'
It worked. I can now access the target system using Netcat, and forcibly unmount '/dev/' in it:
$ nc -n 10.0.3.29 299 sh: can't access tty; job control turned off # uname -a FreeBSD 10.4-RELEASE-p14 FreeBSD 10.4-RELEASE-p14 #4: Sun Feb 1 14:13:54 CST 2026 WHR@beijing-storage.rivoreo:/usr/home/WHR/src/freebsd-10.4/sys/i386/compile/Rivoreo i386 # umount -f /dev/ # mount /dev/md0 on / (ufs, local) # rmdir /dev
Now it is time to implement a new unlink that will allowing me to remove directories links including '.' and '..'!
It takes some more works to bypass all the checks that designed to forbid removing special directory links, and have my unlink request landed into the UFS VFS implementation correctly (the UFS driver usually went insane when being asked to unlink '.' and/or '..', resulted in different kernel panics). I also wrote a tiny command line tool that reads an UFS directory using read(2) system call directly in the process; this allows me to inspect the raw directory content, to help debugging the new KLD module.
This module will register a sysctl entry vfs.unlink; when set with a path name, it will behave like the unlink(2) system call, except it will also try to unlink a directory when the user asked to. Of course the individual VFS implementations may decide to reject such a request, but thankfully it is not the case in ufs. The source code of this KLD module should be available on Rivoreo's source browsing site soon.
Also keep in mind that even if '.' is unlinked from a directory for real, namei(9) will still fake its presence for self reference. So for example the shell commands like 'ls -dl', 'stat .' and 'ufs-readdir .' will keep working even there's no '.' link in the working directory.
I started qemu-system-i386 with a command line similar to:
qemu-system-i386 -enable-kvm -m 512 -net tap,ifname=xxx -net nic,model=virtio,macaddr=52:54:02:22:0a:03 -kernel kernel -append "-e kern.hz=200 -e hw.vtnet.lro_disable=1 -e hw.vtnet.tso_disable=1 -e hw.vtnet.csum_disable=1 -e net.inet.ip.fw.default_to_accept=1 -i /init" -initrd tmpfs.ko,fuse.ko,initrd,ipfw.ko,ipdivert.ko,vfsunlink.ko,imgact_binmisc.ko
Then the following Unix shell interaction is produced:
$ nc -n 10.0.3.29 299
sh: can't access tty; job control turned off
# uname -a
FreeBSD 10.4-RELEASE-p14 FreeBSD 10.4-RELEASE-p14 #4: Sun Feb 1 14:13:54 CST 2026 WHR@beijing-storage.rivoreo:/usr/home/WHR/src/freebsd-10.4/sys/i386/compile/Rivoreo i386
# kldstat
Id Refs Address Size Name
1 13 0xc0400000 ebd8ac kernel
2 1 0xc12bf000 b6bc tmpfs.ko
3 1 0xc12cf000 e9f4 fuse.ko
4 2 0xc32e3000 fe7c ipfw.ko
5 1 0xc32f8000 47d8 ipdivert.ko
6 1 0xc3300000 3aac vfsunlink.ko
7 1 0xc3304000 3c40 imgact_binmisc.ko
# mount
/dev/md0 on / (ufs, local)
devfs on /dev (devfs, local, multilabel)
# pwd
/
# fsck_ufs -n dev/md0
** /dev/md0 (NO WRITE)
** Last Mounted on /
** Root file system
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
873 files, 5115 used, 2700 free (36 frags, 333 blocks, 0.5% fragmentation)
# ls -ail /
total 105
2 drwxr-xr-x 12 0 wheel 512 Feb 2 07:27 .
2 drwxr-xr-x 12 0 wheel 512 Feb 2 07:27 ..
2304 drwxr-xr-x 2 0 wheel 512 Feb 1 07:20 bin
2 dr-xr-xr-x 5 0 wheel 512 Feb 2 07:34 dev
1153 drwxr-xr-x 2 0 wheel 512 May 23 2019 etc
458 -rwxr-xr-x 1 0 wheel 231 Feb 1 07:41 init
2305 drwxr-xr-x 2 0 wheel 512 Feb 1 07:07 lib
2306 drwxr-xr-x 2 0 wheel 512 May 23 2019 libexec
1154 drwxr-xr-x 2 0 wheel 512 May 23 2019 mnt
2307 drwxr-xr-x 2 0 wheel 1024 Feb 2 07:19 sbin
1155 drwxrwxrwt 2 0 wheel 512 May 23 2019 tmp
3 drwxr-xr-x 7 0 wheel 512 May 23 2019 usr
4 drwxr-xr-x 10 0 wheel 512 May 23 2019 var
457 -rwxr-xr-x 1 0 wheel 36 Feb 1 07:42 wait
# ufs-readdir /
OFFSET INDEX T NAME
0 2 d .
12 2 d ..
24 2304 d bin
36 1152 d dev
48 1153 d etc
60 2305 d lib
72 2306 d libexec
88 1154 d mnt
100 2307 d sbin
116 1155 d tmp
128 3 d usr
140 4 d var
172 458 - init
188 457 - wait
# sysctl vfs.unlink=.
-> # ifconfig -a > 1
# for f in usr/bin/*-static usr/sbin/*-static bin/*-static sbin/*-static; do [ -f "$f" ] && [ -x "$f" ] && bn="${f##*/}" && mv "$f" "/${bn%-static}"; done
# fsck_ufs -n dev/md0
** /dev/md0 (NO WRITE)
** Last Mounted on /
** Root file system
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
MISSING '.' I=2 OWNER=0 MODE=40755
SIZE=512 MTIME=Feb 2 07:35 2026
DIR=/
CANNOT FIX, FIRST ENTRY IN DIRECTORY CONTAINS 1
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
874 files, 5116 used, 2699 free (35 frags, 333 blocks, 0.4% fragmentation)
# umount -f dev/
# rm -rf dev etc mnt tmp var
# ls -ail
total 3976
2 drwxr-xr-x 6 0 0 512 Feb 2 07:35 ..
10 -rw-r--r-- 1 0 0 638 Feb 2 07:35 1
2304 drwxr-xr-x 2 0 0 512 Feb 2 07:35 bin
1333 -rwxr-xr-x 1 501 0 455168 Feb 1 07:22 inetrdr
458 -rwxr-xr-x 1 0 0 231 Feb 1 07:41 init
2305 drwxr-xr-x 2 0 0 512 Feb 1 07:07 lib
2306 drwxr-xr-x 2 0 0 512 May 23 2019 libexec
2410 -rwxr-xr-x 1 501 0 963084 Feb 1 07:20 ls
2307 drwxr-xr-x 2 0 0 1024 Feb 2 07:35 sbin
2411 -rwxr-xr-x 1 0 0 446640 Feb 2 07:19 sysctl
3 drwxr-xr-x 7 0 0 512 May 23 2019 usr
457 -rwxr-xr-x 1 0 0 36 Feb 1 07:42 wait
# ufs-readdir /
OFFSET INDEX T NAME
0 10 - 1
12 2 d ..
24 2304 d bin
60 2305 d lib
72 2306 d libexec
100 2307 d sbin
128 3 d usr
152 1333 - inetrdr
172 458 - init
188 457 - wait
204 2410 - ls
216 2411 - sysctl
# mount
/dev/md0 on / (ufs, local)
# rm -rf bin lib libexec sbin usr
rm: usr: Invalid argument
# /sysctl vfs.unlink=..
-> # /ls -al
total 3936
-rw-r--r-- 1 0 0 638 Feb 2 07:35 1
-rwxr-xr-x 1 501 0 455168 Feb 1 07:22 inetrdr
-rwxr-xr-x 1 0 0 231 Feb 1 07:41 init
-rwxr-xr-x 1 501 0 963084 Feb 1 07:20 ls
-rwxr-xr-x 1 0 0 446640 Feb 2 07:19 sysctl
drwxr-xr-x 2 0 0 512 Feb 2 07:35 usr
-rwxr-xr-x 1 0 0 36 Feb 1 07:42 wait
# /ls -al usr/
total 16
drwxr-xr-x 2 0 0 512 Feb 2 07:35 .
drwxr-xr-x 1 0 0 512 Feb 2 07:36 ..
# /sysctl vfs.unlink=usr/.
-> # /sysctl vfs.unlink=usr/..
-> # /sysctl vfs.unlink=usr
sysctl: vfs.unlink=usr: No such file or directory
# /ls -al usr/
ls: usr/: No such file or directory
# /ls -al
total 0
# /ls -dil /
2 drwxr-xr-x 0 0 0 512 Feb 2 07:36 /
# while IFS= read -r line; do printf %s\\n "$line"; done < 1
vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
ether 52:54:02:22:0a:03
inet6 fe80::5054:2ff:fe22:a03%vtnet0 prefixlen 64 scopeid 0x1
inet 10.0.3.29 netmask 0xffffff00 broadcast 10.0.3.255
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
media: Ethernet 10Gbase-T <full-duplex>
status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
inet 127.0.0.1 netmask 0xffff0000
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
# while IFS= read -r line; do printf %s\\n "$line"; done < init
#!/bin/sh
exec 0<> /dev/console 1>&0 2>&0
PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin
export PATH
mount -u -o rw /
ifconfig lo0 127.0.0.1/16 up
inetrdr -p tcp -b :299 -elf /bin/sh -i &
uname -srm
exec sh
# /ls -l 1 init /
drwxr-xr-x 0 0 0 512 Feb 2 07:36 /
-rw-r--r-- 1 0 0 638 Feb 2 07:35 1
-rwxr-xr-x 1 0 0 231 Feb 1 07:41 init
# exit
$ nc -n 10.0.3.29 299
/bin/sh: No such file or directory
Look what I got here:
2 drwxr-xr-x 0 0 0 512 Feb 2 07:36 /
The directory at '/', inode number 2, has a link count of 0!
However the kernel apparently shortcuts the getdirentries(2) to return nothing, if the requested directory have no hard link. This behavior makes sense because you shouldn't continue to use a directory when that directory is subject for removal. It made me think however, what if I try to mount this broken file system somewhere else?
So I dumped the memory of this virtual machine, extracted the 32 MiB UFS image from it, send the image to another virtual machine that runs FreeBSD 11.1-RELEASE, setup a md(4) unit with the UFS image, and successfully mounted it read-only. There's no hard links to the root directory to be found anywhere in its directory structure, but stat(2) show the link count of its root directory is 11, not 0. kFreeBSD doesn't let me to mount it writable, due to its obvious inconsistent state; but I can force the operation:
[backdoor@zuo-freebsd ~]# mount -u -o rw /mnt/4/
mount: /dev/md1: R/W mount of /mnt/4 denied. Filesystem is not clean - run fsck.: Operation not permitted
[backdoor@zuo-freebsd ~]# mount -u -o rw,force /mnt/4/
[backdoor@zuo-freebsd ~]# cd /mnt/4/
[backdoor@zuo-freebsd /mnt/4]# ls -ail
total 3928
10 -rw-r--r-- 1 backdoor wheel 638 Feb 2 07:55 1
1333 -rwxr-xr-x 1 WHR wheel 455168 Feb 1 07:22 inetrdr
458 -rwxr-xr-x 1 backdoor wheel 231 Feb 1 07:41 init
2410 -rwxr-xr-x 1 WHR wheel 963084 Feb 1 07:20 ls
2411 -rwxr-xr-x 1 backdoor wheel 446640 Feb 2 07:19 sysctl
457 -rwxr-xr-x 1 backdoor wheel 36 Feb 1 07:42 wait
[backdoor@zuo-freebsd /mnt/4]# mkdir tmp
[backdoor@zuo-freebsd /mnt/4]# ls -al
total 3936
-rw-r--r-- 1 backdoor wheel 638 Feb 2 07:55 1
-rwxr-xr-x 1 WHR wheel 455168 Feb 1 07:22 inetrdr
-rwxr-xr-x 1 backdoor wheel 231 Feb 1 07:41 init
-rwxr-xr-x 1 WHR wheel 963084 Feb 1 07:20 ls
-rwxr-xr-x 1 backdoor wheel 446640 Feb 2 07:19 sysctl
drwxr-xr-x 2 backdoor wheel 512 Feb 2 09:05 tmp
-rwxr-xr-x 1 backdoor wheel 36 Feb 1 07:42 wait
[backdoor@zuo-freebsd /mnt/4]# ls -dl
drwxr-xr-x 12 backdoor wheel 512 Feb 2 09:05 .
[backdoor@zuo-freebsd /mnt/4]# rmdir tmp
[backdoor@zuo-freebsd /mnt/4]# ufs-readdir .
0 10 - 1
152 1333 - inetrdr
172 458 - init
188 457 - wait
204 2410 - ls
216 2411 - sysctl
[backdoor@zuo-freebsd /mnt/4]# time fsck_ufs -n /tmp/0-linked-root.ufs > fsck.2026-2-2-17
real 0m0.104s
user 0m0.010s
sys 0m0.087s
[backdoor@zuo-freebsd /mnt/4]# ufs-readdir .
0 10 - 1
12 459 - fsck.2026-2-2-17
152 1333 - inetrdr
172 458 - init
188 457 - wait
204 2410 - ls
216 2411 - sysctl
I also realized here that after removal of '.' and/or '..', it is possible to create other links in the directory to take the original place of these special names; this will prevent fsck_ufs(8) from restoring them, as shown below, when I trying to fix this file system:
[backdoor@zuo-freebsd /mnt/4]# cd [backdoor@zuo-freebsd ~]# umount /mnt/4/ [backdoor@zuo-freebsd ~]# time fsck_ufs -y /dev/md1 ** /dev/md1 ** Last Mounted on /mnt/4 ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames MISSING '.' I=3 OWNER=backdoor MODE=40755 SIZE=512 MTIME=May 23 04:11 2019 DIR=? FIX? yes MISSING '.' I=2 OWNER=backdoor MODE=40755 SIZE=512 MTIME=Feb 2 09:06 2026 DIR=/ CANNOT FIX, FIRST ENTRY IN DIRECTORY CONTAINS 1 MISSING '..' I=2 OWNER=backdoor MODE=40755 SIZE=512 MTIME=Feb 2 09:06 2026 DIR=/ CANNOT FIX, SECOND ENTRY IN DIRECTORY CONTAINS fsck.2026-2-2-17 ** Phase 3 - Check Connectivity UNREF DIR I=3575 OWNER=backdoor MODE=40755 SIZE=512 MTIME=May 23 04:13 2019 RECONNECT? yes NO lost+found DIRECTORY CREATE? yes DIR I=3575 CONNECTED. PARENT WAS I=9 UNREF DIR I=3517 OWNER=WHR MODE=40755 SIZE=1536 MTIME=Jul 11 03:19 2014 RECONNECT? yes DIR I=3517 CONNECTED. PARENT WAS I=196 (Long output of fsck_ufs(8) skipped) UNREF FILE I=3580 OWNER=WHR MODE=100444 SIZE=1343488 MTIME=Jul 11 03:19 2014 RECONNECT? yes ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? yes SUMMARY INFORMATION BAD SALVAGE? yes BLK(S) MISSING IN BIT MAPS SALVAGE? yes 876 files, 5149 used, 2666 free (34 frags, 329 blocks, 0.4% fragmentation) ***** FILE SYSTEM STILL DIRTY ***** ***** FILE SYSTEM WAS MODIFIED ***** ***** PLEASE RERUN FSCK ***** real 0m0.155s user 0m0.070s sys 0m0.068s [backdoor@zuo-freebsd ~]# time fsck_ufs -y /dev/md1 ** /dev/md1 ** Last Mounted on /mnt/4 ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames MISSING '.' I=2 OWNER=backdoor MODE=40755 SIZE=512 MTIME=Feb 2 09:06 2026 DIR=/ CANNOT FIX, FIRST ENTRY IN DIRECTORY CONTAINS 1 MISSING '..' I=2 OWNER=backdoor MODE=40755 SIZE=512 MTIME=Feb 2 09:06 2026 DIR=/ CANNOT FIX, SECOND ENTRY IN DIRECTORY CONTAINS fsck.2026-2-2-17 ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 876 files, 5149 used, 2666 free (34 frags, 329 blocks, 0.4% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** real 0m0.044s user 0m0.000s sys 0m0.039s [backdoor@zuo-freebsd ~]# mount -t ufs /dev/md1 /mnt/4/ [backdoor@zuo-freebsd ~]# cd /mnt/4/ [backdoor@zuo-freebsd /mnt/4]# ls -ail total 4192 10 -rw-r--r-- 1 backdoor wheel 638 Feb 2 07:55 1 459 -rw-r--r-- 1 backdoor wheel 95384 Feb 2 09:06 fsck.2026-2-2-17 1333 -rwxr-xr-x 1 WHR wheel 455168 Feb 1 07:22 inetrdr 458 -rwxr-xr-x 1 backdoor wheel 231 Feb 1 07:41 init 460 drwx------ 47 backdoor wheel 36864 Feb 2 09:07 lost+found 2410 -rwxr-xr-x 1 WHR wheel 963084 Feb 1 07:20 ls 2411 -rwxr-xr-x 1 backdoor wheel 446640 Feb 2 07:19 sysctl 457 -rwxr-xr-x 1 backdoor wheel 36 Feb 1 07:42 wait
Apparently fsck_ufs(8) reconnected many files which I removed from the original RAM disk, back into 'lost+found' directory, suggesting the file system updates in the first virtual machine was incomplete.
When I trying to remove the 'lost+found' directory, I got EINVAL:
[backdoor@zuo-freebsd /mnt/4]# time rm -rf lost+found/ rm: lost+found/: Invalid argument real 0m0.106s user 0m0.033s sys 0m0.035s [backdoor@zuo-freebsd /mnt/4]# ls -al total 4192 -rw-r--r-- 1 backdoor wheel 638 Feb 2 07:55 1 -rw-r--r-- 1 backdoor wheel 95384 Feb 2 09:06 fsck.2026-2-2-17 -rwxr-xr-x 1 WHR wheel 455168 Feb 1 07:22 inetrdr -rwxr-xr-x 1 backdoor wheel 231 Feb 1 07:41 init drwx------ 2 backdoor wheel 36864 Feb 2 09:09 lost+found -rwxr-xr-x 1 WHR wheel 963084 Feb 1 07:20 ls -rwxr-xr-x 1 backdoor wheel 446640 Feb 2 07:19 sysctl -rwxr-xr-x 1 backdoor wheel 36 Feb 1 07:42 wait [backdoor@zuo-freebsd /mnt/4]# ls -ail lost+found/ total 80 460 drwx------ 2 backdoor wheel 36864 Feb 2 09:09 . 2 drwxr-xr-x 1 backdoor wheel 512 Feb 2 09:06 .. [backdoor@zuo-freebsd /mnt/4]# rmdir lost+found/ rmdir: lost+found/: Invalid argument
That's interesting. My guess is that when the UFS driver handling my rmdir(2) operation, it saw the parent directory, referenced by '..', has only 1 link remain; it means removing this directory would unlink the '..' inside, therefore reduce the link count of its parent directory to 0! Note that normally the link count of any directory should never be reduced below 2, and this UFS driver will give EINVAL if rmdir(2) would reduce the parent link count below 2; this behavior can already be seen in the first Unix shell interaction log above, where rm(1) failed to remove the last subdirectory 'usr', after '.' been removed first.
But I will not be stopped by this. I can load my KLD module to remove all the remaining directory hard links again:
[backdoor@zuo-freebsd /mnt/4]# kldload /usr/home/WHR/src/vfsunlink/vfsunlink.ko [backdoor@zuo-freebsd /mnt/4]# sysctl vfs.unlink=lost+found/. -> [backdoor@zuo-freebsd /mnt/4]# sysctl vfs.unlink=lost+found/.. -> [backdoor@zuo-freebsd /mnt/4]# sysctl vfs.unlink=lost+found sysctl: vfs.unlink=lost+found: No such file or directory [backdoor@zuo-freebsd /mnt/4]# ls -al total 0 [backdoor@zuo-freebsd /mnt/4]# ls -dl drwxr-xr-x 0 backdoor wheel 512 Feb 2 09:06 .
Of course by doing so the root directory link count of this file system becomes 0 again. But when I tried to step outside and stat(2) the mount point again, something weird happened:
[backdoor@zuo-freebsd /mnt/4]# cd [backdoor@zuo-freebsd ~]# ls -l /mnt/ ls: 4: Bad file descriptor total 23 drwxr-xr-x 4 backdoor wheel 512 Feb 1 17:33 1 drwxr-xr-x 4 backdoor wheel 512 Feb 1 17:33 2 drwxr-xr-x 2 backdoor wheel 104 Jan 1 1970 3 drwxr-xr-x 2 backdoor wheel 2 Jan 31 17:06 fd drwxr-xr-x 2 backdoor wheel 2 Mar 6 2019 fuse drwxr-xr-x 6 backdoor wheel 6 Jun 29 2020 net drwxr-xr-x 2 backdoor wheel 2 Mar 6 2019 piaoc drwxr-xr-x 2 backdoor wheel 2 Jun 28 2018 proc drwxr-xr-x 2 backdoor wheel 2 Jan 26 05:49 root drwxr-xr-x 2 backdoor wheel 2 Nov 2 2024 sysctl [backdoor@zuo-freebsd ~]# ls -l /mnt/4 ls: /mnt/4: Bad file descriptor [backdoor@zuo-freebsd ~]# ls -dl /mnt/4 ls: /mnt/4: Bad file descriptor [backdoor@zuo-freebsd ~]# umount /mnt/4 [backdoor@zuo-freebsd ~]# mount -t ufs /dev/md1 /mnt/4/ [backdoor@zuo-freebsd ~]# ls -dl /mnt/4/ ls: /mnt/4/: Not a directory [backdoor@zuo-freebsd ~]# ls -dl /mnt/4 ls: /mnt/4: Bad file descriptor
Ugh, EBADF? I have no idea what would that means...
Wait a second! Did Unix just deallocated the root directory inode?
Generally, when the last hard link to a file (inode) is removed, the inode is deallocated, releasing the inode as well as space it referenced for reuse. I don't need a manual inspection into this file system image to find out what exactly happened there, because fsck_ufs(8) later confirmed the fact:
[backdoor@zuo-freebsd ~]# time fsck_ufs -n /dev/md1 ** /dev/md1 (NO WRITE) ** Last Mounted on /mnt/4 ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ROOT INODE UNALLOCATED ALLOCATE? no real 0m0.034s user 0m0.001s sys 0m0.032s [backdoor@zuo-freebsd ~]# time fsck_ufs -y /dev/md1 ** /dev/md1 ** Last Mounted on /mnt/4 ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ROOT INODE UNALLOCATED ALLOCATE? yes MISSING '.' I=460 OWNER=backdoor MODE=40700 SIZE=36864 MTIME=Feb 2 09:12 2026 DIR=? FIX? yes ** Phase 3 - Check Connectivity UNREF DIR I=460 OWNER=backdoor MODE=40700 SIZE=36864 MTIME=Feb 2 09:12 2026 RECONNECT? yes NO lost+found DIRECTORY CREATE? yes DIR I=460 CONNECTED. PARENT WAS I=0 ** Phase 4 - Check Reference Counts UNREF FILE I=10 OWNER=backdoor MODE=100644 SIZE=638 MTIME=Feb 2 07:55 2026 RECONNECT? yes UNREF FILE I=457 OWNER=backdoor MODE=100755 SIZE=36 MTIME=Feb 1 07:42 2026 RECONNECT? yes UNREF FILE I=458 OWNER=backdoor MODE=100755 SIZE=231 MTIME=Feb 1 07:41 2026 RECONNECT? yes UNREF FILE I=459 OWNER=backdoor MODE=100644 SIZE=95384 MTIME=Feb 2 09:06 2026 RECONNECT? yes LINK COUNT DIR I=460 OWNER=backdoor MODE=40700 SIZE=36864 MTIME=Feb 2 09:12 2026 COUNT 1 SHOULD BE 2 ADJUST? yes UNREF FILE I=1333 OWNER=WHR MODE=100755 SIZE=455168 MTIME=Feb 1 07:22 2026 RECONNECT? yes UNREF FILE I=2410 OWNER=WHR MODE=100755 SIZE=963084 MTIME=Feb 1 07:20 2026 RECONNECT? yes UNREF FILE I=2411 OWNER=backdoor MODE=100755 SIZE=446640 MTIME=Feb 2 07:19 2026 RECONNECT? yes ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? yes SUMMARY INFORMATION BAD SALVAGE? yes 10 files, 526 used, 7289 free (9 frags, 910 blocks, 0.1% fragmentation) ***** FILE SYSTEM MARKED DIRTY ***** ***** FILE SYSTEM WAS MODIFIED ***** ***** PLEASE RERUN FSCK ***** real 0m0.045s user 0m0.000s sys 0m0.039s [backdoor@zuo-freebsd ~]# time fsck_ufs -y /dev/md1 ** /dev/md1 ** Last Mounted on /mnt/4 ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 10 files, 526 used, 7289 free (9 frags, 910 blocks, 0.1% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** real 0m0.037s user 0m0.000s sys 0m0.032s [backdoor@zuo-freebsd ~]# mount -t ufs /dev/md1 /mnt/4/ [backdoor@zuo-freebsd ~]# ls -ail /mnt/4/ total 19 2 drwxr-xr-x 3 backdoor wheel 4096 Feb 2 10:10 . 231 drwxr-xr-x 13 backdoor wheel 13 Feb 2 05:52 .. 3 drwx------ 3 backdoor wheel 4096 Feb 2 10:10 lost+found [backdoor@zuo-freebsd ~]# ls -ailR /mnt/4/ total 19 2 drwxr-xr-x 3 backdoor wheel 4096 Feb 2 10:10 . 231 drwxr-xr-x 13 backdoor wheel 13 Feb 2 05:52 .. 3 drwx------ 3 backdoor wheel 4096 Feb 2 10:10 lost+found /mnt/4/lost+found: total 4208 10 -rw-r--r-- 1 backdoor wheel 638 Feb 2 07:55 #0010 457 -rwxr-xr-x 1 backdoor wheel 36 Feb 1 07:42 #0457 458 -rwxr-xr-x 1 backdoor wheel 231 Feb 1 07:41 #0458 459 -rw-r--r-- 1 backdoor wheel 95384 Feb 2 09:06 #0459 460 drwx------ 2 backdoor wheel 36864 Feb 2 09:12 #0460 1333 -rwxr-xr-x 1 WHR wheel 455168 Feb 1 07:22 #1333 2410 -rwxr-xr-x 1 WHR wheel 963084 Feb 1 07:20 #2410 2411 -rwxr-xr-x 1 backdoor wheel 446640 Feb 2 07:19 #2411 3 drwx------ 3 backdoor wheel 4096 Feb 2 10:10 . 2 drwxr-xr-x 3 backdoor wheel 4096 Feb 2 10:10 .. /mnt/4/lost+found/#0460: total 80 460 drwx------ 2 backdoor wheel 36864 Feb 2 09:12 . 3 drwx------ 3 backdoor wheel 4096 Feb 2 10:10 ..
I was surprised to see fsck_ufs(8) successfully recovered a file system with the root inode deallocated. But on other hand, the old directory structure is completely gone; there's a freshly created root directory, without any of the original content.
But the store is not ending here. I'd like to capture a state of an UFS image with its root directory inode still there, but with a link count of 0. So I copied the previously dumped UFS image again, and start to recreate the situation:
(Long fsck_ufs(8) output...)
875 files, 5125 used, 2690 free (34 frags, 332 blocks, 0.4% fragmentation)
***** FILE SYSTEM STILL DIRTY *****
***** FILE SYSTEM WAS MODIFIED *****
***** PLEASE RERUN FSCK *****
real 0m0.139s
user 0m0.047s
sys 0m0.079s
[backdoor@zuo-freebsd ~]# time fsck_ufs -y /dev/md1
** /dev/md1
** Last Mounted on /
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
MISSING '.' I=2 OWNER=backdoor MODE=40755
SIZE=512 MTIME=Feb 2 07:55 2026
DIR=/
CANNOT FIX, FIRST ENTRY IN DIRECTORY CONTAINS 1
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
875 files, 5125 used, 2690 free (34 frags, 332 blocks, 0.4% fragmentation)
***** FILE SYSTEM MARKED CLEAN *****
real 0m0.042s
user 0m0.000s
sys 0m0.038s
[backdoor@zuo-freebsd ~]# mount -t ufs /dev/md1 /mnt/4/
[backdoor@zuo-freebsd ~]# cd /mnt/4/
[backdoor@zuo-freebsd /mnt/4]# ls -al
total 4003
drwxr-xr-x 13 backdoor wheel 13 Feb 2 05:52 ..
-rw-r--r-- 1 backdoor wheel 638 Feb 2 07:55 1
-rwxr-xr-x 1 WHR wheel 455168 Feb 1 07:22 inetrdr
-rwxr-xr-x 1 backdoor wheel 231 Feb 1 07:41 init
drwx------ 47 backdoor wheel 36864 Feb 2 10:22 lost+found
-rwxr-xr-x 1 WHR wheel 963084 Feb 1 07:20 ls
-rwxr-xr-x 1 backdoor wheel 446640 Feb 2 07:19 sysctl
-rwxr-xr-x 1 backdoor wheel 36 Feb 1 07:42 wait
[backdoor@zuo-freebsd /mnt/4]# ufs-readdir .
0 10 - 1
12 2 d ..
24 459 d lost+found
152 1333 - inetrdr
172 458 - init
188 457 - wait
204 2410 - ls
216 2411 - sysctl
[backdoor@zuo-freebsd /mnt/4]# time rm -rf lost+found/
ufs_rmdir: Bad link count 2 on parent inode 2 in file system /mnt/4
rm: lost+found/: Invalid argument
real 0m0.098s
user 0m0.000s
sys 0m0.064s
Ah! This time the UFS driver printed a warning onto my terminal, presumably using uprintf(9).
[backdoor@zuo-freebsd /mnt/4]# ls -ail total 4003 231 drwxr-xr-x 13 backdoor wheel 13 Feb 2 05:52 .. 10 -rw-r--r-- 1 backdoor wheel 638 Feb 2 07:55 1 1333 -rwxr-xr-x 1 WHR wheel 455168 Feb 1 07:22 inetrdr 458 -rwxr-xr-x 1 backdoor wheel 231 Feb 1 07:41 init 459 drwx------ 2 backdoor wheel 36864 Feb 2 10:23 lost+found 2410 -rwxr-xr-x 1 WHR wheel 963084 Feb 1 07:20 ls 2411 -rwxr-xr-x 1 backdoor wheel 446640 Feb 2 07:19 sysctl 457 -rwxr-xr-x 1 backdoor wheel 36 Feb 1 07:42 wait [backdoor@zuo-freebsd /mnt/4]# chroot . /ls -ail total 4008 2 drwxr-xr-x 2 0 0 512 Feb 2 07:55 .. 10 -rw-r--r-- 1 0 0 638 Feb 2 07:55 1 1333 -rwxr-xr-x 1 501 0 455168 Feb 1 07:22 inetrdr 458 -rwxr-xr-x 1 0 0 231 Feb 1 07:41 init 459 drwx------ 2 0 0 36864 Feb 2 10:23 lost+found 2410 -rwxr-xr-x 1 501 0 963084 Feb 1 07:20 ls 2411 -rwxr-xr-x 1 0 0 446640 Feb 2 07:19 sysctl 457 -rwxr-xr-x 1 0 0 36 Feb 1 07:42 wait [backdoor@zuo-freebsd /mnt/4]# chroot . /ls -ail lost+found/ total 80 459 drwx------ 2 0 0 36864 Feb 2 10:23 . 2 drwxr-xr-x 2 0 0 512 Feb 2 07:55 ..
Now the root directory have 2 links remain. Removing the last subdirectory 'lost+found' would reduce the link count below 2, an abnormal condition that should never happen. I will have to use vfs.unlink sysctl to remove all the remaining directory links. And because this file system is mounted at '/mnt/4/' rather than '/', the '/mnt/4/..' refers to the parent directory of the parent file system, I have to use chroot(8) to get access to the real '..' link. Note that '.' works here because of the magic of namei(9) that mentioned above.
[backdoor@zuo-freebsd /mnt/4]# sysctl vfs.unlink=lost+found/.
-> [backdoor@zuo-freebsd /mnt/4]# sysctl vfs.unlink=lost+found/..
-> [backdoor@zuo-freebsd /mnt/4]# sysctl vfs.unlink=lost+found
-> [backdoor@zuo-freebsd /mnt/4]# ufs-readdir .
0 10 - 1
12 2 d ..
152 1333 - inetrdr
172 458 - init
188 457 - wait
204 2410 - ls
216 2411 - sysctl
[backdoor@zuo-freebsd /mnt/4]# chroot . /ls -ail
total 3936
2 drwxr-xr-x 1 0 0 512 Feb 2 10:24 ..
10 -rw-r--r-- 1 0 0 638 Feb 2 07:55 1
1333 -rwxr-xr-x 1 501 0 455168 Feb 1 07:22 inetrdr
458 -rwxr-xr-x 1 0 0 231 Feb 1 07:41 init
2410 -rwxr-xr-x 1 501 0 963084 Feb 1 07:20 ls
2411 -rwxr-xr-x 1 0 0 446640 Feb 2 07:19 sysctl
457 -rwxr-xr-x 1 0 0 36 Feb 1 07:42 wait
[backdoor@zuo-freebsd /mnt/4]# chroot . /sysctl vfs.unlink=..
-> [backdoor@zuo-freebsd /mnt/4]# ls -al
total 0
[backdoor@zuo-freebsd /mnt/4]# ls -dl
drwxr-xr-x 0 backdoor wheel 512 Feb 2 10:24 .
[backdoor@zuo-freebsd /mnt/4]# chroot . /ls -ail
total 0
[backdoor@zuo-freebsd /mnt/4]# chroot . /ls -dl /
drwxr-xr-x 0 0 0 512 Feb 2 10:24 /
[backdoor@zuo-freebsd /mnt/4]# mount -u -o ro /mnt/4/
mount: /dev/md0: Device busy
[backdoor@zuo-freebsd /mnt/4]# umount /mnt/4/
umount: unmount of /mnt/4 failed: Device busy
[backdoor@zuo-freebsd /mnt/4]# sync
[backdoor@zuo-freebsd /mnt/4]# cp /dev/md1 /tmp/0-linked-root-for-real.ufs
[backdoor@zuo-freebsd /mnt/4]# ls -l /tmp/0-linked-root-for-real.ufs
-rw-r----- 1 backdoor wheel 33554432 Feb 2 10:25 /tmp/0-linked-root-for-real.ufs
[backdoor@zuo-freebsd /mnt/4]# file /tmp/0-linked-root-for-real.ufs
/tmp/0-linked-root-for-real.ufs: Unix Fast File system [v2] (little-endian) last mounted on /mnt/4, last written at Mon Feb 2 10:24:36 2026, clean flag 0, readonly flag 0, number of blocks 8192, number of data blocks 7815, number of cylinder groups 4, block size 32768, fragment size 4096, average file size 16384, average number of files in dir 64, pending blocks to free 0, pending inodes to free 0, system-wide uuid 0, minimum percentage of free blocks 2, SPACE optimization
The attempt to remount it read-only failed as expected; an inode with 0 links is in a pending-deallocation state, any file system containing such as inode cannot be unmounted or remounted read-only cleanly.
Does the UFS image '/tmp/0-linked-root-for-real.ufs' contain the expect state?
[backdoor@zuo-freebsd ~]# umount /mnt/4/ [backdoor@zuo-freebsd ~]# mdconfig -d -u 1 [backdoor@zuo-freebsd ~]# cp /tmp/0-linked-root-for-real.ufs /tmp/0-linked-root.ufs [backdoor@zuo-freebsd ~]# mdconfig -f /tmp/0-linked-root.ufs md1 [backdoor@zuo-freebsd ~]# mount -t ufs -o ro /dev/md1 /mnt/4/ [backdoor@zuo-freebsd ~]# ls -al /mnt/4/ total 0 [backdoor@zuo-freebsd ~]# ls -dl /mnt/4/ drwxr-xr-x 0 backdoor wheel 512 Feb 2 10:24 /mnt/4/
Yes. This file system is now mounted for read only, and it is currently in an inconsistent state by having an pending-deallocation inode, which happens to be the root directory inode.
But before trying to remount it writable, I found something unexpected:
[backdoor@zuo-freebsd ~]# ufs-readdir /mnt/4/
0 10 - 1
152 1333 - inetrdr
172 458 - init
188 457 - wait
204 2410 - ls
216 2411 - sysctl
[backdoor@zuo-freebsd ~]# cat /mnt/4/1
cat: /mnt/4/1: No such file or directory
[backdoor@zuo-freebsd ~]# cat /mnt/4/init
cat: /mnt/4/init: No such file or directory
[backdoor@zuo-freebsd /mnt/4]# stat /mnt/4/init
stat: /mnt/4/init: stat: No such file or directory
[backdoor@zuo-freebsd ~]# file /mnt/4/ls
/mnt/4/ls: cannot open `/mnt/4/ls' (No such file or directory)
How? These regular files are never being removed by any request. For example, the 'ls' file was accessible before, proven by successfully executing it via chroot(8); but now I could only get ENOENT.
It is pretty clear by this time, that as soon as I remounted this file system for read/write, the UFS driver will deallocate the root inode, rendering the file system completely useless:
[backdoor@zuo-freebsd ~]# mount -u -o rw /mnt/4/ mount: /dev/md1: R/W mount of /mnt/4 denied. Filesystem is not clean - run fsck.: Operation not permitted [backdoor@zuo-freebsd ~]# mount -u -o rw,force /mnt/4/ [backdoor@zuo-freebsd ~]# ls -al /mnt/4/ ls: /mnt/4/: Not a directory [backdoor@zuo-freebsd ~]# ls -al /mnt/4 ls: /mnt/4: Bad file descriptor [backdoor@zuo-freebsd ~]# cat /mnt/4 [backdoor@zuo-freebsd ~]# stat /mnt/4 stat: /mnt/4: stat: Bad file descriptor
But I still want to perform one more test for that file accessibility issue. So I repeated every step, with another copy of the original UFS image, to the point I removed every hard link to its root directory.
Then watch this:
[backdoor@zuo-freebsd /mnt/5]# cat init
#!/bin/sh
exec 0<> /dev/console 1>&0 2>&0
PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin
export PATH
mount -u -o rw /
ifconfig lo0 127.0.0.1/16 up
inetrdr -p tcp -b :299 -elf /bin/sh -i &
uname -srm
exec sh
[backdoor@zuo-freebsd /mnt/5]# mv init init
mv: rename init to init: No such file or directory
[backdoor@zuo-freebsd /mnt/5]# cat init
cat: init: No such file or directory
[backdoor@zuo-freebsd /mnt/5]# ufs-readdir .
0 10 - 1
152 1333 - inetrdr
172 458 - init
188 457 - wait
204 2410 - ls
216 2411 - sysctl
The 'mv init init' shell command is basically equivalent to a 'rename("init", "init");' call, which should be a no-op. Yet this operation rendered the file inaccessible. Similar for another regular file:
[backdoor@zuo-freebsd /mnt/5]# ls -l inetrdr
-rwxr-xr-x 1 WHR wheel 455168 Feb 1 07:22 inetrdr
[backdoor@zuo-freebsd /mnt/5]# chown backdoor inetrdr
[backdoor@zuo-freebsd /mnt/5]# file inetrdr
inetrdr: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 10.1, FreeBSD-style, not stripped
[backdoor@zuo-freebsd /mnt/5]# ./inetrdr
./inetrdr: Must specify -l or -c
Usage: ./inetrdr [-e] [-p {tcp|udp|sctp}] [-b [<address>:]<port>] {-l [-f] | -c <address>:<port>} <command> [<argument>] [...]
[backdoor@zuo-freebsd /mnt/5]# mv inetrdr inetrdr.
mv: rename inetrdr to inetrdr.: No such file or directory
[backdoor@zuo-freebsd /mnt/5]# ls -l inetrdr
ls: inetrdr: No such file or directory
[backdoor@zuo-freebsd /mnt/5]# ls -l inetrdr inetrdr.
ls: inetrdr: No such file or directory
ls: inetrdr.: No such file or directory
[backdoor@zuo-freebsd /mnt/5]# file inetrdr inetrdr.
inetrdr: cannot open `inetrdr' (No such file or directory)
inetrdr.: cannot open `inetrdr.' (No such file or directory)
[backdoor@zuo-freebsd /mnt/5]# ./inetrdr
bash: ./inetrdr: No such file or directory
[backdoor@zuo-freebsd /mnt/5]# ./inetrdr.
bash: ./inetrdr.: No such file or directory
It seems that inode changes such as chown(2) alone is fine, but operations that changes the hard link (thus would require updating the directory containing the link) such as rename(2), will break it.
I suspects the kFreeBSD VFS layer has cached the directory entries, when I first read the directory. As the link count of the directory reaches 0, the VFS layer started to block accesses to anything through the directory. My rename(2) attempts invalidated the associated cache, making the link to that regular file inaccessible.
In the end, I also tried the UFS driver in Linux and grub-mount(1) with my corrupted UFS images:
# mount -t ufs -o loop,ufstype=ufs2,ro /tmp/0-linked-root.ufs /mnt/1/
mount: /mnt/1: mount(2) system call failed: Stale file handle.
dmesg(1) may have more information after failed mount system call.
# mount -t ufs -o loop,ufstype=ufs2,ro /tmp/root-inode-gone.ufs /mnt/1/
mount: /mnt/1: mount(2) system call failed: Stale file handle.
dmesg(1) may have more information after failed mount system call.
# grub-mount /tmp/0-linked-root.ufs /mnt/1/
# ls -al /mnt/1/
total 1824
-r--r--r-- 0 root root 638 Feb 2 15:55 1
-r--r--r-- 0 root root 455168 Feb 1 15:22 inetrdr
-r--r--r-- 0 root root 231 Feb 1 15:41 init
-r--r--r-- 0 root root 963084 Feb 1 15:20 ls
-r--r--r-- 0 root root 446640 Feb 2 15:19 sysctl
-r--r--r-- 0 root root 36 Feb 1 15:42 wait
# file /mnt/1/*
/mnt/1/1: ASCII text
/mnt/1/inetrdr: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 10.1, FreeBSD-style, not stripped
/mnt/1/init: POSIX shell script, ASCII text executable
/mnt/1/ls: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 10.1, FreeBSD-style, with debug_info, not stripped
/mnt/1/sysctl: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, FreeBSD-style, for FreeBSD 11.1, not stripped
/mnt/1/wait: POSIX shell script, ASCII text executable
# umount /mnt/1/
# grub-mount /tmp/root-inode-gone.ufs /mnt/1/
error: not a directory.
Apparently Linux UFS driver rejected both images with ESTALE, but grub-mount(1) can mount the one with a zero-linked root directory; all regular files there can be read correctly, though the grub-mount(1) didn't support reporting the link count values.
So, if you use GRUB to boot up your system and use UFS for root, GRUB will still be able to read all of the critical files required for starting the kernel (provided those regular files are intact), even if you somehow managed to remove every hard link to its root directory, thus reduced the link count to 0.