Bug#1031696: Use of symbolic links in non-free ISO images breaks file system transposition support

Hi,

Just going to report that I have tested media creation from the 
2023-03-13 version of debian-testing-amd64-netinst.iso, using only the 
Windows native utilities (i.e. formatting a USB drive to FAT32 using 
Windows Disk manager and mounting the ISO and copying the files using 
Windows File Explorer) and I can report that everything looks good:

1. The .deb files in /firmware/ are now being listed by File Explorer, 
and therefore copied over.
2. The installer does successfully boot and detect the installation media.

Now, I don't have a platform where any of the /firmware/ files are 
relevant, so I haven't been able to validate that part, but seeing that 
the .deb are present and successfully copied over, I don't anticipate an 
issue.

Thus, as far as I am concerned, this bug can be closed, and I would like 
to thank everyone who devoted some of their time to ensure that a fix 
was applied in a very timely manner.


On 2023.03.15 07:48, Thomas Schmitt wrote:
> Besides such user pitfalls with the produced ISO and the problem of
> symbolic links there are other constraints which an ISO has to fulfill
> for this use case.
> At least:
> - The file tree of the FAT filesystem in the EFI partition needs to be
>    mirrored by a copy in the ISO filesystem.

Indeed. That used to be less of an issue when Linux installation media 
used to treat the whole media as the ESP, but the move towards using 
self-contained efi.img does indeed mean that FST will only work if the 
distro maintainers do take care of duplicating the efi.img content at 
the ISO-9660 file system level (or use a utility, like grub-mkrescue, 
that will do that for them).

As far as Debian is concerned, this is not currently an issue, as Debian 
does not use an efi.img.

> - File names must be unique in respect to case-insensitive comparison.

True, though I have yet to encounter any installation media where 
someone had named two file/folders in the same directory with the same 
name but different capitalization. I'm pretty sure that we can count on 
people mastering an image to avoid that, as it would be very confusing 
to have different files/folders at the same level, that differ only from 
capitalization.

So I think we can discount this as a corner case that's unlikely to be 
an issue.

> - (I am not sure whether file name length can be a problem.)

 From what I can see, the maximum individual file name length when using 
Joliet extensions is 128 "characters" (I'm not going to go into Unicode 
glyphs vs characters here) and 255 for Rock Ridge. The latter is also 
the maximum maximum individual file name length for FAT32 with long 
names. So, even if Debian tends to have fairly long names for some .deb 
packages, I don't anticipate much of an issue. And case sensitivity 
isn't an issue either, meaning that looking for a mixed case file name 
on FAT32 will resolve properly, even as the underlying file system is 
not exactly one that could be qualified as case sensitive.

> I guess Pete Batard can give a more comprehensive list.

Well, once you eliminate the search of installation media by label 
(which Debian doesn't do), the lack of symbolic links, which is what 
we've been dealing with here, is actually the biggest limitation of 
trying to work with FAT.

Otherwise, it's really the volume labelling constraints of FAT that's 
the most problematic, because, this time, you *must* use UPPERCASE and a 
few very limited subset of non alphanum characters for FAT labels, and 
you are also limited to 11 characters, which is very very short. The end 
result of this is that, pretty much any Linux distro that does a search 
of the installation media by label, and doesn't pay attention to file 
system transposition to FAT, is likely to use a regular mixed case label 
that is also longer than 11 characters, and you must alter the 
GRUB/Syslinux kernel options in the config files is you ever want that 
media to work with FST.

Finally, to be completely comprehensive, though this is not an actually 
constraint of the FAT file system, but one of Windows, I am going to 
point out that if you are using Windows' native utilities, you won't be 
able format a partition that is larger than 64 GB to FAT32, which can 
come as an issue for users who picked a large USB Flash Drive. However, 
you can either use non native utilities to format larger partitions to 
FAT32 (since this is not a limitation of the File System, just of the 
default Windows utilities) or you can always create a partition that is 
small enough to be formatted to FAT32 and leave the rest of the media free.

Regards,

/Pete