Re: path name limits

On 2002-10-24, Bart Oldeman wrote:

>>>> What are tha path name limits imposed on freedos? MS-DOS
>>>> has 66 and DR-DOS has 63, I need a few more :(.
>
> This wasn't me.
>
>>> Any _DOS_ (should) limits _path_ to 67 characters and name to 8.3
>>> characters. This gives 67+\+8+.+3=80 characters for full path.
>
> And this wasn't me.
>
> I wonder how you got Arkady and me confused. As if we were the same
> person and me having had lots of discussions with my alter ego :)
> Did you do the quoting manually?

I'm sorry, Bart, my fault. I was just a bit too tired...
Sh**, where's the coffee, now? ;-)

I am only subscribed to the digest version of the FreeDOS list
(since I want this format for archiving purposes, and have too
little bandwidth to receive both formats on my private account -
I don't want to use workplace infrastructure for personal interests).

Yes, all my mails get quoted, edited, and formatted manually.

Now, you might better understand, why I sometimes get quite frustrated
about some careless posters' HTML stuff and TOFU quotings in this list -
it costs the valueable time of other readers to skip or even delete
all this redundant stuff.

> The 10 reserved bytes are not reserved anymore -- they are used as
> access/creation times and dates by various FAT filesystems and
> documented in the MS specification.

Yes, I know, this imposes some problems and that's why I said: optional.
BTW. At source code level, it is optional in DR-DOS as well, although
the corresponding conditional is always set true.

(Actually, my original post was about giving a bit more background on
special cases of the maximum path length in DR DOS - to flesh out
a little, what you and Arkady already wrote. I think, it is always
good to be aware of non-obvious exceptions/extensions to the basic
MS-DOS/PC DOS concepts, so things can be taken into account in other
software developments. It was not directly about adding password
support to FreeDOS now, although I certainly like that idea, that's
why I will go into some better details here.)

Still, with a bit of heuristics it is possible to combine VFAT long
filenames and DR-DOS passwords on the same volume (except for on FAT32
volumes, IIRC) and distinguish between them. In 1999 I developed a
version of the DR-DOS kernel which could cope with both, but it was
too late to get published - the change was not more than maybe a few
dozend bytes, so it should be similarly easy in FreeDOS (assuming it
would otherwise support password protected files already).

> About file passwords. Are they truely secure? One would have to
> encrypt the file itself too, as far as I can see, because otherwise,
> given the nature of DOS as a real mode OS it will always be possible
> to circumvent it using BIOS int13 (and if DRDOS hooks the BIOS you
> could just change the int13 vector to use the original int).

No, the file contents is not encrypted, but it would be possible to
add this (more thoughts on this much further below).

DR-DOS hooks INT 13h as every DOS does (or should do), if for nothing
else, then to deblock multi-sector access to the disk in some cases,
and to first flush the cache in case someone bypasses DOS. Do you mean
INT 2Fh/AH=13h? At least on a secured system, the special $SECURE$
device driver traps this.

For clarification, although both features can be combined, system security
is an optional component of DR-DOS (to be installed during SETUP/INSTALL).
The default security module shipping with DR-DOS is only a single-user
security (and Personal NetWare, yet another component, can be used
to add some aspects of "multi-user", then), however, DR-DOS also works
with the multi-user security component as found in DR DOS "Panther"
and (in another form) in the Multiuser DOS family (including REAL/32).

The password feature in DR-DOS works with or without system security,
but without it, it provides only a very light protection, which may
do a good job against theft on occasion or just human error.
/With/ system security installed it is still nothing for really critical
tasks, just because - as you already pointed out - DOS is an open system.
Knowledgable people like you and me would be able to break into the
system, no question, just because in the worst case we could always
patch around in the running kernel or analyse the disk surface sector
by sector under a different OS. (Since this is so, I can also describe
some of the used mechanisms without risking to tell secrets - it just
won't compromise the system any more than it is compromisable by the
inherent limitations of security under DOS "as is". Still, I think,
it is a reasonable good protection system for use under DOS, much
better than having only the Read-Only and Hidden attributes to play
with (as under MS-DOS/PC DOS), and with a few changes it might become
even better with Linux underneath.)

Password protected files and directories under DR-DOS can easily
be accessed by just booting up any DOS-system, which is not
"password-aware". Just try MS-DOS or PC DOS (or FreeDOS).
But this obvious security loophole exists only as long as
the DR-DOS system security is not enabled.
If it /is/ enabled, you will not be granted access to the
secured partitions without entering the correct system
password. The security module (SECURITY.BIN/SECURITY.OVL)
is loaded as a special type of driver through the preload API,
then it hooks into the operating system (since it relocates
itself several times, this is quite sophisticated). You will
be prompted to enter the password while the system boots up,
and there's no way to bypass this (other then booting from
floppy and trying to patch around directly on the harddisk's
disk surface).
Booting other operating systems, you will not even see these
secured partitions, FDISK will show them as "NON-DOS" or
"DR-DOS secured" partitions (depending on which version of
FDISK you use), and (since Novell DOS 7+) even changing the
partition types back to something recognized by other OSes
won't help you any further (the old DR DOS 6.0 security
was easily vulnerable there).
System security will change the partition types 0xh to Cxh
or Dxh (remember the trick to misuse the C5h partition ID
to utilize up to 8 Gb of disk space for DR-DOS while not
wasting the remaining space for other OS?). In addition to
just changing the partition type, it scrambles the boot sectors
and, I think, parts of the FAT, so it is not easily possible
to bypass system security just by changing the partition type.
(With a fair amount of time and knowledge of the FAT file system
internals and advanced disk editors it /is/ possible to break
into the system and try to discover the seemlingly damaged
partitions, though. So, the system has inherent limitations,
but still convenient to have for home or office desktop use
and areas were its limited security is sufficient.)

Once you have logged in to the system (or system security
was not activated in the first place, so you did not have
to log in first), password protected files or directories
will occur has hidden files under MS-DOS/PC DOS, while under
DR-DOS they will occur as hidden files only with ATTRIB and
similar tools. Normal tools like DIR will show them just
as if they were usual files.

Now, in case you want to access a file, the kernel tries to
open the file and recognizes that it is password protected
by the info stored in the directory entry.
If the user has appended a password to the filespec using
the semicolon syntax, the hash of this password is compared
with the hash of the original password stored in the
directory entry. If they match, access is granted according
to the access permissions as detailed in another entry in
the ten reserved bytes in the directory entry. If you are
not permitted to open the file the way you try to open it,
an "access denied" error will be returned.
There are Read/Write/Delete/Execute permissions for each
of the three levels World/Group/Owner. When the multi-user
security is loaded, the system will also store the current
owner ID in yet another entry in the reserved area and it
will distinguish between these three hierarchy levels.
If you have a default DR-DOS system, all three levels will
be set to the same permission pattern, and only the owner
permissions will be used on query. The owner ID is zero on
a single-user system. The Execute flag works as a Read
permission in normal issues of DR-DOS, as this is only
used under DR FlexOS (and NetWare).
In case you tried to open the file for read-write access,
gave the correct password, and have read and write permissions,
everything is fine (given that the read-only file attribute
is not set - not to be mixed up with the permission flags,
this becomes more important in case the various file sharing
modes for concurrent file access (SHARE) have to be taken
into account as well).
If you would only have read permissions, you will receive an
error. If you would try to open the file in read-only mode,
you will be granted access to the file. If the password is
not specified or invalid, no access will be granted at all
(there's a "second choice" those, see below).

There are special hooks in the kernel, that is, the kernel
advertises all this stuff to optionally loadable external
modules (INT 2Fh/AX=102xh). If they are loaded, they can take
over and make a decision, otherwise the kernel does it itself.
This undocumented interface is vulnerable, but it is quite
cryptic and AFAIK it cannot easily be taken over by simple
TSRs or so because certain other conditions must be met
first. Anyway, it is definitely not 100% bullet-proof.
But, again, to get this far, you will already have to log in
to the system as a whole, so this only works, once the system
is already up and running. And since you can configure the
DR-DOS screen saver LOCK to ask you for a password (if you
don't specify one, it will compare your input with the global
password, which I will explain in a minute), running but
unattended systems are quite safe as well.
The default password handling in the kernel does not consume
much memory, a few hundred bytes, maybe. So this does not
cause any "bloating". This way, you can plug-in multi-user
or network security modules, or update the security mechanisms
with more sophisticated ones (interface with Linux for example). 

Anyway, if the password hashes do not match, there is one
fall-back mechanism so that you can still work with password
protected files from inside of applications which do not
support the semicolon syntax in any way (there always has
to be a fall-back system if you add extensions to DOS,
since there are always application programmers, which are
not aware of the extensions, and if you lock their applications
out completely, you won't gain much):

The kernel compares the file's password hash with the hash
of the so called global password. By default, this password
is not set, so you still won't get access to password
protected files (but you regain control to the system
in case this is used for the screen saver LOCK). However,
the external DR-DOS PASSWORD utility allows you to
set/change/unset the global password using the
PASSWORD /G and /NG options.
When the DR-DOS system security module is loaded and the
one-time-login feature is activated, the password you
used to log into the system automatically becomes the
global password.
Assume you would protect all your private files with the
password "Bart" and I would protect all my private files
with the password "Matthias", if you would log in to the
system as Bart, the system would only grant access to
all unprotected files, all files which are protected
with "Bart" (no need to specify "Bart" all the time,
since this is the global password), and all files,
for which you will explicitly specify a different
password using the semicolon syntax.
If I specify "Matthias" as a password when I log in,
I cannot access your files unless I know your password
and either change the global password to "Bart" or append
";Bart" to all the files I want to access.

(Of course, with multi-user security installed, group
associations have to be taken into account as well.
This is handled by an encrypted database maintained
by the security module. So this is outside the scope
of the kernel.)

When both, the individual file password and the global
password do not match the password associated with
the file, the kernel will return the error "password
protected" or "invalid password" (I would have to look
up the actual error number).
The calling application is now free to just display
the error and skip the file, return to the prompt, or
otherwise abort the action (similar to trying to open
a file for read-write which has the read-only file
attribute set), or it can display a prompt to enter
the correct password. Once it is entered, this will
be appended to the filespec using the semicolon syntax
and the application will try to open the file again.
If the password is correct, the kernel will now grant
access (according to the permission flags detailed
above), otherwise you will get the same error again.

On user level, there are two ways to assign passwords to
files and directories:

You can use the PASSWORD utility, which has a number of
options to set/change/unset file or directory passwords
(the /V option is not enabled in normal issues):

|PASSWORD R2.00 (981224) Display or change password protection level
|Copyright (c) 1987,1998 Caldera, Inc.  All rights reserved.
|
|PASSWORD [/Help] [@][d:][path][filename.ext] [/R|W|D|P|V|G[:password]] [/N][/S]
|
|  /R[:password] Password will be required to read, write or delete the file
|  /W[:password] Password will be required to write or delete the file
|  /D[:password] Password will only be required to delete the file
|  /P[:password] Password will be required for all access to the directory
|  /V[:password] Password will be required for all access to the volume label
|  /G[:password] Set global default password
|                Use '*' in place of password to request hidden entry: eg. /R:*
|
|  /N            Remove password protection from file
|  /NP           Remove password protection from directory
|  /NG           Remove global default password
|
|  /S            Operate on files or directories in subdirectories
|
|Multiple files may be specified on the command line.
|Use '@' to specify that the filename that follows is the name of a file list.

In DR DOS "Panther" and Multiuser DOS, there is also an XATTRIB
utility which looks similar to:

|XATTRIB R1.01 (921016)  Change file access permissions
|Copyright (c) 1991,1992 Novell Inc.  All rights reserved.
|
|XATTRIB [/Help] [@][d:][path]filename[.ext] [/switches]
|
|  /A:name       Change file owner to be name, where name is
|                the login name of the new owner
|  /B            Brief - display file name only
|  /D            Operate on directories, not files
|  /F            Display or change default file create permissions
|  /OGW:RWD      O=Owner, G=Group, and W=World (left of ':')
|                R=Read, W=Write, and D=Delete (right of ':')
|  /P            Prompt before operating on each file,
|                or paginate if displaying only
|  /S            Search for files in subdirectories
|  /U:name       Only operate on files with given owner or group name
|
|The file specification is the drive, path and name of file(s) for which
|access permissions are to be changed (wildcard and multiple filenames
|allowed). Use '@' to specify that the given file contains a file list.

Alternatively, file and directories will implicitly become
password protected when you create them while the given
filespec contains a password. In this case, the file will
get maximum protection, that is, you will need the password 
to read/write/delete the file/directory again. Example:

   COPY autoexec.bat protect.bat;mypwd

will create a file named PROTECT.BAT, which has the password
"mypwd" associated with it. (Again, this is handled at /kernel/
level, COPY (COMMAND.COM that is) will just try to open
a new file using the filespec "protect.bat;mypwd" not
knowing anything special about passwords.)
Further, the file will have the hidden attribute set
(this is just so that the file is not "easily visibible"
under MS-DOS/PC DOS, but has no further influence on the
actual password mechanism, if you remove the hidden attribute,
the file will still be password protected - still, I suggest
not to change this, as it is part of the pattern for the
heuristics, I mentioned above). To access the file, say type
it, you will either need to specify the password:

   TYPE protect.bat;mypwd

or you will have to set the global password to "mypwd" using
PASSWORD /G. Since many DR-DOS tools (like XCOPY etc.) already
know about passwords (how come? ;-), they have a simply special
case added and will prompt you for the password when they
receive the "password protected" error code, while most
3rd party tools will just display some kind of generic
"access denied" error, when you try to access such files.

The same applies to directories, for example:

   MD mydir;mypwd

will create a password protected directory \MYDIR\.
If you try to make \MYDIR\ the current directory, you
will have to make "mypwd" the glocal password first,
or just specify the password in the filespec:

  CD mydir;mypwd

CD just passes its argument to the corresponding kernel
function, which takes care of the password stuff.

BTW. This is also the reason, why, as I explained in my
previous post, a simple

 CD y:=c:\drdos

under DR DOS 3.31 - 6.0 (up to 1992) allows to implicitly
assign floating drives similar to

 SUBST y: c:\drdos

(Another neat trick with passwords is to password-protect
the kernel files themselves, so noone can accidently
overwrite your kernel files, whilst the kernel in its
initial bootstrap phase (before IBMDOS.COM is loaded)
can read the files without problems. I added a new option
to my internal version of DR-DOS SYS /W[:password]
to automatically assign passwords to the relevant files
during SYSing.)

---

On 2002-10-25, Tom Ehlert wrote:

>> DR DOS 3.31 - 6.0 should have no problems to traverse through
>> directory structures of any depth, at least when using
>> relative paths.
>
> I wonder, how you implement
>
> INT 21 - DOS 2+ - "CWD" - GET CURRENT DIRECTORY
>         AH = 47h
>         DL = drive number (00h = default, 01h = A:, etc)
>         DS:SI -> 64-byte buffer for ASCIZ pathname
> [...]
> for directory size >= 64

Valid question. Assume, you are in a overly-long directory like

C:\1\2\3\4\5\6\7\8\9\0\1\2\3\4\5\6\7\8\9\0\1\2\3\4\5\6\7\8\9\0\1\2\3\4\5\

Under DR DOS 6.0 (1991), the PROMPT $P$G will display

C:\4\5\6\7\8\9\0\1\2\3\4\5\6\7\8\9\0\1\2\3\4\5\6\7\8\9\0\1\2\3\4\5>

So, it is truncating the string at the beginning. INT 21h/AH=47h will
return with CY set and AX=0003h "path not found", the 64-bytes buffer
at DS:SI, however, will be filled with

"4\5\6\7\8\9\0\1\2\3\4\5\6\7\8\9\0\1\2\3\4\5\6\7\8\9\0\1\2\3\4\5",NUL

If we would not use directory names with a length of 1 here, the
kernel would truncate the string, so that it always starts with a
full directory name, even if it would not use up all 64 bytes of
the buffer then. So, you can be sure, you won't get cutted directory
names. Usually, the corresponding emulated CDS entry will contain

"C:\",NUL

no matter, if you are in a normal directory or in an overly-long directory
I haven't tried to fiddle around with this now - it may contain other
strings at certain times. Also, I haven't tried with DR DOS 3.31 - 5.0
now, but it should be the same (or almost the same) as with DR DOS 6.0.

Under the Windows 9x/SE/ME GUI, the corresponding VM's CDS array contains:

"C:\",NUL

as well - but only, when you are in an overly-long directory, otherwise
it will contain the usual contents. In the former case, INT 21h/AH=47h
returns with CY set and AX=000Fh "Invalid drive". The DS:SI buffer is
not updated. The LFN-API still provides the long name, of course.

I assume, but have not tested, plain MS-DOS would return the same error.

BTW. An easy way to run into the scenario of too deep directory structures
is to create a directory tree which almost reaches the maximum length,
and then rename one of the parent directory names (RENDIR etc.) so that
it becomes longer than before - bang. So, it's even a problem for
DOS-only users, not taking advantage of DR DOS or Windows 9x...

Well, definitely interesting to document all this stuff for RBIL.
I once sent Ralf a few - still incomplete - notes about the DR-DOS
CDS handling, but that must have been seven or more years ago and
it didn't show up in RBIL (probably my English was too bad at that
time ;-). I should probably restate all that stuff and submit it again,
but then, he still has much more than a meg or two of recent stuff
still in his queue at the moment... ;-)

---

On 2002-10-25, Arkady V.Belousov wrote:

> I suggest Matthias mean next feature:
>
>    setcurdir(path1);
>    open(filepath2);
>
> where path1 less than 67 and path1+filepath2 > 80 and filepath2
> is a relative path.

Yes.

---

On 2002-10-25, Steffen Kaiser wrote:

> You would need to make known to the kernel that the right-hand drive
> letter is a logical one, too, instead of a physical one. (Hence your
> reference to the flags??)

Yes.

> You can bypass these problems using UNC, e.g.:
>
>> Y: =
>> C:\1\2\3\4\5\6\7\8\9\0\1\2\3\4\5\6\7\8\9\0\1\2\3\4\5\6\7\8\9\0\1\2
>> Z: = Y:\A\B\C\D\E\F\G\H\I\J\K\L\M\N\O\P\Q\R\S\T\U\V\W\X\Y\Z
>
> Z: = \\?\-Y:\A\B\ ...
>
> It is actually derived from WinNT4's filename-mangle-bypass notation:
> \\?\C:\...

What do you mean by filename-mangle-bypass notation?

> Now you won't need to tag those entries particularily in order to let
> the kernel know that the right-hand drive letter is to look up as
> logical drive letter again (transitiveness).

Hm, I don't see how we could avoid that at the moment in one way or
the other. I'm probably missing some important aspect of your idea.
Before I start speculating, could you explain this in better
details, please?

> Because \\? is in invalid UNC path otherwise, you can play nice things
> with it. Also, because it's UNC now, most programs should keep their
> hands (er, bytes) away from these drives.

That /might/ be a good thing, indeed.

But I am not completely sure, if it would /always/ be a good thing
to have - I guess, this needs testing with some real world programs
like NC, XTREE, etc. If they could be fooled into believing the
right-hand drive letter (Y: in the example) would be a true drive
letter, but without starting to do nasty low level things with
it, it might even be good to let them use the entry as if it was
a normal one...
On the other hand, when they start making assumptions about which
partition drive Y: might be associated with, we might get into
trouble - but could they reasonable make such assumptions?

Right now, I just don't know. Any pro's and con's?

> I would actually prefer having something like this:
> You, the user, tell the kernel passwords beforehand, which are
> automatically applied to a range of files/directories on access. If
> you enter the wrong or none password, you get garbish. Easy, nifty,
> few traces, no checksums, no permission denied. However, would need
> block-encryption, passwords need to be kept in memory, automatization
> is limited, performance drop.

This sounds similar to the encryption method used by tools
like PKZIP. The idea is as follows: Don't store the password
(or a derivation code, which is used to perform the encryption,
or not even a checksum) anywhere, so noone can find that place
and read/reset/modify the value.
Only the user, who wrote the file knows the password, and if he
does not use the same password on any later access to the file,
the password, which is used as sieve for the encryption, will
only create binary garbage (always assuming we would use a
non-trivial encryption method). This can be very dangerous
for the stability of the data or the system, but at least
it does not compromise the security. It's the responsibility
of the calling application/user to take care of this.
If that password is still physically stored somewhere (not just
in the head of the user), it is again not in the responsibility
of the kernel to hide it, that's up to the user/application etc.
and since every application can store it somewhere else and
in a different format, it won't be easy to derive a pattern
for a global attack from it. If one password is found by an
intruder, the corresponding file is compromised, but not
the system. Is this more or less a description of what
you suggest?

As far as I see it, it would be possible to add something like
this to the existing "DR DOS password model" as well, it just
hasn't been done yet.
I think, the most difficult part would be the encryption itself,
which should be moved into an external module (maybe a DPMS
enabled driver, so it runs in Extended Memory in Protected Mode?).

I think, it would be very useful to have a new file-by-file flag
in the filesystem, something like ahypothetical file attribute
"Encrypted". If this attribute is set, the correct password
(1 to 8 characters) is required to recreate the original
contents of the file - the password itself is not stored
anywhere in the filesystem, but could be given in the filespec
similar to DR DOS access passwords. (This means, fragments of
the password may be floating around in memory or disk, but not
"inside" the filesystem.)

Hm, if we could assume the access password and the encryption
password could always be the same (so we could use the same
syntax for both of them), and if the encryption is done with
the original verbatim password and only the hash codes are
optionally stored in the filesystem (for the permission system),
you could still not derive the original password to decrypt
the file when you would find out the 16-bit hash (which is
stored on disk). Even resetting the hash on disk will break
the permission system, but not the encryption system. The
hash would be kind of a checksum over the password, but
with up to 8 bytes for the actual passwords and only 2 bytes
for the hash, I guess, this leaves enough false matches
to not allow unauthorized encryption. If the encryption
althorithm is good enough, it should not be possible to
derive any pattern from it, should it?

Of course, adding a new "Encrypted" file attribute will
break some low level disk tools, but maybe it could be
added to the access permissions, where 4 bits are still
unused. Well, this may break some heuristics to distinguish
between LFNs and password protected files, but it might
be possible to further refine the heuristics...
Not storing an "Encrypted" flag at all, you would have
the problem to tell apart, which files should get encrypted
and which should not. For example, encrypting the kernel
files or some other binary files would have fatal
consequences in normal use. So, how ever this flag will
have to be stored, it must be stored somewhere.
Another issue is that one file can only have one encryption
key (in this model), so it really seems like a good idea
to combine this with access permissions somehow, assuming,
that whoever is granted access to the file will then also
have the correct key to encrypt it.
(If someone breaks into the permission system without
actually knowing the password (only the hash) and then
only receives binary garbage, I guess, we can safely
assume this to be his own fault. ;-)

Greetings,

 Matthias

-- 
<mailto:[email protected]>; <mailto:[email protected]>
http://www.uni-bonn.de/~uzs180/mpdokeng.html; http://mpaul.drdos.org

"Programs are poems for computers."