Mail Archives: opendos/2000/10/09/10:56:03
On Thu, 28 Sep 2000, Florian Xaver wrote:
> >Given the memory footprint of the shipping issue (with
> >EMS and DPMS support disabled) and the fact that it is not
> >cached by NWCACHE, this is not actually a solution.
>
> Why isn't it cached? Why can't nwcache cache it? I think, this
> is a "bug", because also smartdrv support caching a redirector.
Florian, you may call this a missing feature, but it definitely is
not a bug.
Caching volumes accessable through INT 13h and caching drives hooked
into the system via the redirector interface are rather different
tasks.
To understand the differences and the various implications, I have
to explain some details of the hierarchical driver model of the DOS
file and disk sub-system:
When DOS gets called to create/delete/open/read/write etc. a file or
a directory (via its qualified filespec), it will call the redirector
interface to inform external components of the request. If an external
driver hooking the redirector (like a network redirector, a CDEX, or
DRFAT32) detects that the request addresses one of the logical volumes
handled by that driver (e.g. by checking the drive letter in the
filespec), it will perform the function by itself (see below).
When the request is not handled by any such external component, it is
processed by the internal file-system logic built into DOS (which, of
course, only supports the FAT12/16 file-systems). All the pathspec
crunching, directory navigation, FAT allocation etc. is handled at
this level.
On the lower end of the FAT12/FAT16 file-system logic, any file
operatings are translated into read/write requests via the INT 25h/
INT 26h "Absolute disk read/write" interface. At this level, volumes
are addressed in terms of *logical* sectors (counted from 0 upwards,
depending on the size of the partition) on *logical* volumes (e.g.
0 = A;, 1: = B, 2 = C:, ...). The size of these sector buffers is 512
bytes. The DOS BIOS (IBMBIO.COM), which implements the low level disk
driver, will transform these requests into requests to the *physical*
medium, taking into account the relative starting position of the
corresponding partition on the physical (hard)disk, translating the
logical addressed sectors into the traditional Cylinder/Heads/Sectors
(CHS) addressing scheme (this time *physical* sectors a 512 bytes) and
pass this down to the ROM BIOS INT 13h level, which handles *physical*
media (e.g. 00h = 1st floppy, 01h = 2nd floppy, 80h = 1st harddisk,
81h = 2nd harddisk). Finally, the ROM BIOS actually performs the
physical read/write operations on the disk by directly accessing the
hardware.
For native FAT32 drives (MS-DOS 7.10+ and OEM DR-DOS 7.04+), this
works similar to FAT12/FAT16 except that DOS uses INT 21h/7305h
"Extended Absolute Disk Read/Write" instead of INT 25h/INT 26h, and
that the DOS BIOS will issue CHS or LBA calls at the INT 13h level
(MS-DOS up to including 7.0, and DR-DOS up to 7.03 will only issue the
traditional CHS calls).
This staggered architecture and in particular the abstraction level
of the redirector interface allow to "plug" foreign filesystems into
DOS, even for media which must not necessarily be available at the
INT 13h level.
For example, a network driver (using the redirector interface like
PNW's VLM) would intercept attempts to access network drives and
emulate the filesystem while the requests would actually be tunnelled
through the network protocol stack, transmitted through the network
hardware and may then touch the disk (or other media) on a remote
machine (for example using a non-FAT-filesystem on the remote server).
The redirector interface also allows to plug in filesystems like
HPFS, NTFS, ext2fs, FAT32 (and virtually anything you like) into DOS,
as long as these filesystems can be presented in a manner used to
access files under DOS (that is using the well-known filespec naming
conventions).
Depending on possibly advanced file-system concepts you may not be
able to make use of all its features under DOS, however, you *are*
able to access files and navigate in directory trees as if the volume
would be natively supported by DOS. The main problem with drivers
hooking the redirector interface is that they usually are memory hogs,
because they have to implement the whole filesystem logic (and more),
and usually advanced filesystems like HPFS, NTFS, or ext2fs are
designed for speed and not for minimal resource footprint. (A solution
to the memory footprint problem could be DPMS, Cloaking, etc.)
FAT32 is kind of an exception, because on one hand it is much more
that just a FAT filesystem with 32-bit placeholders in the FAT, but
on the other hand it is still not *that* different from FAT12/FAT16,
that parts of the FAT12/FAT16 code in the kernel could not be reused.
This is the reason, why native FAT32 makes sense (although it would
occupy ca. 3- 5 Kb of resident memory, and internal data structures
will have to change for FAT32, making the kernel incompatible with
DOS before 7.10), but native HPFS, NTFS, etx2fs etc. are simply
illusoric under DOS.
CDEX drivers like NWCDEX or MSCDEX use the redirector to plug in the
ISO 9660 filesystem, as it is used on CD-ROMs. At the lower level of
the ISO 9660 file system logic, the driver cannot use the INT 25h/
INT 26h level, because CD-ROMs use 2048 (and more) bytes per sector,
not 512 bytes. For the same reasons, these lower levels would not be
able to handle CD-ROM drives because they are not usually represented
at INT 13h (the exceptions are "El Torito" emulation modes in
conjunction with bootable CDs). This is why NWCDEX/MSCDEX directly
talks to a CD-ROM specific device driver (/D:OEMCD001 option). While
these device drivers are used to access drives, they still cannot be
original block device drivers (represented by drive letters - see
MEM /A) because of the incompatible nature of CD-ROMs. Instead, they
install themselfs as character device drivers (e.g. "OEMCD001"), but
they are only dummy character device drivers having a special
signature in their header that identifies them as *CD-ROM* device
drivers. NWCDEX verifies that this signature is OK before it
attempts to communicate with the driver by issueing CD-ROM device
driver specific commands.
The CD-ROM device driver handles all the hardware dependent stuff:
A generic CD-ROM device driver like ATAPI.SYS would directly access
the CD-ROM hardware on the corresponding IDE channel.
Older proprietary CD-ROM hardware required proprietary hardware-
specific drivers, e.g. the MTMCDAE/MTMCDAS.SYS drivers were used
for Mitsumi FX001 double-speed CD-ROMs back in the early 90th.
In the modular SCSI APSI architecture, a CD-ROM device driver like
SYMCD.SYS (for Symbios/LSI based host adapters) would pass the request
down to the ASPI manager (e.g. ASPI8xx.SYS for the SYM8xx family of
SCSI chips), which would physically access the SCSI controller to talk
to the SCSI CD-ROM drive.
Well, finally back to DRFAT32.EXE:
This is very similar to NWCDEX, but it implements a different file-
system: Not ISO 9660, but FAT32. At the lower end it talks to INT
21h/7305h to provide some means for caching, but hooks these functions
to get called back, so that the requests can be transformed into calls
to DRFAT32.SYS, a device driver that installs as character device
driver, but provides a number of DRFAT32 specific functions (see
INTER61: INT 21h/4402h"DRFAT32" and INT 2Fh/0802h for details). The
architecture is very similar to the CD-ROM device driver + CDEX
redirector model, but the implementation and the device driver
commands are unique (in fact early ALPHA issues of DRFAT32.EXE fully
emulated a CDEX, while the shipping DRFAT32.EXE 1.00 is just a
redirector, not a CDEX). Depending on the ROM BIOS support for LBA,
the used partition types, and DRFAT32.SYS configuration switches,
DRFAT32.SYS will translate/remap between CHS and LBA addressing
schemes and then call the ROM BIOS INT 13h.
> BTW: I have some questions to redirectors. Maybe you can answer it
> quickly:
> Is there any documentation on the web, how to make a redirector?
The redirector interface is not documented. The book "Undocumented
DOS" (2nd edition, Addison & Wesley) by Andrew Schulman et al is the
definite source. Search for the "Phantom RAM disk" in there.
Microsoft has documented the MSCDEX 2.1 interface (but only the CDEX
API, not the redirector API, and not most of the peculiarities of the
MSCDEX implementation, which make it so difficult to design a CDEX
being 100% compatible with MSCDEX). This should be available on
SimTel and other places. Search the sub-directories holding CDROM
drivers for something like MSCD*.*. (Sorry, I don't have a link
right now.)
For more accurate and up-to-date info always refer to Ralf Brown's
"Interrupt List":
Download the current issue INTER61, and search the INTERRUP.? files
for "CDEX" (INT 2Fh/15xxh), "redirector" (INT 2Fh/11xxh), and
"DRFAT32" (e.g. INT 21h/4402h and INT 2Fh/0802h). This will give
you a very good overview on the required functionality and some
implementation details (not on file system internals, of course).
> Why have all redirectory a sys driver and an exe file
> (like atapi.sys and nwcache or drFAT32.sys and drFAT32.exe)?
See above.
> Why it is so slow? Or isn't is so slow, only nwcache doesn't support
> redirectors?
Volumes handled by redirectors are not necessarily slower than volumes
handled by built-in drivers (if not taking the amount of optimization
into account). It just seems to be so, because redirector drivers
usually access relatively slow media like networks, CD-ROM devices,
or complex file systems like HPFS, NTFS, or FAT32. However, it is not
possible to cache at the redirector level itself, so you would need
a cache, that intercepts one of the lower levels.
For network drives, this could be some kind of packet buffering in
the protocol stack, for CD-ROM drives this is the device driver level.
For the DRFAT32 driver, this may be INT21/7305h (or INT 13h). (At the
INT 13h level, the cache must be specifically designed to look for
drives controlled by DRFAT32.SYS since the usual methods to retrieve
the information about the drive obviously don't work for DRFAT32.SYS.
This would be easy, but it needs special handling in the cache's
initalization code, so none of the currently existing caches will do
this. Also, the cache would need to handle both, LBA and CHS. Hence
the INT 21/7305h level is preferrable for DRFAT32.)
Similar to NWCDEX.EXE, DRFAT32.EXE itself implements a rudimentary
cache below the file system logic (/M:nnn). NWCDEX handles approx. 2Kb
sized CD-ROM sectors, but DRFAT32.EXE handles 512 byte sector buffers.
The total size of this buffering is 60 Kb (/M:120 for DRFAT32.EXE),
which is "almost nothing" for FAT32 drives, which usually have very
large File Allocation Tables (FATs) per se.
NWCACHE and (pre-MS-DOS 6.2 issues of SMARTDRV) cache drives at the
ROM-BIOS INT 13h CHS level, not at the higher INT 25h/INT 26h or the
block device driver level. SMARTDRV 4.2+ of MS-DOS 6.20+ seems
to cache at the INT 25h/INT 26h "Absolute disk read/write", and AFAIK
at the CD-ROM device driver level, when it caches CD-ROM drives. At
least, some 3rd party stand-alone caches for CD-ROMs (like CDQUICK)
for sure cache at the CD-ROM device driver level.
I don't know, if SMARTDRV of MS-DOS 7.10+ (Windows 95 OSR2 and later)
will cache FAT32 drives at the INT 21h/7305h level. If it would, this
may allow to cache DRFAT32 drives as well. I just never tried.
Someone willing to test this out?
Matthias
-------------------------------------------------------------------
Matthias Paul, Ubierstrasse 28, D-50321 Bruehl, Germany
eMail: <Matthias DOT Paul AT post DOT rwth-aachen DOT de>
Web : http://www.rhrz.uni-bonn.de/~uzs180/mpdokeng.html
-------------------------------------------------------------------
- Raw text -