Date: Mon, 29 Jun 1998 20:08:04 +0300 (IDT)
From: Eli Zaretskii <eliz AT is DOT elta DOT co DOT il>
To: djgpp-workers AT delorie DOT com
Subject: Windows 9X/NT file times
Message-ID: <Pine.SUN.3.91.980629200514.4204A-100000@is>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Precedence: bulk

You might remember the discussions related to the ``File has
modification time in the future'' warning of GNU Make.  In the course
of that discussion, I've heard several reports about all kinds of
weird behavior related to time stamps of files on Windows 9X and NT.
So I decided to investigate a bit myself.

This message summarizes what I've learned.  At the end of the message
you will find two test programs which I used to find out the facts.  I
would like to ask those who have access to Windows 9X/NT systems to
please run the programs and report results.  I only have access to
Windows 95 version 4.00.950 (where the above warning from Make doesn't
happen), so information about any other versions will be most
appreciated.  Thanks in advance.

Since libc functions only support DOS 2-sec granularity of file times,
I decided to use raw Int 21h calls to get the best time resolution I
can have.  The first test program below will allow you to display the 3
time stamps of every file on its command line.  At least on my version
of Windows, the last-access time is actually only the date (the time
is zero), and the modification time (aka last-write time) has DOS
2-sec granularity and therefore always lags behind the creation time
by a second or so.  Creation time is the only one which has 10-msec
resolution on this version of Windows.  I would like to know how later
Windows versions behave in this aspect.

Even this simple program already reveals some really strange
behavior.  Consider the following commands:

   C:\test>touch foo1 foo2 foo3 foo4 foo5 foo6 foo7 foo8 foo9
   C:\test>ftimes foo[1-9]
   foo1   06/29/1998 19:41:16.700  06/29/1998 00:00:00.000  06/29/1998 19:41:16.000
   foo2   06/29/1998 19:41:16.810  06/29/1998 00:00:00.000  06/29/1998 19:41:16.000
   foo3   06/29/1998 19:41:16.960  06/29/1998 00:00:00.000  06/29/1998 19:41:16.000
   foo4   06/29/1998 19:41:17.140  06/29/1998 00:00:00.000  06/29/1998 19:41:16.000
   foo5   06/29/1998 19:41:17.360  06/29/1998 00:00:00.000  06/29/1998 19:41:16.000
   foo6   06/29/1998 19:41:17.620  06/29/1998 00:00:00.000  06/29/1998 19:41:16.000
   foo7   06/29/1998 19:41:17.910  06/29/1998 00:00:00.000  06/29/1998 19:41:16.000
   foo8   06/29/1998 19:41:16.800  06/29/1998 00:00:00.000  06/29/1998 19:41:16.000
   foo9   06/29/1998 19:41:16.870  06/29/1998 00:00:00.000  06/29/1998 19:41:16.000

See how erratically the creation time changes?  Now, we *know* that
DJGPP programs process the argv[] array in sequence, so how the heck
did Windows manage to create `foo8' before `foo7'??  Any ideas?

The second program was written to gather statistics about differences
between the system clock and each one of the file time stamps.  Run
"w32time -h", and it will produce a (hopefully) self-explanatory usage
info.  Please play with the different options and tell me what results
did you see on which platforms.

Here's a sample output from my version of Windows 95:

     Statistics:   	   average           min              max
   -----------------------------------------------------------------------
   Creation time   	     +203msec	       +0msec	   +1.200sec
   Last-access time	+16514.577sec	-71144.000sec	-71131.000sec
   Last-write time 	     -997msec	   -1.989sec	      -49msec

(The large numbers in the Last-access row are because its time is
zero, so disregard it.)

Somebody reported on c.o.m.d. that if you call findfirst and record
file times, then after some time call findfirst again, you sometimes
will see different times.  I have never seen this on my system, but if
anybody can reproduce this, please tell the details.

I run the statistics program on NT, and got these results:

     Statistics:   	   average           min              max
   -----------------------------------------------------------------------
   Creation time   	   +1.108sec	     +131msec	   +2.101sec
   Last-access time	   +1.108sec	     +131msec	   +2.101sec
Last-write time 	   +1.108sec	     +131msec	   +2.101sec

So on NT, the file times are ahead of the system clock by more than 1
second on the average and by more than 2 sec(!!) in extreme cases.

Plain DOS doesn't have any surprises: the file time is always behind
the system clock, as you'd expect.

Here are the two programs.  Thanks in advance for any additional info.

------------------------ ftimes.c --------------------------
#include <errno.h>
#include <string.h>
#include <fcntl.h>
#include <time.h>
#include <dos.h>
#include <dpmi.h>
#include <go32.h>
#include <sys/farptr.h>
#include <libc/dosio.h>

void
ftime_convert (unsigned fdate, unsigned ftime, unsigned hundredths,
	       struct timeval *ftv)
{
  struct tm file_tm;

  memset (&file_tm, 0, sizeof file_tm);
  file_tm.tm_isdst = -1; /* mktime() determines if DST is in effect */
  file_tm.tm_mday = fdate & 0x1f;
  file_tm.tm_mon  = ((fdate >> 5) & 0x0f) - 1; /* 0 = January */
  file_tm.tm_year = (fdate >> 9) + 80;
  file_tm.tm_sec  = (ftime & 0x1f) * 2;
  file_tm.tm_min  = (ftime >>  5) & 0x3f;
  file_tm.tm_hour = (ftime >> 11) & 0x1f;
  ftv->tv_sec = mktime (&file_tm);
  ftv->tv_usec = hundredths * 10000L;
  if (hundredths > 100)
    {
      ftv->tv_sec  += 1;
      ftv->tv_usec -= 1000000L;
    }
}

int
fetch_file_times (int offset, struct timeval ftimes[])
{
  __dpmi_regs r;
  int i;
  long long times64b[3];

  for (i = 0; i < 3; i++)
    dosmemget (__tb + offset + i * sizeof (long long), sizeof (long long),
	       &times64b[i]);

  for (i = 0; i < 3; i++)
    {
      dosmemput (&times64b[i], sizeof (long long), __tb + offset);
      r.x.ax = 0x71a7;
      r.h.bl = 0;
      r.x.ds = __tb >> 4;
      r.x.si = offset;
      __dpmi_int (0x21, &r);
      if ((r.x.flags & 1))
	{
	  if (r.x.ax == 0x7100)
	    errno = ENOSYS;
	  else
	    errno = __doserr_to_errno (r.x.ax);
	  return -1;
	}
      ftime_convert (r.x.dx, r.x.cx, r.h.bh, &ftimes[i]);
    }
  return 0;
}

#include <stdio.h>

int
get_file_times (const char *pathname, struct timeval ftimes[])
{
  __dpmi_regs r;
  int use_lfn = _use_lfn (pathname);
  int pathlen;

  if (!pathname)
    {
      errno = EINVAL;
      return -1;
    }

  _put_path (pathname);
  pathlen = strlen (pathname) + 1;

  if (!use_lfn)
    {
      /* Set the DTA to the transfer buffer.  */
      r.x.ds = __tb >> 4;
      r.x.dx = (__tb & 0x0f) + pathlen;
      r.h.ah = 0x1a;
      __dpmi_int (0x21, &r);
    }

  r.x.ax = use_lfn ? 0x714e : 0x4e00;
  r.x.cx = 0;
  r.x.ds = __tb >> 4;		/* DS:DX -> ASCIZ file name */
  r.x.dx = __tb & 0x0f;
  if (use_lfn)
    {
      r.x.es = r.x.ds;
      r.x.di = r.x.dx + pathlen;
      r.x.si = 0;
    }
  __dpmi_int (0x21, &r);
  if ((r.x.flags & 1))
    {
      if (use_lfn && r.x.ax == 0x7100)
	errno = ENOSYS;
      else
	errno = __doserr_to_errno (r.x.ax);
      return -2;
    }
  if (use_lfn)
    {
      int offset = pathlen + sizeof (int);

      /* Free the search handle.  */
      r.x.bx = r.x.ax;
      r.x.ax = 0x71a1;
      __dpmi_int (0x21, &r);

      /* Get the file times from the transfer buffer.  */
      if (fetch_file_times (offset, ftimes) == -1)
	return -3;
    }
  else
    {
      unsigned short ft = _farpeekw (_dos_ds, __tb + pathlen + 0x16);
      unsigned short fd = _farpeekw (_dos_ds, __tb + pathlen + 0x18);

      ftime_convert (fd, ft, 0, &ftimes[0]);
      ftimes[1] = ftimes[2] = ftimes[0];
    }
  return 0;
}

void
usage (char *prog)
{
  printf ("Usage: %s file...\n\nPrints the file 3 times recorded in the filesystem\n", prog);
}

int
main (int argc, char *argv[])
{
  if (argc <= 1)
    {
      usage (argv[0]);
      return 0;
    }

  for (argc--, argv++; argc; argc--, argv++)
    {
      struct timeval ftime[3] = { {0U, 0L}, {0U, 0L}, {0U, 0L} };
      int i;
      char time_buf[80], *pbuf = time_buf;

      if ((i = get_file_times (argv[0], ftime)) < 0)
	{
	  perror (argv[0]);
	  fprintf (stderr, "Error code: %d\n", i);
	  fprintf (stderr, "times: %x %ld %x %ld %x %ld\n",
		   ftime[0].tv_sec, ftime[0].tv_usec,
		   ftime[1].tv_sec, ftime[1].tv_usec,
		   ftime[2].tv_sec, ftime[2].tv_usec);

	  return 1;
	}

      for (i = 0; i < 3; i++)
	{
	  pbuf += strftime (pbuf, sizeof (time_buf) - (pbuf - time_buf) - 1,
			    "  %m/%d/%Y %T.XXX", localtime (&ftime[i].tv_sec));
	  sprintf (pbuf - 3, "%3.3ld", (ftime[i].tv_usec + 500L) / 1000L);
	}
      printf ("%s %s\n", *argv, time_buf);
    }
  return 0;
}

--------------------------- w32time.c -----------------------
#include <errno.h>
#include <string.h>
#include <fcntl.h>
#include <time.h>
#include <dos.h>
#include <dpmi.h>
#include <go32.h>
#include <sys/farptr.h>
#include <libc/dosio.h>

void
ftime_convert (unsigned fdate, unsigned ftime, unsigned hundredths,
	       struct timeval *ftv)
{
  struct tm file_tm;

  memset (&file_tm, 0, sizeof file_tm);
  file_tm.tm_isdst = -1; /* mktime() determines if DST is in effect */
  file_tm.tm_mday = fdate & 0x1f;
  file_tm.tm_mon  = ((fdate >> 5) & 0x0f) - 1; /* 0 = January */
  file_tm.tm_year = (fdate >> 9) + 80;
  file_tm.tm_sec  = (ftime & 0x1f) * 2;
  file_tm.tm_min  = (ftime >>  5) & 0x3f;
  file_tm.tm_hour = (ftime >> 11) & 0x1f;
  ftv->tv_sec = mktime (&file_tm);
  ftv->tv_usec = hundredths * 10000L;
  if (hundredths > 100)
    {
      ftv->tv_sec  += 1;
      ftv->tv_usec -= 1000000L;
    }
}

int
fetch_file_times (int offset, struct timeval ftimes[])
{
  __dpmi_regs r;
  int i;
  long long times64b[3];

  for (i = 0; i < 3; i++)
    dosmemget (__tb + offset + i * sizeof (long long), sizeof (long long),
	       &times64b[i]);

  for (i = 0; i < 3; i++)
    {
      dosmemput (&times64b[i], sizeof (long long), __tb + offset);
      r.x.ax = 0x71a7;
      r.h.bl = 0;
      r.x.ds = __tb >> 4;
      r.x.si = offset;
      __dpmi_int (0x21, &r);
      if ((r.x.flags & 1))
	{
	  if (r.x.ax == 0x7100)
	    errno = ENOSYS;
	  else
	    errno = __doserr_to_errno (r.x.ax);
	  return -1;
	}
      ftime_convert (r.x.dx, r.x.cx, r.h.bh, &ftimes[i]);
    }
  return 0;
}

#include <stdio.h>

int
get_file_times (const char *pathname, struct timeval ftimes[])
{
  __dpmi_regs r;
  int use_lfn = _use_lfn (pathname);
  int pathlen;

  if (!pathname)
    {
      errno = EINVAL;
      return -1;
    }

  _put_path (pathname);
  pathlen = strlen (pathname) + 1;

  if (!use_lfn)
    {
      /* Set the DTA to the transfer buffer.  */
      r.x.ds = __tb >> 4;
      r.x.dx = (__tb & 0x0f) + pathlen;
      r.h.ah = 0x1a;
      __dpmi_int (0x21, &r);
    }

  r.x.ax = use_lfn ? 0x714e : 0x4e00;
  r.x.cx = 0;
  r.x.ds = __tb >> 4;		/* DS:DX -> ASCIZ file name */
  r.x.dx = __tb & 0x0f;
  if (use_lfn)
    {
      r.x.es = r.x.ds;
      r.x.di = r.x.dx + pathlen;
      r.x.si = 0;
    }
  __dpmi_int (0x21, &r);
  if ((r.x.flags & 1))
    {
      if (use_lfn && r.x.ax == 0x7100)
	errno = ENOSYS;
      else
	errno = __doserr_to_errno (r.x.ax);
      return -2;
    }
  if (use_lfn)
    {
      int offset = pathlen + sizeof (int);

      /* Free the search handle.  */
      r.x.bx = r.x.ax;
      r.x.ax = 0x71a1;
      __dpmi_int (0x21, &r);

      /* Get the file times from the transfer buffer.  */
      if (fetch_file_times (offset, ftimes) == -1)
	return -3;
    }
  else
    {
      unsigned short ft = _farpeekw (_dos_ds, __tb + pathlen + 0x16);
      unsigned short fd = _farpeekw (_dos_ds, __tb + pathlen + 0x18);

      ftime_convert (fd, ft, 0, &ftimes[0]);
      ftimes[1] = ftimes[2] = ftimes[0];
    }
  return 0;
}

int
extended_open (const char *file, int mode, int attrib, int action, int numtail,
	       int *action_taken)
{
  __dpmi_regs r;
  int use_lfn = _use_lfn (file);

  if (action_taken)
    *action_taken = 0;
  
  if (file == 0)
    {
      errno = EINVAL;
      return -1;
    }

  _put_path (file);

  r.x.ax = use_lfn ? 0x716c : 0x6c00;
  r.x.bx = mode;
  r.x.cx = attrib;
  r.x.dx = action;
  r.x.ds = __tb >> 4;
  r.x.si = __tb & 0x0f;
  if (use_lfn)
    r.x.di = numtail;
  r.x.flags = 1;		/* portability with DOS before v7.0 */
  r.x.ss = r.x.sp = 0;
  __dpmi_simulate_real_mode_interrupt (0x21, &r);
  if ((r.x.flags & 1))
    {
      if (use_lfn && r.x.ax == 0x7100)
	errno = ENOSYS;
      else
	errno = __doserr_to_errno (r.x.ax);
      return -1;
    }

  if (action_taken)
    *action_taken = r.x.cx;
  return r.x.ax;
}

#include <unistd.h>

typedef struct {	/* all diffs measured in milliseconds */
  long min_diff;
  long max_diff;
  long sum_of_diffs;	/* does not include extremal values */
  int  nsamples;
} TDiff_stats;

TDiff_stats tdiffs[3] = {	/* there are 86000 seconds in a day */
  {100000000L, -100000000L, 0L, 0},
  {100000000L, -100000000L, 0L, 0},
  {100000000L, -100000000L, 0L, 0},
};

void
add_time_diff_sample (long tdiff, TDiff_stats *td)
{
  td->nsamples++;
  switch (td->nsamples)
    {
      case 1:
	td->min_diff = td->max_diff = tdiff;
	break;
      case 2:
	if (tdiff > td->max_diff)
	  td->max_diff = tdiff;
	else if (tdiff < td->min_diff)
	  td->min_diff = tdiff;
	break;
      default:
	/* Add the sample to the statistics data.  */
	if (tdiff > td->max_diff)
	  {
	    td->sum_of_diffs += td->max_diff;
	    td->max_diff = tdiff;
	  }
	else if (tdiff < td->min_diff)
	  {
	    td->sum_of_diffs += td->min_diff;
	    td->min_diff = tdiff;
	  }
	else
	  td->sum_of_diffs += tdiff;
    }
}

void
gather_time_info (const char *file)
{
  struct timeval system_time, file_times[3];
  long sec_diff, usec_diff, diff_sample;
  int i;

  gettimeofday (&system_time, NULL);
  if (get_file_times (file, file_times) == -1)
    perror ("Cannot get file times");
  else
    {
      for (i = 0; i < 3; i++)
	{
	  sec_diff  = file_times[i].tv_sec - system_time.tv_sec;
	  usec_diff = file_times[i].tv_usec - system_time.tv_usec;
	  if (sec_diff > 100L || sec_diff < -100L)
	    diff_sample = sec_diff * 1000L; /* ignore usecs for large diffs */
	  else
	    diff_sample = sec_diff * 1000L + (usec_diff + 500L) / 1000L;
	  add_time_diff_sample (diff_sample, &tdiffs[i]);
	}
    }
}

#include <stdio.h>
#include <stdlib.h>

const char *
to_text (long val)
{
  char *buf = (char *)malloc (20);

  sprintf (buf,
	   val >= 1000L || val <= -1000L ? "%+5ld.%.3ldsec" : "%+9ldmsec",
	   val >= 1000L || val <= -1000L ? val / 1000L : val,
	   abs (val) % 1000L);
  return buf;
}

void
print_time_diff_statistics (void)
{
  printf ("  Statistics:   \t   average           min              max\n");
  printf ("-----------------------------------------------------------------------\n");
  printf ("Creation time   \t%s\t%s\t%s\n",
	  to_text ((tdiffs[0].sum_of_diffs + tdiffs[0].nsamples / 2 - 1)
		   / (tdiffs[0].nsamples - 2)),
	  to_text (tdiffs[0].min_diff), to_text (tdiffs[0].max_diff));
  printf ("Last-access time\t%s\t%s\t%s\n",
	  to_text ((tdiffs[1].sum_of_diffs + tdiffs[1].nsamples / 2 - 1)
		   / (tdiffs[1].nsamples - 2)),
	  to_text (tdiffs[1].min_diff), to_text (tdiffs[1].max_diff));
  printf ("Last-write time \t%s\t%s\t%s\n",
	  to_text ((tdiffs[2].sum_of_diffs + tdiffs[2].nsamples / 2 - 1)
		   / (tdiffs[2].nsamples - 2)),
	  to_text (tdiffs[2].min_diff), to_text (tdiffs[2].max_diff));
}

#include <io.h>

int
try_file (const char *file, int count, unsigned mode, unsigned action)
{
  int act;

  while (count--)
    {
      int fd = extended_open (file, mode, 0, action, 0, &act);

      if (fd < 0)
	{
	  perror ("_lfn_open");
	  return 1;
	}
      _write (fd, "foo1\r\n", 6);
      _close (fd);
      gather_time_info (file);
      remove (file);
      /* The following call to `usleep' is so that our samples are
	 evenly distributed inside the DOS 2-sec file times granularity.  */
      usleep (111111U);
    }

  return 0;
}

void
usage (const char *prog)
{
  printf ("\n\
       This program computes and displays the differences\n\
           between file times and the system clock\n\
\n\
Usage:  %s [count] [mode-bits] [action-spec]\n\
\n\
  COUNT is the number of times the program will compute the\n\
  differences before it prints the statistics (default: 100).\n\
\n\
  MODE-BITS can be a combination of the following bits:\n\
\n\
       ACCESS_READ-ONLY                 0000h\n\
       ACCESS_WRITEONLY                 0001h\n\
       ACCESS_READWRITE                 0002h (default)\n\
       ACCESS_READONLY_NOMODLASTACCESS  0004h\n\
\n\
       SHARE_COMPAT                     0000h (default)\n\
       SHARE_DENY_READWRITE             0010h\n\
       SHARE_DENY_WRITE                 0020h\n\
       SHARE_DENY_READ                  0030h\n\
       SHARE_DENY_NONE                  0040h\n\
\n\
       FLAGS_NOINHERIT                  0080h\n\
       FLAGS_NO_COMPRESS                0200h\n\
       FLAGS_NOCRITERR                  2000h\n\
       FLAGS_AUTO_COMMIT_ON_EVERY_WRITE 4000h\n\
\n\
\n\
  ACTION-SPEC can be one of these:\n\
\n\
       FILE_OPEN                        0001h\n\
       FILE_TRUNCATE                    0002h\n\
       FILE_CREATE                      0010h\n\
       OPEN_CREATE_IF_DOESNT_EXIST      0011h (default)\n\
       CREATE_TRUNCATE_IF_EXISTS        0012h\n\
\n",
	  prog);
}

int
main (int argc, char *argv[])
{
  static const char file1[] = "timetest.$$$";
  const unsigned
    default_mode = 0x0002,	/* RW, SHARE_COMPAT */
    default_action = 0x0011;	/* OPEN, CREATE if doesn't exist */
  unsigned mode = default_mode, action = default_action;
  const int default_count = 100;
  int count = default_count;

  if (argc > 1)
    {
      char *endp;

      if (strcmp (argv[1], "-h") == 0 || strcmp (argv[1], "?") == 0)
	{
	  usage (argv[0]);
	  return 0;
	}

      count = (int)strtol (argv[1], &endp, 0);
      if (*endp != '\0' || count < 0)
	{
	  count = default_count;
	  fprintf (stderr, "%s: invalid count; using %d instead\n",
		   argv[1], count);
	}
      count += 2;	/* 2 for extreme values excluded from the average */
      
      if (argc > 2)
	{
	  mode = (unsigned)strtoul (argv[2], &endp, 0);
	  if (*endp != '\0' || mode > 0x6fffUL)
	    {
	      mode = default_mode;
	      fprintf (stderr, "%s: invalid mode value; using 0x%x instead\n",
		       argv[2], mode);
	    }
	  if (argc > 3)
	    {
	      action = (unsigned)strtoul (argv[3], &endp, 0);
	      if (*endp != '\0' || (action != 0x12 && action != 0x11))
		{
		  action = default_action;
		  fprintf (stderr,
			   "%s: invalid action value; using 0x%x instead\n",
			   argv[3], action);
		}
	    }
	}
    }

  if (_get_dos_version (1) < 0x0400)
    {
      fprintf (stderr, "This program requires DOS version 4.0 or later\n");
      return 1;
    }

  if (try_file (file1, count, mode, action) == 0)
    {
      print_time_diff_statistics ();
      return 0;
    }
  return 1;
}