Subject: B19: mprotect() bugs
From: [email-masked] (Gilbert W. Pilz Jr)

17 Apr 1998 22:05:36 -0700
Message-ID: <[email-masked]>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
To: [email-masked]

I'm new to this list so I don't quite know the correct way to do this. I
found two bugs in the mprotect() call. The most serious is that mprotect()
will not set the protections correctly since it calls VirtualProtect() (the
Win32 call) with its UNIX-arguement "prot" instead of the Win32-translated
protection "new_prot". The second bug occurs if you try to set a range of
memory to PROT_NONE. You will always fail with EINVAL. 

Below is the diff between the original and fixed versions.

*** mmap.cc.orig	Tue Feb 10 19:14:59 1998
--- mmap.cc	Thu Apr 16 15:40:32 1998
***************
*** 273,279 ****
    syscall_printf ("mprotect (addr = %x, len = %d, prot = %x)\n",
                    addr, len, prot);
  
!   if (prot & PROT_NONE)
      new_prot = PAGE_NOACCESS;
    else 
      {
--- 273,279 ----
    syscall_printf ("mprotect (addr = %x, len = %d, prot = %x)\n",
                    addr, len, prot);
  
!   if (prot == PROT_NONE)
      new_prot = PAGE_NOACCESS;
    else 
      {
***************
*** 298,304 ****
           }
       }
  
!   if (VirtualProtect (addr, len, prot, &old_prot) == 0)
      {
        __seterrno ();
        syscall_printf ("-1 = mprotect (): lasterror = %x\n", GetLastError
());
--- 298,304 ----
           }
       }
  
!   if (VirtualProtect (addr, len, new_prot, &old_prot) == 0)
      {
        __seterrno ();
        syscall_printf ("-1 = mprotect (): lasterror = %x\n", GetLastError
());

Gilbert W. Pilz Jr.
Systems Software Consultant
[email-masked]
www.scruz.net/~gwpilz/
-
For help on using this list (especially unsubscribing), send a message to
"[email-masked]" with one line of text: "help".