X-Spam-Check-By: sourceware.org
Message-ID: <ae9f30a30702040918i5de90f9t48b332b564192521@mail.gmail.com>
Date: Sun, 4 Feb 2007 12:18:13 -0500
From: "Dave Wright" <dave AT jungledisk DOT com>
To: cygwin AT cygwin DOT com
Subject: Bug with AES key byte order in cygcrypto-0.9.8.dll?
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
X-Google-Sender-Auth: d78e02d1593951c3
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

I'm seeing a difference byte order handling between
cygcrypto-0.9.8.dll (0.9.8d-1) and cygcrypto-0.9.7.dll (0.9.7l-1) in
AES_set_encrypt_key (and set_decrypt_key).

Specifically, the 0.9.8 version doesn't appear to swap the byte order
into LE correctly. The behavior and key byte order of 0.9.7 matches
the other non-cygwin platforms I've tested on, including libcrypto
0.9.7 and 0.9.8 on Linux and OSX, so the problem appears to be
something specific to the cygwin build.

Test case:

unsigned char key[256] = {0xb6, 0x6d, 0x7b, 0x21, 0x71, 0x34, 0x2e,
0xe7, 0x99, 0xce, 0x20, 0x6d, 0x5f, 0x10, 0x26, 0x55,
  0x8e, 0xa8, 0xc3, 0xdf, 0xfc, 0x1a, 0x39, 0x59, 0x6a, 0x7c, 0x8f,
0xa3, 0x91, 0xa7, 0xbe, 0xd6};

AES_KEY kv;
AES_set_encrypt_key(key, 128, &kv);

print /x kv (on cygcrypto 0.9.8):
{rd_key = {0x217b6db6, 0xe72e3471, 0x6d20ce99, 0x5526105f,...
(on cygcrypto 0.9.7):
 {rd_key = {0xb66d7b21, 0x71342ee7, 0x99ce206d, 0x5f102655,...

..the first four values of rd_key SHOULD match the bytes of the key
(as in 0.9.7), but on 0.9.8 they have not been byte swapped correctly
for the x86 platform.

There are #defines in aes_locl.h that control the byte ordering, but
they appear to only do something different for MSVC compilers.

-Dave Wright

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/