Message-ID: <4A470ADF.5010306@cwilson.fastmail.fm>
Date: Sun, 28 Jun 2009 02:17:03 -0400
From: Charles Wilson <cygwin AT cwilson DOT fastmail DOT fm>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.22) Gecko/20090605 Thunderbird/2.0.0.22 Mnenhy/0.7.6.666
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: Re: [RFC] jpeg library
References: <4A466538 DOT 8030007 AT cwilson DOT fastmail DOT fm> <4A46FAA4 DOT 2 AT users DOT sourceforge DOT net>
In-Reply-To: <4A46FAA4.2@users.sourceforge.net>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com

Yaakov (Cygwin/X) wrote:
> On 27/06/2009 13:30, Charles Wilson wrote:
>> Well, I no longer need to deal with that sort of imagery, and "private"
>> never really was very private, and it DID cause lots of people grief.
> 
> Moreso, the patch removed the height_in_blocks/width_in_blocks members
> from struct jpeg_component_info, which are used in a transupp.c file
> which is part of several desktop image viewers (I know of at least six),
> as well as the progressive_mode member from the jpeg_compress_struct
> struct, which is used in gtk+-2.12 and up.

This is precisely what I'm talking about.  Those fields in struct
jpeg_component_info are explicitly marked private in jpeglib.h:

  /* Remaining fields should be treated as private by applications. */

But they ARE PRESENT in that file, so there is an argument as to exactly
how private they really are. (There's a well established technique for
using opaque pointers to truly private data structures; why doesn't
libjpeg use those techniques?)

So, arguably, those six desktop image viewers as well as gtk+2.12 are
being slightly naughty.

But that's all by the by.  I want to revert the lossless patch going
forward; the question here is how to manage the transition.

>> And, our build environment for packages is much nicer now than it was in
>> the early days, so any Joe who needs lossless jpeg can easily build it
>> themselves.  So, it'd be nice to get rid of the lossless jpeg patch.
> 
> +1
> 
>> Why shouldn't we get rid of it?  Well, because over the years those
>> other clients have added lots of workarounds to accommodate cygwin's
>> jpeg library.  If we removed the lossless jpeg stuff, then they wouldn't
>> need them -- but until they too removed their workarounds, their builds
>> would break on cygwin again!
> 
> What is the chance of breakage for regular clients?

You mean for clients that aren't naughty, and do not/never did access
these "private" fields?  None, as far as I can tell.  The *size* of the
struct doesn't change. Only the names of some of the fields (not their
type), and their meaning.  Some purely internal structs change
significantly, but that shouldn't affect external clients, even naughty
ones.

Although the actual strings associated with various error code numbers
changes, so that could lead to some surprising incorrect error messages
until clients are recompiled.

In any event, if I release a new cygjpeg-62.dll (same as current name)
without the lossless jpeg patch, and it DOES break everything in sight,
well then, I'll withdraw it and we'll go with the plan B transition.

>> I propose to release a new libjpeg for cygwin-1.7 using gcc4 with the
>> shared libgcc runtime, once both cygwin-1.7 and gcc4 are officially
>> non-beta (e.g. moved to curr, not test).  This libjpeg will be named:
> 
> I presume that an upstream (ABI) version bump -- the easiest time to
> make changes like this -- is not in sight.

Nope. Upstream development is DEAD.  There was some flurry of activity
about two years ago, but it never went anywhere.   If IJG's libjpeg
wasn't so widespread and widely used, I'd want to look at some other
library that supports the format...

>> just like the current packages, where N>  20  The DLL number, name, and
>> package name will NOT be incremented.  Existing packages that use
>> libjpeg, AND that "illegally" accessed the so-called "private" data
>> members, will break and will need to be recompiled, without whatever
>> internal workarounds they had previously implemented to deal with our
>> lossless stuff.
> 
> May I suggest making these available for a limited-time manual download
> for maintainers to test and, if necessary, rebuild their own packages to
> be ready when these are released.

Well, "test" release, sure.  But given the interlocking dependencies of
libraries (esp. graphics libraries), you'd need an ever-increasing
number of these libraries in pending/test state. I don't have the
bandwidth to manage that outside of the sourceware facilities.  Since we
have a perfectly good mirror system and "test status" already available
for such things...

>> I could be persuaded to release it earlier.  That is, before cygwin-1.7
>> goes live although I'd rather not cause instability in the distro this
>> close to cygwin-1.7's release.  Or, sometime after cygwin-1.7.1, but
>> before gcc-4.x goes gold.
> 
> I think earlier is better, but it should be coordinated.

Well, with 1.7 due out sometime in the next week or so, I really don't
want to destabilize that.

>> The downside of this approach is cygwin's jpeg DLL would have a
>> different name than is normally implied by the stock build machinery, and
>>    a) we would continue to have to patch our jpeg builds to use the new
>> numbering sequence in perpetuity
> 
> Which is one reason I was against gcc4-ABI-bumps in the first place.

Where I begin acting as the devil's advocate:

Yeah, but on the other hand, every distro out there has a stack of
patches for libjpeg, because upstream is dead but the code has bugs.
Thus, debian has a list of libjpeg patches that, for all intents and
purposes, they must now "maintain in perpetuity".

That's the danger of such heavy reliance on a dead project; no place to
send the fixes.  So either live with the bugs, or maintain external patches.

This would not be all that different -- and we, too, have a stack of
bugfix patches for libjpeg (extracted from debian).

But this isn't a "pseudo-incompatibility" caused by a simple switch to
gcc4/shared-libgcc. This is a real ABI violation we're contemplating.
It's exactly what DLL version number bumps are FOR.

>>    b) any applications that dlopen libjpeg would need patching, in
>> perpetuity. I'm not sure this is much of an issue.
> 
> Can't think of any off the top of my head, but it's possible.

Yeah, I couldn't either. libjpeg isn't really the sort of library you
dlopen, is it?  Besides, does this hypothetical app know already about
our 'cyg*-n.dll' naming convention? If not then it's already broken and
needs patching for cygiwn. If so, then it already has cygwin-specific
workaround code that could be tweaked.

--
Chuck


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple