X-Recipient: archive-cygwin AT delorie DOT com DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:subject:to:references:from:message-id:date :mime-version:in-reply-to:content-type; q=dns; s=default; b=viK5 qSM/5EGcsonHgMi2is8OSPQqAfeGLzw7jSiopQQNbZ6iw4idadoPogNrsfY2INOA VhzujD39vR0lb2pCE9x7A9y2UxFpFILK1C7xUPtA04apl/C12gzWQbLFsxRU9K6P WeW0y5oYWHdBfhhlra9VTExdKUKkWlvL2mzgYPo= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:subject:to:references:from:message-id:date :mime-version:in-reply-to:content-type; s=default; bh=lvTMs75qLo gp42SqxSBkH8LiYUo=; b=q+olMQbXgA0k2QuojiWmjH1ndXRfHJ3O5bLQ4cNAiy CvGwkx7CcJX1osTV6OzaayzurRMAkCmNcQ49VzXaRgZrVocbcCgM0PcTWhJv+JKh sUi8Ntj3fB88e4ZP51EiHUYFm4gdjtweS0xQruvNfP3oIiaEpizpRMou2kCmtrEN c= Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-6.9 required=5.0 tests=BAYES_00,GIT_PATCH_2,SPF_HELO_PASS,T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 spammy=GOOD, HAD, price, wishes X-HELO: mx1.redhat.com DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 635746525 Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx10.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=eblake AT redhat DOT com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 635746525 Subject: Re: CR-LF handling behavior of SED changed recently - this breaks a lot of MinGW cross build scripts To: cygwin AT cygwin DOT com References: <0F7D3B1B3C4B894D824F5B822E3E5A175B2636E4 AT IRSMSX103 DOT ger DOT corp DOT intel DOT com> <0F7D3B1B3C4B894D824F5B822E3E5A175B26CE47 AT IRSMSX102 DOT ger DOT corp DOT intel DOT com> <59399CC5 DOT 60900 AT tlinx DOT org> From: Eric Blake Openpgp: url=http://people.redhat.com/eblake/eblake.gpg Message-ID: <417f84ac-5d9f-dc50-e912-973e90b8a128@redhat.com> Date: Thu, 8 Jun 2017 14:57:20 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.0 MIME-Version: 1.0 In-Reply-To: <59399CC5.60900@tlinx.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="aOpHDQtNx64G95usMMPfiwiGFD68eWlSB" X-IsSubscribed: yes --aOpHDQtNx64G95usMMPfiwiGFD68eWlSB Content-Type: multipart/mixed; boundary="bUAqd226t2nEAMgOXBdKvWftv5Ujg5kdO"; protected-headers="v1" From: Eric Blake To: cygwin AT cygwin DOT com Message-ID: <417f84ac-5d9f-dc50-e912-973e90b8a128 AT redhat DOT com> Subject: Re: CR-LF handling behavior of SED changed recently - this breaks a lot of MinGW cross build scripts References: <0F7D3B1B3C4B894D824F5B822E3E5A175B2636E4 AT IRSMSX103 DOT ger DOT corp DOT intel DOT com> <0F7D3B1B3C4B894D824F5B822E3E5A175B26CE47 AT IRSMSX102 DOT ger DOT corp DOT intel DOT com> <59399CC5 DOT 60900 AT tlinx DOT org> In-Reply-To: <59399CC5 DOT 60900 AT tlinx DOT org> --bUAqd226t2nEAMgOXBdKvWftv5Ujg5kdO Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 06/08/2017 01:51 PM, L A Walsh wrote: >> 1.) I build many (~ 50) unix libs and tools MinGW cross on cygwin from >> sources and this breaks many of the configure and other scripts.=20 > --- > But didn't one have to use 'sed -b' before, in order to > strip out CR's? No, the exact opposite. It used to be that you HAD to use 'sed -b' to preserve CRs on a binary mount; now binary mounts preserve CRs automatically, making 'sed -b' a no-op on binary mounts. (This is closer to Linux behavior, where 'sed' preserves CRs automatically because everything is binary mount, and 'sed -b' is a no-op). On text mounts, 'sed -b' allows you to preserve CRs where they would otherwise be stripped automatically. > I.e. wouldn't all the individual lib/tool maintainers have > been required to add '-b' to their sed scripts? Sort of. The problem was that it used to be difficult to write portable scripts that worked on Cygwin and non-cygwin and still dealt with CRs. That's because you could not rely on 'sed -b' existing (not all the world uses GNU sed, and POSIX doesn't require -b to exist). But if you omitted the -b on Cygwin, your data was silently corrupted. With the change back in February, now Cygwin sed defaults to POSIX behavior on binary mounts, and the ONLY people that still have to use 'sed -b' are those who use text mounts; while remembering that text mounts are not the default. > Seems either way, > you have the undesirability of forcing external products to change to > support cygwin. External products were being lazy by relying on cygwin to strip CR when they should have stripped it themselves. But 'sed -b' does NOT strip CR (it is the exact opposite, of keeping CR unstripped). >=20 > Whereas, what I'd wonder is, how you are supplying input to sed > in the first place? I.e. how did CR's get into the stream to begin with. > If you used cygwin and some tool on cygwin generated CR's into the output > stream, I'd think that'd be a problem (or bug). But if you are mixing > DOS/Win-progs w/cygwin, then you need to adapt the DOS/Win progs' > outputs to > not have CR in them. Exactly - it used to be you could be lazy and feed the DOS/Win prog output (with CRs) to cygwin, and cygwin would ignore the CR - but that laziness came at a price that it would silently corrupt data for someone that was not aware that they needed the non-portable 'sed -b' to preserve CR when operating on known-binary data. Yes, the change is forcing clients of external data to be more explicit about the CR in their data, but in my mind, that's a GOOD thing - it's always better to be explicit about intentions, and the new behavior is something YOU control by whether you pre-filter the data, and not something that sed FORCED on you by using text mode against your wishes. --=20 Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org --bUAqd226t2nEAMgOXBdKvWftv5Ujg5kdO-- --aOpHDQtNx64G95usMMPfiwiGFD68eWlSB Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJZOawgAAoJEKeha0olJ0Nq7JsH/1YmOy9SbE/Pj+CzgDIo3skV IPsdoTlUTbGbz6o/R3iLj17SXJe7ThK5EmqVSn3JxFTRtvlHIeWeqv5EyVbtyzqG ogK+WtUHksGekuUaf0J/RPaz7lh7M0h64iGaZjpWWDzFXm7p0Oynx5P3FMX8LZqx Lr/6leRgnofAgWD30dK/BAHJmqEnka+uHboE87nU1oSmdnSZ9UR9f066/C415ftb Uuy0KM71/r8oy+T98GNxuirGXxnA1EFJ4hcKg8KC6/xv95x5y6MkrqEwp9/hkUNJ pDPozXerBSKQt+CDbm+KGXP1C/5otHwUzgl5VvZKwZj7W0CKRl1TMA7SLSnrBWc= =zVfx -----END PGP SIGNATURE----- --aOpHDQtNx64G95usMMPfiwiGFD68eWlSB--