X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org 919BB3858000
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1692059463;
	bh=rfillGIebWeVg2Zmzgqvbd7tLAobTtmeLHI+0Hvfr9Q=;
	h=Date:To:Cc:Subject:In-Reply-To:References:List-Id:
	List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe:
	From:Reply-To:From;
	b=hgmuxtytwDshUPVFsQcSr2wLQFAfa8T1zDlDgaLQV3o+D3RvlT9vv+3bEfWr2sRIY
	/GmFYWPl2HJG5APgnrRKJru14JGkrOeMr6yLWErwLvd/ngThSYkix8Rk+kP/QA9u6Z
	2Nff+CySf235zDqSV1Hhf4OREbIXWZfvFKHx6T2U=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.4.2 sourceware.org 97BF73858422
Date:	Tue, 15 Aug 2023 09:30:42 +0900
To:	cygwin AT cygwin DOT com
Cc:	=?ISO-2022-JP?B?GyRCJS0lYyVtJSYhISVeITwlLxsoQg==?= <github AT callow DOT im>
Subject:	Re: Pipes truncating data in cygwin from main and cygwin-3_4-branch
Message-Id:	<20230815093042.4c5fd1034837098aaff45e10@nifty.ne.jp>
In-Reply-To:	<13B0370E-B61A-44B9-A885-5FF1B8F4AC5F@callow.im>
References:	<13B0370E-B61A-44B9-A885-5FF1B8F4AC5F AT callow DOT im>
X-Mailer:	Sylpheed 3.7.0 (GTK+ 2.24.30; i686-pc-mingw32)
Mime-Version:	1.0
X-Spam-Status:	No, score=-5.2 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS,
	NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_PASS,
	TXREP autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version:	SpamAssassin 3.4.6 (2021-04-09) on
	server2.sourceware.org
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.29
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe:	<https://cygwin.com/mailman/options/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<https://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Takashi Yano via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>
Errors-To:	cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender:	"Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On Mon, 14 Aug 2023 20:51:39 +0900
$B%-%c%m%&!!%^!<%/(B wrote:
> I have a problem that looks like pipes are truncating data when I cat a file to my program$B!G(Bs stdin. A simple `cat foo | cat > bar` works fine. bar ends up identical to foo. It is more complicated than that. My application is doing this
> std::stringstream buffer;
> buffer << std::cin.rdbuf();
> std::istream* isp = &buffer;
> Initial reads after this work fine. Once the app has read everything up to the payload data in the file, it does
> 
> off_t dataStart = (off_t)(isp->tellg());
> isp->seekg(0, ios_base::end);
> off_t dataEnd = (off_t)(isp->tellg());
> dataSizeInFile = dataEnd - dataStart;
> The tellg result shows the size is significantly less than the actual file data. 43k less in a 170k file. It is seemingly being truncated somewhere.
> 
> Later the app does
> 
>    isp->seekg(0);
>    std::streambuf* _streambuf = (isp->rdbuf());
> and starts reading from _streambuf. All data read from _streambuf is gibberish.
> 
> The application code makes no distinction between a pipe and stdin redirection from a file. It just uses std::cin. stdin redirection still works.
> 
> I created a minimal reproducer. More on that in a moment.
> 
> I first encountered this in Git for Windows 2.41.0. I had no problem in previous versions. I reported this to the Git for Windows project. See https://github.com/git-for-windows/git/issues/4464. You can find the minimal reproducer over there. It consists of 2 parts, a script and a small c++ program. The script finds the size of the target file then cats it to the test program passing the file size as a command line option. The test program does what I have described above and compares the file size determined from the seek to the end with the provided size.
> 
> A G4W project member reports that the problem reproduces on vanilla Cygwin in the branches mentioned in the subject and that G4W and MSYS2 are on the cygwin-3_4-branch release train. He recommends reporting the bug to you, You can find his(?) full comment here <https://github.com/git-for-windows/git/issues/4464#issuecomment-1671137446>.

Your test case does not work in command prompt as well.

Try
type testfile | test-pipe sizeoftestfile
in command prompt. It will fail.

New pipe implementation since cygwin 3.4.x provides the pipes
more similar to pipe in command prompt for non-cygwin apps.

Since your test case is compiled with cl.exe, it is non-cygwin
apps.

-- 
Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -