X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 919BB3858000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1692059463;
	bh=rfillGIebWeVg2Zmzgqvbd7tLAobTtmeLHI+0Hvfr9Q=;
	h=Date:To:Cc:Subject:In-Reply-To:References:List-Id:
	 List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe:
	 From:Reply-To:From;
	b=hgmuxtytwDshUPVFsQcSr2wLQFAfa8T1zDlDgaLQV3o+D3RvlT9vv+3bEfWr2sRIY
	 /GmFYWPl2HJG5APgnrRKJru14JGkrOeMr6yLWErwLvd/ngThSYkix8Rk+kP/QA9u6Z
	 2Nff+CySf235zDqSV1Hhf4OREbIXWZfvFKHx6T2U=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 97BF73858422
Date: Tue, 15 Aug 2023 09:30:42 +0900
To: cygwin AT cygwin DOT com
Cc: =?ISO-2022-JP?B?GyRCJS0lYyVtJSYhISVeITwlLxsoQg==?= <github AT callow DOT im>
Subject: Re: Pipes truncating data in cygwin from main and cygwin-3_4-branch
Message-Id: <20230815093042.4c5fd1034837098aaff45e10@nifty.ne.jp>
In-Reply-To: <13B0370E-B61A-44B9-A885-5FF1B8F4AC5F@callow.im>
References: <13B0370E-B61A-44B9-A885-5FF1B8F4AC5F AT callow DOT im>
X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.30; i686-pc-mingw32)
Mime-Version: 1.0
X-Spam-Status: No, score=-5.2 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS,
 NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_PASS, SPF_PASS,
 TXREP autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Takashi Yano via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>
Content-Type: text/plain; charset="iso-2022-jp"
Content-Transfer-Encoding: 7bit
Errors-To: cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie DOT com AT cygwin DOT com>

On Mon, 14 Aug 2023 20:51:39 +0900
キャロウ　マーク wrote:
> I have a problem that looks like pipes are truncating data when I cat a file to my program’s stdin. A simple `cat foo | cat > bar` works fine. bar ends up identical to foo. It is more complicated than that. My application is doing this
> std::stringstream buffer;
> buffer << std::cin.rdbuf();
> std::istream* isp = &buffer;
> Initial reads after this work fine. Once the app has read everything up to the payload data in the file, it does
> 
> off_t dataStart = (off_t)(isp->tellg());
> isp->seekg(0, ios_base::end);
> off_t dataEnd = (off_t)(isp->tellg());
> dataSizeInFile = dataEnd - dataStart;
> The tellg result shows the size is significantly less than the actual file data. 43k less in a 170k file. It is seemingly being truncated somewhere.
> 
> Later the app does
> 
>    isp->seekg(0);
>    std::streambuf* _streambuf = (isp->rdbuf());
> and starts reading from _streambuf. All data read from _streambuf is gibberish.
> 
> The application code makes no distinction between a pipe and stdin redirection from a file. It just uses std::cin. stdin redirection still works.
> 
> I created a minimal reproducer. More on that in a moment.
> 
> I first encountered this in Git for Windows 2.41.0. I had no problem in previous versions. I reported this to the Git for Windows project. See https://github.com/git-for-windows/git/issues/4464. You can find the minimal reproducer over there. It consists of 2 parts, a script and a small c++ program. The script finds the size of the target file then cats it to the test program passing the file size as a command line option. The test program does what I have described above and compares the file size determined from the seek to the end with the provided size.
> 
> A G4W project member reports that the problem reproduces on vanilla Cygwin in the branches mentioned in the subject and that G4W and MSYS2 are on the cygwin-3_4-branch release train. He recommends reporting the bug to you, You can find his(?) full comment here <https://github.com/git-for-windows/git/issues/4464#issuecomment-1671137446>.

Your test case does not work in command prompt as well.

Try
type testfile | test-pipe sizeoftestfile
in command prompt. It will fail.

New pipe implementation since cygwin 3.4.x provides the pipes
more similar to pipe in command prompt for non-cygwin apps.

Since your test case is compiled with cl.exe, it is non-cygwin
apps.

-- 
Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple