DKIM-Filter: OpenDKIM Filter v2.11.0 delorie.com 47VJOvZK197940
Authentication-Results: delorie.com;
	dkim=pass (1024-bit key, unprotected) header.d=cygwin.com header.i=@cygwin.com header.a=rsa-sha256 header.s=default header.b=AdbFcyhY
X-Recipient: archive-cygwin AT delorie DOT com
DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org A7A0F3858D35
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1725132295;
	bh=1lEJNJr9ceJQeuRxdZo2WlVAgs4LVTixypaPfRD5lMg=;
	h=Date:To:Cc:Subject:In-Reply-To:References:List-Id:
	 List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe:
	 From:Reply-To:From;
	b=AdbFcyhYv5Jagfiaxd3sKtjfX0/PoNSAWG6a/BfU/u1jT3Svzs3L/Hc/SO3A0D439
	 VLBFmMnuFVlc4sCp3hotIR13Jml4GymORvUVm78fhGaxi5+778OEjVlyJlNiZISLWY
	 ti1yJQ8FiCFS5G+f0zlYMU63QWPgzdy14LublxLw=
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 9CE3C3858D28
ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 9CE3C3858D28
ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1725132272; cv=none;
 b=ubwWaQx4ANUZS8LIyQ5NYa9a4uzptc/hXgEt1jlgNAqVr+HRNVHLyD9O/owPSxdQam6b4gbmAW/IA4rwmcogFiemWTD4b9bLsjxuTNBbsIgEeNKfu4zYlTEZsWg8PfSEIsJ84v6Hqhabf0gqtwNSSH8tV1zuHnhSkqEthUGUztU=
ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key;
 t=1725132272; c=relaxed/simple;
 bh=Vm7z44JtMobENiUr/TzAr0tLG3gQWd7hVAvjJZj0DdY=;
 h=Date:From:To:Subject:Message-Id:Mime-Version:DKIM-Signature;
 b=TSyb5fe87Fqltq/YWLVLNpjU2swx2/HR3gKtlfw2tBnmNkc0Je8Syrztu9DiY8UZl+N552rhrH8bVF38D1BHlAaoz1kueqg9TlCN6jNTemsiQ1rBQxurYF4uruxABFP8BGqfw42ZUe4+Vzu+WxD2o0BitzL8m/7SDncBhtfwRek=
ARC-Authentication-Results: i=1; server2.sourceware.org
Date: Sun, 1 Sep 2024 04:24:25 +0900
To: cygwin AT cygwin DOT com
Cc: Jim Reisert AD1C <jjreisert AT alum DOT mit DOT edu>
Subject: Re: grepping a large file through a pipe takes eons
Message-Id: <20240901042425.702a5242c4bd5573ae993497@nifty.ne.jp>
In-Reply-To: <CAK-n8j6cjd5mHah6y1EVgbRsXLrdbati-j1QS1r1+aDc8jwg=g@mail.gmail.com>
References: <CAK-n8j6cjd5mHah6y1EVgbRsXLrdbati-j1QS1r1+aDc8jwg=g AT mail DOT gmail DOT com>
X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.30; i686-pc-mingw32)
Mime-Version: 1.0
X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00, DKIM_SIGNED,
 DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, NICE_REPLY_A, RCVD_IN_DNSWL_NONE,
 SPF_HELO_PASS, SPF_PASS, TXREP,
 T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.30
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From: Takashi Yano via Cygwin <cygwin AT cygwin DOT com>
Reply-To: Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: cygwin-bounces~archive-cygwin=delorie DOT com AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces~archive-cygwin=delorie DOT com AT cygwin DOT com>

On Sat, 31 Aug 2024 09:59:11 -0600
Jim Reisert AD1C wrote:
> Something has changed in the last month or two.  I have a very large
> file I am trying to grep (465 MB):
> 
> -rwxrw----+ 1 jjrei jjrei 465092052 Aug 31 09:39 all_spots.txt
> 
> 
> If I grep for something near the end of the file, the results return right away:
> 
> # time grep -n N0FUL all_spots.txt
> 
> 17027336:N0FUL,20240615,20240615,1
> 17027337:N0FUL,20240629,20240629,1
> 
> real    0m0.190s
> user    0m0.078s
> sys     0m0.078s
> 
> 
> If I pipe the file through cat, grep takes much longer:
> 
> # time cat all_spots.txt | grep -n N0FUL
> 
> 17027336:N0FUL,20240615,20240615,1
> 17027337:N0FUL,20240629,20240629,1
> 
> 
> real    1m4.934s
> user    0m0.031s
> sys     0m0.124s

Thanks for the report. This seems to be a regression of cygwin 3.5.4.
I'll submit a patch for this issue shortly.

-- 
Takashi Yano <takashi DOT yano AT nifty DOT ne DOT jp>

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple