DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 9CE3C3858D28
Date: Sun, 1 Sep 2024 04:24:25 +0900
To: cygwin@cygwin.com
Cc: Jim Reisert AD1C <jjreisert@alum.mit.edu>
Subject: Re: grepping a large file through a pipe takes eons
Message-Id: <20240901042425.702a5242c4bd5573ae993497@nifty.ne.jp>
In-Reply-To: <CAK-n8j6cjd5mHah6y1EVgbRsXLrdbati-j1QS1r1+aDc8jwg=g@mail.gmail.com>
References: <CAK-n8j6cjd5mHah6y1EVgbRsXLrdbati-j1QS1r1+aDc8jwg=g@mail.gmail.com>
Mime-Version: 1.0
Precedence: list
From: Takashi Yano via Cygwin <cygwin@cygwin.com>
Reply-To: Takashi Yano <takashi.yano@nifty.ne.jp>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: cygwin-bounces~archive-cygwin=delorie.com@cygwin.com
Sender: "Cygwin" <cygwin-bounces~archive-cygwin=delorie.com@cygwin.com>

On Sat, 31 Aug 2024 09:59:11 -0600
Jim Reisert AD1C wrote:
> Something has changed in the last month or two.  I have a very large
> file I am trying to grep (465 MB):
> 
> -rwxrw----+ 1 jjrei jjrei 465092052 Aug 31 09:39 all_spots.txt
> 
> 
> If I grep for something near the end of the file, the results return right away:
> 
> # time grep -n N0FUL all_spots.txt
> 
> 17027336:N0FUL,20240615,20240615,1
> 17027337:N0FUL,20240629,20240629,1
> 
> real    0m0.190s
> user    0m0.078s
> sys     0m0.078s
> 
> 
> If I pipe the file through cat, grep takes much longer:
> 
> # time cat all_spots.txt | grep -n N0FUL
> 
> 17027336:N0FUL,20240615,20240615,1
> 17027337:N0FUL,20240629,20240629,1
> 
> 
> real    1m4.934s
> user    0m0.031s
> sys     0m0.124s

Thanks for the report. This seems to be a regression of cygwin 3.5.4.
I'll submit a patch for this issue shortly.

-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple