X-Recipient: archive-cygwin AT delorie DOT com
X-Original-To: cygwin AT cygwin DOT com
Delivered-To: cygwin AT cygwin DOT com
DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 8BE8B3858002
Authentication-Results: sourceware.org; dmarc=none (p=none dis=none)
 header.from=SystematicSw.ab.ca
Authentication-Results: sourceware.org;
 spf=none smtp.mailfrom=brian DOT inglis AT systematicsw DOT ab DOT ca
X-Authority-Analysis: v=2.4 cv=Yq/K+6UX c=1 sm=1 tr=0 ts=6034144a
 a=T+ovY1NZ+FAi/xYICV7Bgg==:117 a=T+ovY1NZ+FAi/xYICV7Bgg==:17
 a=IkcTkHD0fZMA:10 a=QweF5vw4ChSBs4rJw5EA:9 a=QEXdDO2ut3YA:10
Subject: Re: CRITICAL ls MEMORY LEAK
To: cygwin AT cygwin DOT com
References: <003401d70864$cd3b3400$67b19c00$@gmail.com>
 <CAJ1FpuMbx1pyOqoRSwgieQbGPgcXOOiUusFtsmqKWoyJdzF0pg AT mail DOT gmail DOT com>
From: Brian Inglis <Brian DOT Inglis AT SystematicSw DOT ab DOT ca>
Organization: Systematic Software
Message-ID: <306dd40d-666d-4a27-0a2c-dc03053d2f8c@SystematicSw.ab.ca>
Date: Mon, 22 Feb 2021 13:30:01 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.7.1
MIME-Version: 1.0
In-Reply-To: <CAJ1FpuMbx1pyOqoRSwgieQbGPgcXOOiUusFtsmqKWoyJdzF0pg@mail.gmail.com>
Content-Language: en-CA
X-CMAE-Envelope: MS4xfNfNaLWw93QGME0tBFfJgbj/ykOEenXt4cjLIKRkEsTcsSQ7K60C9ujUyFg18wtAyiFcv9vuutsMe+z+/Q7m3rMDr/KDh8tR4OCYlfr8gKYlaKYff97T
 OzADBfy+Rx9Y23H1s1uKueiyuQtnbUqVJFgenap+ClHiz+7PhLcR3m6rkDDeiC5YetX2ZzM+X0uC7/RyW7fZksC1wId7c6DaMwE=
X-Spam-Status: No, score=0.6 required=5.0 tests=BAYES_00, KAM_DMARC_STATUS,
 KAM_LAZY_DOMAIN_SECURITY, NICE_REPLY_A, RCVD_IN_BARRACUDACENTRAL,
 RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE,
 SPF_NONE, TXREP autolearn=no autolearn_force=no version=3.4.2
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on
 server2.sourceware.org
X-BeenThere: cygwin AT cygwin DOT com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
Reply-To: cygwin AT cygwin DOT com
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"; Format="flowed"
Errors-To: cygwin-bounces AT cygwin DOT com
Sender: "Cygwin" <cygwin-bounces AT cygwin DOT com>

On 2021-02-22 13:12, Doug Henderson wrote:
> On Sun, 21 Feb 2021 at 08:21, Satalink wrote:
>> I deal with a lot of very large files on a regular basis.  I've noticed that
>> when I delve into these directories using in mintty and issue the command ls
>> -l (or ls -color=auto),  a very large junk of memory is consumed.   The
>> memory leak seems to be proportionate to the number and size of files within
>> the containing folder.

> This is likely due to your virus scanner. If those files contain
> non-executable content, it is probably safe to disable virus scans for
> those files.
> 
> Something that ls does is triggering the scan. That scan causes the
> virus scanner to read the entire file. You should see extraordinary
> GPU and disk activity for some time after the ls completes. There
> might be processes or at least threads for each file being scanned.
> 
> Hopefully you will be able to identify a common folder in the path to
> those files where it is safe to disable scanning for that folder and
> all folders and files within that folder.

I've often wondered if the heavy activity is due to Windows' defaults to writing 
files with F+RX perms which triggers executable virus scans?

You can't have separate directory and file default perms and unlike Unix, 
Windows appears dumb about applying the X bit to files, probably because that 
would render downloaded executables non-X, and there is no easy way for users to 
change that, whereas Unix requires tools and users explicitly grant X perms.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple