X-Recipient: archive-cygwin AT delorie DOT com
X-Spam-Check-By: sourceware.org
From: Jim Meyering <jim AT meyering DOT net>
To: =?utf-8?Q?P=C3=A1draig?= Brady <P AT draigBrady DOT com>
Cc: Eric Blake <ebb9 AT byu DOT net>, cygwin AT cygwin DOT com,
   bug-coreutils <bug-coreutils AT gnu DOT org>
Subject: Re: "du -b --files0-from=-" running out of memory
In-Reply-To: <492C1512.9020706@draigBrady.com> (=?utf-8?Q?=22P=C3=A1draig?=  Brady"'s message of 	"Tue, 25 Nov 2008 15:09:06 +0000")
References: <nacii4p76633jbufvfoj4qjesrph05rjga AT 4ax DOT com> 	<49296551 DOT 4010801 AT byu DOT net> <87bpw5a5tp DOT fsf AT rho DOT meyering DOT net> 	<874p1v52od DOT fsf AT rho DOT meyering DOT net> <492C1512 DOT 9020706 AT draigBrady DOT com>
Date: Tue, 25 Nov 2008 17:56:22 +0100
Message-ID: <87myfn3ft5.fsf@rho.meyering.net>
Lines: 60
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
Precedence: bulk
List-Id: <cygwin.cygwin.com>
List-Unsubscribe: <mailto:cygwin-unsubscribe-archive-cygwin=delorie DOT com AT cygwin DOT com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by delorie.com id mAPGvJjg031312

Pádraig Brady <P AT draigBrady DOT com> wrote:
> Jim Meyering wrote:
>> Subject: [PATCH 1/2] argv-iter: new module
>>
>> * gl/lib/argv-iter.h: New file.
>> * gl/lib/argv-iter.c: New file.
>> * gl/modules/argv-iter: New file.
>
> Very useful module!
>
> I see that --files0-from was added to `du` in Mar 2004,
> so it's a nice solution to this 4 year old issue.

Thanks.
I'm surprised it took so long to bite.

> I notice that argv_iter does a malloc() + memcpy() per entry.
> Since the sources are already NUL terminated strings
> perhaps it could just return a pointer to a getdelim
> realloc'd buffer which was referenced in the argv_iterator struct.

The only per-entry allocation I see is:
  - in argv-mode: strdup
  - in stream-reading mode: getdelim

Did I miss something?

char *
argv_iter (struct argv_iterator *ai, enum argv_iter_err *err)
{
  if (ai->fp)
    {
      char *name = NULL;
      size_t buf_len = 0;
      ssize_t len = getdelim (&name, &buf_len, '\0', ai->fp);
      if (len < 0)
        {
          free (name);
          *err = feof (ai->fp) ? AI_ERR_EOF : AI_ERR_READ;
          return NULL;
        }

      *err = AI_ERR_OK;
      ai->item_idx++;
      return name;
    }
  else
    {
      if (*(ai->p) == NULL)
        {
          *err = AI_ERR_EOF;
          return NULL;
        }
      else
        {
          *err = AI_ERR_OK;
          return strdup (*(ai->p++));
        }
    }
}

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/