Mail Archives: cygwin/2006/12/13/14:18:15
Christopher Faylor wrote on Sunday, December 10, 2006 1:34 AM:
> On Sun, Dec 10, 2006 at 01:21:19AM -0500, Christopher Faylor wrote:
>> On Sat, Dec 09, 2006 at 11:31:05AM -0800, Karr, David wrote:
>>> If the point of this note is to get your pipeline to work, would it
>>> help if you added something like "-n 30" to the xargs command? This
>>> should execute one instance of 'grep -l -e "error '1234567890'"' for
>>> every 30 lines of output from the previous pipe entry.
>>>
>>> $ echo path/* | \
>>> tr ' ' '\n' | \
>>> grep -v -e '\*' | \
>>> xargs -r grep -l -e "error '1234567890'" | \
>>> rest_of_pipe
>>
>> I'm sure that the point of the note was to report a Cygwin bug.
>
> ...but I can't duplicate this with the latest snapshot...
Sorry for my delayed follow-up, but between family duties and work ...
Below is a test that looks at length of the command line and the number
of arguments. I started out by looking for the exact line length that
would cause problems. The line length that resulted well was under 32k
so I started looking at the number of arguments. I initially came up
with a number in the 5000s. It initially seemed to be consistent but
after I took a break, did some things in other bash shells, and started
up again (I think I was in the same shell, the number changed.
As you can see below, I'm now getting errors with half as many
arguments. (Different machines (laptop vs. desktop) but the setups
should be very close, if not identical.) Especially interesting, is
that as the line length gets longer, a number of arguments that had been
giving error stops.
Also, inserting tr ' ' '\0' between gawk and xargs and adding -0 to
xargs shifted the point where the error start down one from 2868/2869 to
2867/2868.
As for my scripts, changing the directory from which I was doing the
echo path/* fixed things, though now that I've done these experiments, I
may switch to using the -s or -n options of xargs. Thanks for the
suggestions.
And a big thanks in general to all the cygwin developers and package
maintainers and in particular to whoever may decide to debug this
(presuming that this is a problem with cygwin or one of its packages and
that one of the volunteers decides to debug it). And if this is not a
cygwin/package problem, I'd still appreciate any hints about what I
might be doing wrong.
- Barry
===
for X in 1 2 4 8 16
do
for A in 2968 2969
do
echo
echo $X $A | \
gawk '{
for (N = 1; N <= $1; N++) { x = x "x" }
x = " " x
for (N = 1; N <= $2; N++) { o = o x }
print "arg len: " $1 ", no args: " $2 ", tot len: "
length(o) - 1 > "/dev/stderr"
print substr(o,2)
}' | \
xargs -r echo | \
wc | \
grep -v -e ' 0 * 0 * 0'
done
done
arg len: 1, no args: 2968, tot len: 5935
1 2968 5936
arg len: 1, no args: 2969, tot len: 5937
xargs: echo: Argument list too long
arg len: 2, no args: 2968, tot len: 8903
1 2968 8904
arg len: 2, no args: 2969, tot len: 8906
xargs: echo: Argument list too long
arg len: 4, no args: 2968, tot len: 14839
1 2968 14840
arg len: 4, no args: 2969, tot len: 14844
xargs: echo: Argument list too long
arg len: 8, no args: 2968, tot len: 26711
1 2968 26712
arg len: 8, no args: 2969, tot len: 26720
1 2969 26721
arg len: 16, no args: 2968, tot len: 50455
2 2968 50456
arg len: 16, no args: 2969, tot len: 50472
2 2969 50473
===
Doing it again later in a different bash shell gave the following. So
it still is changing with immediate conditions.
===
arg len: 1, no args: 2968, tot len: 5935
1 2968 5936
arg len: 1, no args: 2969, tot len: 5937
1 2969 5938
arg len: 2, no args: 2968, tot len: 8903
1 2968 8904
arg len: 2, no args: 2969, tot len: 8906
1 2969 8907
arg len: 4, no args: 2968, tot len: 14839
1 2968 14840
arg len: 4, no args: 2969, tot len: 14844
1 2969 14845
arg len: 8, no args: 2968, tot len: 26711
xargs: echo: Argument list too long
arg len: 8, no args: 2969, tot len: 26720
xargs: echo: Argument list too long
arg len: 16, no args: 2968, tot len: 50455
xargs: echo: Argument list too long
arg len: 16, no args: 2969, tot len: 50472
xargs: echo: Argument list too long
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
- Raw text -