X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=1.8 required=5.0	tests=AWL,BAYES_50,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SARE_FREE_WEBM_LAPOSTE,TW_MK,TW_NP,TW_PD,T_RP_MATCHES_RCVD,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: sourceware.org
Message-ID: <4CC0CF60.20206@laposte.net>
Date: Fri, 22 Oct 2010 01:40:16 +0200
From: Cyrille Lefevre <cyrille DOT lefevre-lists AT laposte DOT net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.3) Gecko/20070326 Thunderbird/2.0.0.0 Mnenhy/0.7.5.666
MIME-Version: 1.0
To: cygwin AT cygwin DOT com
Subject: Re: Bash problems, strace, performance, etc.
References: <4CBE119C DOT 9070504 AT veritech DOT com> <4CBE179C DOT 7020008 AT veritech DOT com> <4CBFCE23 DOT 3010706 AT laposte DOT net> <4CC0AF02 DOT 1090600 AT veritech DOT com>
In-Reply-To: <4CC0AF02.1090600@veritech.com>
Content-Type: multipart/mixed; boundary="------------040002040507050801050806"
X-me-spamlevel: not-spam
X-me-spamrating: 34.799999
X-me-spamcause:  OK, (-130)(0000)gggruggvucftvghtrhhoucdtuddrfedthedrtdduucetggdotefuucfrrhhofhhilhgvmecuoehnohhnvgeqnecuuegrihhlohhuthemuceftddtnecuucdlqddutddtmdenthhhvgcuphhrohgslhgvmhculddqfedtmd
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

--------------040002040507050801050806
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Le 21/10/2010 23:22, Lee D. Rothstein a =E9crit :
>=20
>  On 10/21/2010 1:22 AM, Cyrille Lefevre wrote:
>=20
>> seems to work here ! even on cpan.1 generated from pod2man...
>=20
> Just curious, what part of 'man2pdf' did you run that indicates
> that it works on your system? (Commands and options, would be
> most helpful.)

dos2unix < man2pdf > /usr/local/bin/man2pdf
perl -pi -e 's/=3Dletter/=3Da4/' /usr/local/bin/man2pdf
echo echo /tmp > /usr/local/bin/tmpdir
echo > /usr/local/bin/man_blrm
echo 'case $1 in *.*) echo ${1##*.} ;; esac' \
> /usr/local/bin/fn_ext_last
chmod +x /usr/local/bin/man2pdf /usr/local/bin/tmpdir \
/usr/local/bin/man_blrm /usr/local/bin/fn_ext_last
mkdir -p /tmp/man/man1
cd /tmp/man
pod2man /usr/lib/perl5/5.10/CPAN.pm > man1/cpan.1
MANPATH=3D$PWD man2pdf cpan.1
cd /usr/lib/perl5/5.10/pods
ls |
sed 'h;s|\.pod|.1|;x;G;s|\n| > /tmp/man/man1/|;s|^|eval pod2man |' |
parallel # see attachment, please, keep the copyright, thanks :-)

>> do you have bash-completion, if yes, get rid of it and try again.
>=20
> Bash completion is installed on my system but never loaded; i.e.,
> /etc/bash_completion is not sourced in '~/.bashrc', or
> '~/.profile', or anywhere else.
>=20
> Why do you think this is the cause of the problem?

bcoz bash-completion is known to cause performance problem to bash...

> You might be combining two parts of my report in a
> misleading way. The slowdown was due to the errant Bash
> sessions not the cause of them. A reboot eventually returned
> the system to its normal sluggish state (for Cygwin on a 64-bit
> version of Windoze). Or, what don't I understand?

well, I'm running 32bit windoz...

> Notice:
> --=20
>   $ time bash -i -c echo
>=20
>   real    0m0.683s
>   user    0m0.015s
>   sys     0m0.186s
>=20
>   $ time bash -c echo
>=20
>   real    0m0.342s
>   user    0m0.015s
>   sys     0m0.062s
> --=20

bash is also known to be a *piece of crap* in I/O, done.

however, these exemples aren't representative, specifically the first
one since you have the interactive initialisation. a r/w loop would be
more representative (1000 times is sufficient)

i.e. :

v2$ time bash -c 'i=3D0; while (( (i+=3D1) < 1000 )); do while read -r; do
echo -E "$REPLY"; done < /etc/passwd; done > /dev/null'

real    0m25.965s
user    0m2.838s
sys     0m22.494s

# pdksh
v2$ time ksh -c 'i=3D0; while (( (i+=3D1) < 1000 )); do while read -r; do
echo -E "$REPLY"; done < /etc/passwd; done > /dev/null'

real    0m8.250s
user    0m0.764s
sys     0m7.065s

v2$ time ksh88 -c 'i=3D0; while (( (i+=3D1) < 1000 )); do while read -r; do
echo -E "$REPLY"; done < /etc/passwd; done > /dev/null'

real    0m1.270s
user    0m0.296s
sys     0m0.810s

v2$ time ksh93 -c 'i=3D0; while (( (i+=3D1) < 1000 )); do while read -r; do
echo -E "$REPLY"; done < /etc/passwd; done > /dev/null'

real    0m1.472s
user    0m0.374s
sys     0m0.967s

the -i doesn't change anything, bash is still so ssslllooowwww !

> I'm more inclined to believe that it has something to do with
> either:
>=20
>   * the "extreme" number of processes generated by:
>     'mkperlmanpdfs'/'man2pdf' and interaction possibly with a
>     'bash' bug

no problem, here, except a windows memory exhausted ! then bash died...
almost the first time I see this windows box !!!

maybe you launch too many processes at a time ?

however, I propose you to modify your mkperlmanpdfs script using the
attached parallel shell script... please, keep the copyright, thanks :-)

first time posted on
http://www.mail-archive.com/ast-users AT research DOT att DOT com/msg00942.html

i.e. :

cd /tmp/man
cat << 'EOF' > mkperlmanpdfs
ls /usr/lib/perl5/5.10/pods |
sed 's|\.pod||;s|^|man2pdf -n |' |
parallel
EOF
chmod +x mkperlmanpdfs
MANPATH=3D$PWD ./mkperlmanpdfs

no memory exhausted and only 98% cpu :-)
elapsed time is 425 seconds (aka 7+ minutes) on a Q6600.

> or
>   * corrupt fonts in Windows or Cygwin that 'man'/'groff' is
>     "barfing" on.

no problem here ?

> WRT the font "issue" (real or imagined on my part), I'm unclear
> on whether Cygwin can use Windows TT fonts for things like
> 'groff', and X windows rendering. (I would like to know, for
> example, how to specify which fonts to 'enscript'.)
> *But*, *I know I don't know what I'm talking about and that's why
> I asked the Cygwin list.*

out of topic.

>> PS : where do you find the Club-G package ?
>=20
> The Club-G (TM) scripts are an interdependent set that I wrote,
> which will be published, RSN, on OpenEnterprise.org/Club-G.
> "Club-G" (TM), BTW, stands for:

well, and how do you expect I test your man2pdf script the right way w/
these missing pieces ?
for instance, I wrote them using a simple a one liner... and they seems
to make it :-P

Regards,

Cyrille Lefevre
--=20
mailto:Cyrille DOT Lefevre-lists AT laposte DOT net


--------------040002040507050801050806
Content-Type: text/plain;
 name="parallel"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="parallel"

#!/usr/bin/sh
#
#!ident @(#) parallel.sh 1.1 (Cyrille.Lefevre-lists%nospam AT laposte DOT net DOT invalid) Mon Jun 28 17:42:42     2010
# supprimer "%nospam" et ".invalid" pour me repondre.
# remove "%nospam" and ".invalid" to answer me.
#
# Copyright (c) 2010 Cyrille Lefevre. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# 1. Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in
#    the documentation and/or other materials provided with the
#    distribution.
# 3. The name of the authors and contributors may not be used to
#    endorse or promote products derived from this software without
#    specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS''
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
# PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHORS
# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.

function loop {
	typeset maxpid=$1
	typeset pid= newpids=
	typeset -i sleep=0
	while (( npid >= maxpid )); do
		newpids= sleep=1
		for pid in ${pids}; do
			if kill -0 "${pid}" 2> /dev/null; then
				newpids="${newpids} ${pid}"
			else
				sleep=0
				(( npid -= 1 ))
				wait "${pid}"
				echo "${pid}: done"
			fi
		done
		pids=${newpids}
		(( sleep != 0 )) && sleep 1
	done
}

function parallel {
	typeset ncpu=$1 nice=$2
	shift 2
	typeset -i npid=0
	typeset pids=
	while read -r; do
		case ${REPLY} in
		''|'#'*)
			continue
			;;
		ncpu=*|nice=*)
			echo ${REPLY}
			eval ${REPLY}
			continue 
			;;
		rset)
			echo rset
			ncpu=${_ncpu}
			continue 
			;;
		wait)
			echo wait
			loop 1
			continue 
			;;
		esac
		loop "${ncpu}"
		${REPLY} &
		(( npid += 1 ))
		pids="${pids} $!"
		[[ -n ${nice} && ${nice} != 0 ]] && sleep 1 && renice ${nice} $!
		echo "$!: $REPLY"
	done
	loop 1
}

_ncpu=$1
if (( ${_ncpu:-0} < 1 )); then
	case $(uname) in
	AIX)
		_ncpu=$(LC_ALL=C lsdev -c processor | grep -c Avail)
		;;
	Darwin)
		#noht#_ncpu=$(sysctl -n hw.physicalcpu)
		_ncpu=$(sysctl -n hw.availcpu) # was logicalcpu
		;;
	FreeBSD)
		_ncpu=$(sysctl -n hw.ncpu)
		;;
	HP-UX)
		_ncpu=$(ioscan -fkC processor | grep -c processor)
		;;
	CYGWIN*)
		# _ncpu=${NUMBER_OF_PROCESSORS}
		_ncpu=$(grep -c processor /proc/cpuinfo)
		;;
	Linux)
		#noht#_ncpu=$(grep 'physical id' /proc/cpuinfo | sort -u | wc -l)
		_ncpu=$(grep -c processor /proc/cpuinfo)
		;;
	SunOS)
		_ncpu=$(LC_ALL=C psrinfo -v | grep -c on-line)
		;;
	esac
fi

_coef=$2
if (( ${_coef:-0} > 1 )); then
	(( _ncpu *= _coef ))
fi

_nice=$3

_time0=${SECONDS}
parallel ${_ncpu:-1} ${_nice:-0} "$@"
(( _time0 -= -${SECONDS} ))
echo elapsed: ${_time0}


--------------040002040507050801050806
Content-Type: text/plain; charset=us-ascii

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
--------------040002040507050801050806--