X-Recipient: archive-cygwin AT delorie DOT com
DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:reply-to:subject:to:references:from:message-id
	:date:mime-version:in-reply-to:content-type
	:content-transfer-encoding; q=dns; s=default; b=Ei7kfBA+kYKBBWij
	B/ml8i7JWkgxCqlxoCw2gAP5Hr76eGOXGNgfo/kcuaWfuI6I4B+M9k39vKLn8IWz
	DNXarhUBMoF2tyaA7TtGiDn0P97zUNkjkZYSVXr9ircP98yi4ojgo4hEPul9HRRK
	E1t7XLrgpb3T2HDwP/orDWHa1jk=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id
	:list-unsubscribe:list-subscribe:list-archive:list-post
	:list-help:sender:reply-to:subject:to:references:from:message-id
	:date:mime-version:in-reply-to:content-type
	:content-transfer-encoding; s=default; bh=6e87t0DPEu+we6UaAgQmSS
	7nq1A=; b=Bg2515FUaQDTcIQ3LCub6xNKzQAxNyr3164ya6VFU1nSVAq9tUQTQw
	1tG05Yvtdr5drCABZA3pd+8xp7faYEShEl2aW3qF6UHlZsmXgl/0lq9HanMJazj7
	kJIShmbhOdZMx0B2q/VcQj7h5S9zfoBvi4KqaB3y1eNVHdzjlrqjE=
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com
Authentication-Results: sourceware.org; auth=none
X-Spam-SWARE-Status: No, score=0.7 required=5.0 tests=AWL,BAYES_50,KAM_LAZY_DOMAIN_SECURITY,RCVD_IN_DNSWL_LOW autolearn=no version=3.3.2 spammy=canada, H*R:D*ca, mintty, Canada
X-HELO: smtp-out-so.shaw.ca
Reply-To: Brian DOT Inglis AT SystematicSw DOT ab DOT ca
Subject: Re: speeding up a paste operation
To: cygwin AT cygwin DOT com
References: <CAD8GWstSbJHnP3XWKvhbtRisB=jJL7SJA9aMOWMJVDbMC_VdYA AT mail DOT gmail DOT com> <5b80a0d3 DOT 1c69fb81 DOT 386aa DOT 8658 AT mx DOT google DOT com> <CAD8GWsuPDeQEoBSMDRy36+2G5Ej=LBvsRt0sNVajL3Bs_D0VtA AT mail DOT gmail DOT com>
From: Brian Inglis <Brian DOT Inglis AT SystematicSw DOT ab DOT ca>
Openpgp: preference=signencrypt
Message-ID: <9a4a9ea3-c4fc-6f60-db11-b6fe5fa6dede@SystematicSw.ab.ca>
Date: Tue, 28 Aug 2018 22:36:49 -0600
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1
MIME-Version: 1.0
In-Reply-To: <CAD8GWsuPDeQEoBSMDRy36+2G5Ej=LBvsRt0sNVajL3Bs_D0VtA@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-IsSubscribed: yes

On 2018-08-24 18:45, Lee wrote:
> On 8/24/18, Steven Penny <svnpenn AT gmail DOT com> wrote:
>> On Fri, 24 Aug 2018 19:30:10, Lee wrote:
>>> In retrospect, I should have created the file some other way, but still..
>>> - grab the top 1M hosts from from
>>> http://s3-us-west-1.amazonaws.com/umbrella-static/index.html
>>> - open w/ libreoffice
>>> - select the host name column, right click & select copy (all 1 million
>>> lines)
>>> - (mintty 2.9.0 window already open) vi /tmp/hosts
>>> - i  (get into insert mode)
>>> - right click (which I have set to "paste")
>>>
>>> data is still scrolling by & it's not even up to 100K lines yet :(
>>>
>>> Is there some way to make a paste operation faster in mintty (or vim
>>> or whatever the slowpoke is)?
>>
>> Use the right tool for the job:
>>
>>     $ time awk -F, '{print $2}' top-1m.csv > hosts.txt
>>     real    0m0.686s
> 
> I'd forgotten how slow pasting is in cygwin.. otherwise I'd have just
> deleted column 1, saved as text & dos2unix'ed the file
> 
> Oh well...  I was hoping I'd set up something wrong in cygwin, but at
> least I remember why not to paste lots o lines in cygwin.

Don't save what you don't need:

$ time curl -LSs
http://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip | funzip | awk
-F, '{print $2}' > hosts.txt

real    0m6.396s
user    0m3.046s
sys     0m0.372s

If you need to save intermediate files, insert tee commands.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple