X-Recipient:	archive-cygwin AT delorie DOT com
DKIM-Filter:	OpenDKIM Filter v2.11.0 sourceware.org EFB8A38930C8
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com;
	s=default; t=1591366450;
	bh=NCaMzh890pBcxYmbHAIsYFQubfCqa8prP38colIe+ag=;
	h=References:In-Reply-To:Date:Subject:To:List-Id:List-Unsubscribe:
	List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:
	From;
	b=tlCiQc8Iw8d2McW7m1w26eW41bW9K3QV1pgd5bMG35FKhGXLaHQQtZD30Q3D/4Uwc
	SHKhQ44m57omVjvPbWllDpdKsE6lREiYS5MSJ2YqsXK1IrkK78Wvm8cqa5X4PLEKTh
	3rMGoPMnomboWPZUmQ4oFzf+5V1b3eUsFxyplzlw=
X-Original-To:	cygwin AT cygwin DOT com
Delivered-To:	cygwin AT cygwin DOT com
DMARC-Filter:	OpenDMARC Filter v1.3.2 sourceware.org C5CB53851C28
X-Google-DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:references:in-reply-to:from:date
	:message-id:subject:to;
	bh=G2we/mQous6k7zn2pBS+v9kHtEXYuoNDyg3+65dr2IU=;
	b=sMpw6HcLj9NfR11jf8Yj2DgdvJzweOOEv231EUAC6VlJcP40ZSjWvYPAIyiOhL7mzM
	cK1lmH1g5/1R3wrGPvb1+3JhAzCb4bLioLaEbAqpiD+1Q2cK6u7VRGUPBXgHUBo15sil
	URdso+I688wv7sd8IFRwzSro7Ta0vZZhbKb9DKUcJy/YDkcuhXeQHn39iylpo4nIFtNa
	OrWPz6BgTTsFBs1hPGj1BXWOhjedd72809oa8dWYTPvqnW1WbKpwBLulCJ0XC7pPkeRl
	gZAmJPGgKtuVJv8vkETdp8xlTcUfFFositL/8/wjALxTNmvLfeiqlghNc0b4MXDOJMGn
	475Q==
X-Gm-Message-State:	AOAM533KT22pm7fBeQigBiV22huIpNUGnUuB0G1E5/lAvSCAW4XJ+6SO
	2l0dk4VTZOxDelC1M/XLFsfypxEvcoiwab/UTJAYCEc93LsEdbodjA==
X-Google-Smtp-Source:	ABdhPJw0Wl62etZ2hM61AD9Doqah+9KD4qsMImS5rl+ODt6DP4fGpu/bX/P3iX/7QOsS0lL82sTV77Jydwh4jFPE97Y=
X-Received:	by 2002:a2e:b892:: with SMTP id r18mr5114442ljp.319.1591366444072;
	Fri, 05 Jun 2020 07:14:04 -0700 (PDT)
MIME-Version:	1.0
References:	<154000859 DOT 3440830 DOT 1567799052771 DOT ref AT mail DOT yahoo DOT com>
	<154000859 DOT 3440830 DOT 1567799052771 AT mail DOT yahoo DOT com>
	<1795784342 DOT 3888023 DOT 1567906713266 AT mail DOT yahoo DOT com>
In-Reply-To:	<1795784342.3888023.1567906713266@mail.yahoo.com>
Date:	Fri, 5 Jun 2020 10:13:52 -0400
Message-ID:	<CAEF1h+WFjU08Brk3ngkjmWGTCfnUXUUsc=pNcEyoqenNDiOKVw@mail.gmail.com>
Subject:	Re: cURL uploads 0 length file for sftp transfer.
To:	cygwin AT cygwin DOT com
X-Spam-Status:	No, score=-2.0 required=5.0 tests=BAYES_00, DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, FREEMAIL_FROM, HTML_MESSAGE,
	KAM_LINEPADDING, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS,
	TXREP autolearn=no autolearn_force=no version=3.4.2
X-Spam-Checker-Version:	SpamAssassin 3.4.2 (2018-09-13) on
	server2.sourceware.org
X-Content-Filtered-By:	Mailman/MimeDel 2.1.29
X-BeenThere:	cygwin AT cygwin DOT com
X-Mailman-Version:	2.1.29
List-Id:	General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Archive:	<https://cygwin.com/pipermail/cygwin/>
List-Post:	<mailto:cygwin AT cygwin DOT com>
List-Help:	<mailto:cygwin-request AT cygwin DOT com?subject=help>
List-Subscribe:	<http://cygwin.com/mailman/listinfo/cygwin>,
	<mailto:cygwin-request AT cygwin DOT com?subject=subscribe>
From:	Cary Lewis via Cygwin <cygwin AT cygwin DOT com>
Reply-To:	Cary Lewis <cary DOT lewis AT gmail DOT com>
Sender:	"Cygwin" <cygwin-bounces AT cygwin DOT com>

Curl under cygwin - TCP keepalive

I need to download a file from a remote server via a https request, and the
remote server takes several minutes to prepare the file.

Without any keepalives, the remote end will terminate the socket, but
it doesn't close it (which I know is weird).

Running curl with --keepalive 45 on a linux centos7 machine, enables the
transfer to work. With tcpdump, I see the nop packet every 45 seconds.

But on cygwin, the --keepalive parameter doesn't seem to have effect.
Running wireshark, I don't see any keepalives.

Running the request in a browser, chrome or firefox, works fine, and I can
see keepalives in wireshark, so clearly they are either setting the socket
options correctly, or they are sending their own nop.

Running on a vm, vmware workstation /  centos8 , on the same machine where
I am running curl, I can see the keepalives.

I have downloaded and rebuilt curl and that did not resolve the issue.



 Can someone shed some light on this?

On Sat, Sep 7, 2019 at 9:39 PM ALbert via cygwin <cygwin AT cygwin DOT com> wrote:

> I upgraded to the latest version curl 7.65.3 with libcurl 7.65.3, same
> issue.
> curl 7.65.3 (i686-pc-cygwin) libcurl/7.65.3 OpenSSL/1.1.1c zlib/1.2.11
> brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.0.4)
> libssh/0.8.7/openssl/zlib nghttp2/1.37.0
> Thanks.
>
>     On Friday, September 6, 2019, 3:44:13 PM EDT, ALbert <
> albert_76 AT yahoo DOT com> wrote:
>
>  We recently upgraded cURL from version:curl 7.50.3 (i686-pc-cygwin)
> libcurl/7.50.3 OpenSSL/1.0.2j zlib/1.2.8 libidn/1.29 libpsl/0.14.0
> (+libidn/1.29) libssh2/1.7.0 nghttp2/1.14.0tocurl 7.64.1 (i686-pc-cygwin)
> libcurl/7.65.3 OpenSSL/1.1.1c zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0
> libpsl/0.21.0 (+libidn2/2.0.4) libssh/0.8.7/openssl/zlib nghttp2/1.37.0
> Since the upgrade, we've seen problems uploading text files to sftp
> servers. Here's the call that we make:
> curl--trace-ascii trace.txt -k sftp://<user>:<pass>@<server>/file.txt-T
> file.txt
>
> On initial call, a 0-length file is uploaded, with successful return code.
> On immediate rerun, a full file is uploaded (if we wait few minutes,
> 0-length file will get uploaded). Below are the traces from initial and
> subsequent calls. We also tried using same version curl on a linux box,
> without any issues. We also downloaded latest version of curl from curl for
> Windows, "curl 7.65.3 (i386-pc-win32) libcurl/7.65.3 OpenSSL/1.1.1c
> (Schannel) zlib/1.2.11 brotli/1.0.7 WinIDN libssh2/1.9.0 nghttp2/1.39.2",
> and its also working correctly.
> Unfortunately, the server we're uploading to belongs to a client, and we
> can't see if this is an issue on a server side. However, even if it was,
> why would linux curl and windows curl (from curl.haxx.se) would work
> correctly?
> Any help or pointers would be appreciated. Thank you.
>
> Here's the trace files:
> From initial call:== Info: STATE: INIT => CONNECT handle 0x80068340; line
> 1356 (connection #-5000)
> == Info: Added connection 0. The cache now contains 1 members
> == Info: STATE: CONNECT => WAITRESOLVE handle 0x80068340; line 1397
> (connection #0)
> == Info:   Trying XXX.XX.XX.XX...== Info: TCP_NODELAY set
> == Info: STATE: WAITRESOLVE => WAITCONNECT handle 0x80068340; line 1476
> (connection #0)
> == Info: Connected to XXXXXXX  ( XXX.XX.XX.XX) port 22 (#0)== Info: STATE:
> WAITCONNECT => SENDPROTOCONNECT handle 0x80068340; line 1532 (connection #0)
> == Info: Marked for [keep alive]: SSH default
> == Info: User: Opus_efg
> == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_INIT (line 2115)
> == Info: SSH 0x80048f90 state change from SSH_INIT to SSH_S_STARTUP (line
> 582)
> == Info: STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x80068340; line
> 1547 (connection #0)
> == Info: SSH 0x80048f90 state change from SSH_S_STARTUP to SSH_HOSTKEY
> (line 595)
> == Info: SSH 0x80048f90 state change from SSH_HOSTKEY to SSH_AUTHLIST
> (line 605)
> == Info: SSH 0x80048f90 state change from SSH_AUTHLIST to
> SSH_AUTH_PKEY_INIT (line 628)
> == Info: Authentication using SSH public key file
> == Info: SSH 0x80048f90 state change from SSH_AUTH_PKEY_INIT to
> SSH_AUTH_KEY_INIT (line 697)
> == Info: SSH 0x80048f90 state change from SSH_AUTH_KEY_INIT to
> SSH_AUTH_KEY (line 743)
> == Info: completed keyboard interactive authentication
> == Info: SSH 0x80048f90 state change from SSH_AUTH_KEY to SSH_AUTH_DONE
> (line 761)
> == Info: Authentication complete
> == Info: SSH 0x80048f90 state change from SSH_AUTH_DONE to SSH_SFTP_INIT
> (line 807)
> == Info: SSH 0x80048f90 state change from SSH_SFTP_INIT to
> SSH_SFTP_REALPATH (line 833)
> == Info: SSH CONNECT phase done
> == Info: SSH 0x80048f90 state change from SSH_SFTP_REALPATH to SSH_STOP
> (line 850)
> == Info: STATE: PROTOCONNECT => DO handle 0x80068340; line 1566
> (connection #0)
> == Info: DO phase starts
> == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_SFTP_QUOTE_INIT
> (line 2332)
> == Info: SSH 0x80048f90 state change from SSH_SFTP_QUOTE_INIT to
> SSH_SFTP_GETINFO (line 868)
> == Info: SSH 0x80048f90 state change from SSH_SFTP_GETINFO to
> SSH_SFTP_TRANS_INIT (line 1060)
> == Info: SSH 0x80048f90 state change from SSH_SFTP_TRANS_INIT to
> SSH_SFTP_UPLOAD_INIT (line 1080)
> == Info: SSH 0x80048f90 state change from SSH_SFTP_UPLOAD_INIT to
> SSH_SFTP_CLOSE (line 1143)
> == Info: STATE: DO => DOING handle 0x80068340; line 1608 (connection #0)
> == Info: SFTP DONE done
> == Info: SSH 0x80048f90 state change from SSH_SFTP_CLOSE to SSH_STOP (line
> 1651)
> == Info: DO phase is complete
> == Info: STATE: DOING => DO_DONE handle 0x80068340; line 1693 (connection
> #0)
> == Info: STATE: DO_DONE => PERFORM handle 0x80068340; line 1743
> (connection #0)
> == Info: STATE: PERFORM => DONE handle 0x80068340; line 1933 (connection
> #0)
> == Info: multi_done
> == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_SFTP_CLOSE (line
> 2390)
> == Info: SFTP DONE done
> == Info: SSH 0x80048f90 state change from SSH_SFTP_CLOSE to SSH_STOP (line
> 1651)
> == Info: Connection #0 to host XXXXXXX left intact
>
>
> Trace from subsequent call:== Info: STATE: INIT => CONNECT handle
> 0x80068340; line 1356 (connection #-5000)
> == Info: Added connection 0. The cache now contains 1 members
> == Info: STATE: CONNECT => WAITRESOLVE handle 0x80068340; line 1397
> (connection #0)
> == Info:   Trying XXX.XX.XX.XX...== Info: TCP_NODELAY set
> == Info: STATE: WAITRESOLVE => WAITCONNECT handle 0x80068340; line 1476
> (connection #0)
> == Info: Connected to XXXXXXX (XXX.XX.XX.XX) port 22 (#0)
> == Info: STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x80068340; line
> 1532 (connection #0)
> == Info: Marked for [keep alive]: SSH default
> == Info: User: Opus_efg
> == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_INIT (line 2115)
> == Info: SSH 0x80048f90 state change from SSH_INIT to SSH_S_STARTUP (line
> 582)
> == Info: STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x80068340; line
> 1547 (connection #0)
> == Info: SSH 0x80048f90 state change from SSH_S_STARTUP to SSH_HOSTKEY
> (line 595)
> == Info: SSH 0x80048f90 state change from SSH_HOSTKEY to SSH_AUTHLIST
> (line 605)
> == Info: SSH 0x80048f90 state change from SSH_AUTHLIST to
> SSH_AUTH_PKEY_INIT (line 628)
> == Info: Authentication using SSH public key file
> == Info: SSH 0x80048f90 state change from SSH_AUTH_PKEY_INIT to
> SSH_AUTH_KEY_INIT (line 697)
> == Info: SSH 0x80048f90 state change from SSH_AUTH_KEY_INIT to
> SSH_AUTH_KEY (line 743)
> == Info: completed keyboard interactive authentication
> == Info: SSH 0x80048f90 state change from SSH_AUTH_KEY to SSH_AUTH_DONE
> (line 761)
> == Info: Authentication complete
> == Info: SSH 0x80048f90 state change from SSH_AUTH_DONE to SSH_SFTP_INIT
> (line 807)
> == Info: SSH 0x80048f90 state change from SSH_SFTP_INIT to
> SSH_SFTP_REALPATH (line 833)
> == Info: SSH CONNECT phase done
> == Info: SSH 0x80048f90 state change from SSH_SFTP_REALPATH to SSH_STOP
> (line 850)
> == Info: STATE: PROTOCONNECT => DO handle 0x80068340; line 1566
> (connection #0)
> == Info: DO phase starts
> == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_SFTP_QUOTE_INIT
> (line 2332)
> == Info: SSH 0x80048f90 state change from SSH_SFTP_QUOTE_INIT to
> SSH_SFTP_GETINFO (line 868)
> == Info: SSH 0x80048f90 state change from SSH_SFTP_GETINFO to
> SSH_SFTP_TRANS_INIT (line 1060)
> == Info: SSH 0x80048f90 state change from SSH_SFTP_TRANS_INIT to
> SSH_SFTP_UPLOAD_INIT (line 1080)
> == Info: SSH 0x80048f90 state change from SSH_SFTP_UPLOAD_INIT to SSH_STOP
> (line 1222)
> == Info: DO phase is complete
> == Info: STATE: DO => DO_DONE handle 0x80068340; line 1621 (connection #0)
> == Info: STATE: DO_DONE => PERFORM handle 0x80068340; line 1743
> (connection #0)
> => Send data, 10688 bytes (0x29c0)
>   SENSITIVE DATA DELETED
> == Info: We are completely uploaded and fine
> == Info: STATE: PERFORM => DONE handle 0x80068340; line 1933 (connection
> #0)
> == Info: multi_done
> == Info: SSH 0x80048f90 state change from SSH_STOP to SSH_SFTP_CLOSE (line
> 2390)
> == Info: SFTP DONE done
> == Info: SSH 0x80048f90 state change from SSH_SFTP_CLOSE to SSH_STOP (line
> 1651)
> == Info: Connection #0 to host XXXXXXX left intact
> == Info: Expire cleared (transfer 0x80068340)
>
>
>
>
>
>
>
>
> --
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>
>
--
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple

- Raw text -