DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 2CB2F384B806
Subject: Re: Python for Windows reports wrong local time when run under Cygwin
 on Europe/Moscow TZ
To: cygwin@cygwin.com
References: <CAAHpriO6A+6bLXt0Qe-xpdXT1CiLjk7=7G31cHBtnVJ0kXb2Eg@mail.gmail.com>
 <CAAHpriOJ5EZ0YzXjNOV-fydCb6bWHmPKiFZUC7KtFAi8DnkkDQ@mail.gmail.com>
 <02d5b40f-aa35-f56c-5c5c-b10780355e91@SystematicSw.ab.ca>
 <0fcfeb54-53c5-b83d-bb13-e83a68eed469@cornell.edu>
 <28d9253d-71f4-c265-2861-9a3b0667799b@SystematicSw.ab.ca>
 <be106324-4371-a804-5ccc-dec22b37f4af@cornell.edu>
 <de7f3163-74c0-f304-ee58-a12064864196@SystematicSw.ab.ca>
 <55e1f8cd-ae55-dbda-ea6e-b9c000ec329c@cornell.edu>
 <2fbd2f7a-f08e-eb92-174e-f9380420c31d@SystematicSw.ab.ca>
Message-ID: <0b87c828-68c0-1d73-679e-15c14c8cce3f@cornell.edu>
Date: Sat, 12 Jun 2021 18:44:46 -0400
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
In-Reply-To: <2fbd2f7a-f08e-eb92-174e-f9380420c31d@SystematicSw.ab.ca>
Content-Language: en-US
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [192.168.0.18] (24.194.34.31) by
 MN2PR07CA0001.namprd07.prod.outlook.com (2603:10b6:208:1a0::11) with
 Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4219.21 via Frontend
 Transport; Sat, 12 Jun 2021 22:44:46 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 773e2446-9c1f-439d-7869-08d92df3add3
X-MS-TrafficTypeDiagnostic: BN7PR04MB5299:
X-Microsoft-Antispam-PRVS: <BN7PR04MB5299B20B9A34255423409561D8339@BN7PR04MB5299.namprd04.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: XZFvo4pN10Amkxgf41f0cS/HRPuwnUyUbaciqF4USJM3riELYdmu/96qWY9Zg63IK9D9r7lVWTuBGEZwwVtsqH7WUCXRf15KYH4hfkBpySFAx/AA2sq1U6iF2i3/kWuFOj1Wy0Yqi4+GH4BvomJ1q0XGJJSxd7HMJ3lDYxDp7hA+WmHrWBcKAqRY7IKfzwe54ObOpHjW27Ft/m1eHGBXR8J5zZaqVJGldcGHIU8604hFI350EBvyCx6ZT9SJt80lcL/vqAsQxa0bwO+Vf4twxoLeTL5W1Df79l9tHmHiL/9BvOT52RcoAqRoIZVmPv+DyFzN+vGN/GihV3HW/wlcqf0Sv7yuipb/N8sx/+WvqqfJDWIgUAAUJwusOTrxKHQSPOOVTtXOivOFNTNMmGk+UhNuXC67IIo47o3YAvvjKMvWcSjz1sDfJxjAFqNa8TxCUvnFMDQejm+TVLWwPF1ph2NcCn00Lz6508gCn4AKVMU71PJ0K1uUpu7Uho79sepwlo28nQZKgp4UrPkcTgEAVz4Q8s9NkTUthG89NGCj+wZtw8cEDGyMchOP1zdssDsDnUUcuOLElV6Vdq3V8B03q66tL4faxLqNdtMW+k/YixU5EUZggnz+MESKipsUK3qpge1ATnaitRQtYua0Di7C7jv2zfa9IGYMEIZ+zCa7XXYAdbAJ6DqTs5J59ji0WY7IYOD9ck8PE1dBmWAuq9ZwVX8rbxic/9Vsza/8WvLqabtzH8m5c6hziDU6bvKVkVk+oJJXMZ+LbUdU4KC2BFJarj0OITrgJ/Y2RG1E9qy5IuO/IhP/YpmjU9dnhcCAFF8W9x3Ke1Ofl7E4vjXX4i2YRw==
X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:BN7PR04MB4388.namprd04.prod.outlook.com; PTR:; CAT:NONE;
 SFS:(4636009)(396003)(346002)(366004)(136003)(376002)(39860400002)(8936002)(36756003)(38100700002)(2906002)(6486002)(31686004)(31696002)(2616005)(956004)(86362001)(83380400001)(478600001)(16526019)(186003)(75432002)(66476007)(66946007)(26005)(66556008)(6916009)(786003)(16576012)(53546011)(8676002)(5660300002)(966005)(316002)(45980500001)(43740500002);
 DIR:OUT; SFP:1102; 
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?Windows-1252?Q?dJl6Wz2yOC9EfK1Db4R7z3wq0odcC1Hi0NybrBsL4ll4z3YH35ZhXjH1?=
 =?Windows-1252?Q?5CHpaQt/iDpChBvZt58oWErKIvH8pAILIYg+c+CbmZTRLNAFN12R+H46?=
 =?Windows-1252?Q?Qcb6i9n65yqbIWoN5FZ319NmeDR97gFxCNxMP1Tek0cqfoo/BQgx9GaZ?=
 =?Windows-1252?Q?Qpt+PFLCw/b5Z4lwaTUcMplKQftUjM2i+uJ7T278yRp69G2Cae1gBVLN?=
 =?Windows-1252?Q?RuHLG7UAGpe9g8Y+wTWDOMfaRvtmQMeub7nJLrB6zfnkA5ivy90/MBk9?=
 =?Windows-1252?Q?4LCWW4JSyFEPcQiBM8iX2wrL3kJ/eu4FLIfKtr06dqZqD4LO5B5LIIgs?=
 =?Windows-1252?Q?PZuP33sDuHui3/u9SPz3c9UA6hY2xp2jdSJ1rTxcGISoaB0w856Vk5jc?=
 =?Windows-1252?Q?ksWmM7ZKFiC7yDtA1UVAvSEvFlXLcSnPt2CP2s9Hnu69sGKr1mEUFe7q?=
 =?Windows-1252?Q?wqvAxfFw0Chbv3A1i2CeY/7SXEbde6BifF36Q4SYTFaxVEXlLAMOKPaF?=
 =?Windows-1252?Q?xCKPOfQpyX+5x+L4lHkNNhJ5IzOKJHTB2YS0hNd7oVzudh1mssa2eNmo?=
 =?Windows-1252?Q?QZy03c6oJ5/eUds+pN1vXL/8gKz/bTk66dxbsXU2aejEz9lq7EF9r405?=
 =?Windows-1252?Q?SeN7Tl55WIhcU1kiR/Rv+mAyo92CTtuNwX4nNmpffGl6gBwn7tXsuPkJ?=
 =?Windows-1252?Q?sJCH1QIeOOvXyhnOQ//x40B26EAcJqABt+6md3DPlfSNEhibT7OOUS4p?=
 =?Windows-1252?Q?nA9jsXTDGp9Nw5hhhiDLfpQKi/0ReDmlYQl5P31f3sVOebnQVT+8py7h?=
 =?Windows-1252?Q?LfzTF5yQVRU1K+Ga9s1MEH7kmz+ni8maKkgfLkxceEWvhO2BOZ39PvK6?=
 =?Windows-1252?Q?aZwuCGF6oJfZ4LDdwd8K/pHNdyWvLzNP0IVcjlFDTDqhtUxdGmXuQP6r?=
 =?Windows-1252?Q?wN0DS9qaxZjr5i3jjkNd2sI/fRqBpOsrtkBsEHa84Wc7yAXitU7z7Gib?=
 =?Windows-1252?Q?BIzzpClZgG2GGboGAkp2RaSBthDOHM0dmHbGSwYyL4gyg5q5A+ATGUoG?=
 =?Windows-1252?Q?IfCQ8/qcfW7fp4uiMx35lnNxaZ5kyK+zHSlt4wMOFehEKMhifUCrVA9d?=
 =?Windows-1252?Q?+yDCqVxgbsM83mhosHfEPSFeSG0NY4LbYZnMpQCQwXW5PpgufVU7HYXH?=
 =?Windows-1252?Q?QDCcAovjqZJpWQrYrRU8Qe3krx98IgyP3uQPuQUTMFXFh1H+ysoVnAxJ?=
 =?Windows-1252?Q?dMNuilaFSEfTQSPQFKSig0vVJlHYGGMZ//r0ihNgcEnMxocCHPwvmi+C?=
 =?Windows-1252?Q?VJ8L0K+au8hm8v0ej3o3aOp7hq29nvvLyo1YXQ35mLCbGl4sOP+ZF18d?=
 =?Windows-1252?Q?35HHIltmCLgmXa9xCA6H8BUSS2q4wO0eZ0/qLAHW80JAQ0b4k4lUIJUN?=
X-OriginatorOrg: cornell.edu
X-MS-Exchange-CrossTenant-Network-Message-Id: 773e2446-9c1f-439d-7869-08d92df3add3
X-MS-Exchange-CrossTenant-AuthSource: BN7PR04MB4388.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jun 2021 22:44:46.8724 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5d7e4366-1b9b-45cf-8e79-b14b27df46e1
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bHQ+TsUp4Tymizkge9OtotTASk77P6eKyce6+65BcSAvpnUTWkNDpoF5R6q2i5uqF3YVfP+i/9gFuFAv1cWhmA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR04MB5299
X-Spam-Status: No, score=-0.1 required=5.0 tests=BAYES_00, DKIM_INVALID,
 DKIM_SIGNED, KAM_DMARC_STATUS, MSGID_FROM_MTA_HEADER, NICE_REPLY_A,
 RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS,
 TXREP autolearn=no autolearn_force=no version=3.4.2
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on
 server2.sourceware.org
X-BeenThere: cygwin@cygwin.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: General Cygwin discussions and problem reports <cygwin.cygwin.com>
List-Unsubscribe: <https://cygwin.com/mailman/options/cygwin>,
 <mailto:cygwin-request@cygwin.com?subject=unsubscribe>
List-Archive: <https://cygwin.com/pipermail/cygwin/>
List-Post: <mailto:cygwin@cygwin.com>
List-Help: <mailto:cygwin-request@cygwin.com?subject=help>
List-Subscribe: <https://cygwin.com/mailman/listinfo/cygwin>,
 <mailto:cygwin-request@cygwin.com?subject=subscribe>
From: Ken Brown via Cygwin <cygwin@cygwin.com>
Reply-To: Ken Brown <kbrown@cornell.edu>
Content-Type: text/plain; charset="windows-1252"; Format="flowed"
Errors-To: cygwin-bounces+archive-cygwin=delorie.com@cygwin.com
Sender: "Cygwin" <cygwin-bounces+archive-cygwin=delorie.com@cygwin.com>
Content-Transfer-Encoding: 8bit
X-MIME-Autoconverted: from quoted-printable to 8bit by delorie.com id 15CMjRZ1024886

On 6/12/2021 12:34 PM, Brian Inglis wrote:
> On 2021-06-11 12:05, Ken Brown via Cygwin wrote:
>> On 6/11/2021 1:33 PM, Brian Inglis wrote:
>>> On 2021-06-10 13:50, Ken Brown via Cygwin wrote:
>>>> On 6/10/2021 2:31 PM, Brian Inglis wrote:
>>>>> On 2021-06-10 08:57, Ken Brown via Cygwin wrote:
>>>>>> On 6/9/2021 10:36 PM, Brian Inglis wrote:
>>>>>>> On 2021-06-09 16:31, Keith Thompson via Cygwin wrote:
>>>>>>>> [Sorry if the threading is messed up.  I don't subscribe, so I'm
>>>>>>>> constructing this message from the web interface.  It should at least
>>>>>>>> show up under the correct subject.]
>>>>>>>>
>>>>>>>> Brian Inglis wrote:
>>>>>>>>> On 2021-06-08 14:03, Mike Kaganski via Cygwin wrote:
>>>>>>>>>> On 08.06.2021 16:04, L A Walsh wrote:
>>>>>>>>>>> You might ask on a python list if anyone else has experienced
>>>>>>>>>>> something similar with python or any other program.  I'm fairly sure
>>>>>>>>>>> that neither MS nor cygwin design their OS with python in mind and
>>>>>>>>>>> that it is python that is interacting funny when running under some
>>>>>>>>>>> merge of both.  Have you asked the python people about this problem?
>>>>>>>>>>> What did they suggest?
>>>>>>>>>>
>>>>>>>>>> FTR: filed https://bugs.python.org/issue44352.
>>>>>>>>>
>>>>>>>>> See Keith Thompson subthread and my reply with suggested fix:
>>>>>>>>>
>>>>>>>>> https://cygwin.com/pipermail/cygwin/2021-June/248692.html
>>>>>>>>>
>>>>>>>>> Windows does not recognize zoneinfo time zone identifiers in TZ only
>>>>>>>>> base format POSIX TZ strings with three alphabetic character identifiers:
>>>>>>>>>
>>>>>>>>> https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/tzset?view=msvc-160 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> That assumes US switch date "rules": for all years up to current, or
>>>>>>>>> just DST, and whether pre- or post-2007 is unstated!
>>>>>>>>>
>>>>>>>>> Otherwise it defaults to regional settings, used by Cygwin to map to
>>>>>>>>> zoneinfo time zone identifiers, so if Python for Windows could clear TZ
>>>>>>>>> before it is read by MSVCRT, it should DTRT.
>>>>>>>>>
>>>>>>>>> Windows does not recognize expanded POSIX TZ format strings with <>
>>>>>>>>> quoted alphanumeric characters, "-", "+", and start and end dates/times:
>>>>>>>>>
>>>>>>>>> https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#bottom 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> which make them usable outside of the US.
>>>>>>>>
>>>>>>>> Summary: IMHO Cygwin should adapt its default TZ setting to work
>>>>>>>> with Windows.
>>>>>>>>
>>>>>>>> The suggestion is to modify Python for Windows so it can deal with
>>>>>>>> the TZ format used by Cygwin.  I haven't used Python for Windows, but
>>>>>>>> as far as I know it's unrelated to Cygwin; rather it, like Cygwin, is
>>>>>>>> intended to work on top of Windows.  I'm not convinced it's appropriate
>>>>>>>> to ask Python for Windows to make a change purely for the sake of
>>>>>>>> interoperating with Cygwin, which many PfW users presumably aren't
>>>>>>>> even using.
>>>>>>>>
>>>>>>>> I've run into another application that has problems with Cygwin's
>>>>>>>> settings of $TZ.  It was a internal test application that isn't
>>>>>>>> going to change its timezone handling just for this problem.
>>>>>>>>
>>>>>>>> The ideal solution would be for Windows to recognize TZ values like
>>>>>>>> "America/Los_Angeles", but that's not likely to happen any time soon.
>>>>>>>>
>>>>>>>> My suggestion, since Cygwin is supposed to interoperate with Windows,
>>>>>>>> is one of the following:
>>>>>>>>
>>>>>>>> - Cygwin should avoid setting TZ to a value that Windows doesn't recognize
>>>>>>>>    (if I set TZ=PST8PDT, everything seems to work correctly); OR
>>>>>>>>
>>>>>>>> - Cygwin shouldn't set TZ at all by default.  (I've updated my
>>>>>>>>    $HOME/.bash_profile on Cygwin to unset TZ, and Cygwin commands seem
>>>>>>>>    to work correctly with TZ unset); OR
>>>>>>>>
>>>>>>>> - Cygwin, when invoking a non-Cygwin executable, should first either
>>>>>>>>    unset TZ or translate it to a format that Windows will recognize.
>>>>>>>>    I have no idea how difficult that would be.
>>>>>>>
>>>>>>> Impossible to set Windows TZ usefully as it obeys unstated US DST rules 
>>>>>>> (like posixrules, perhaps 2007+?), and may have limits on hour offset 
>>>>>>> magnitudes.
>>>>>>> MS libraries are stuck at POSIX 1996 and C 99 subset compatibility, but 
>>>>>>> non-standard-conformant including which headers contain definitions:
>>>>>>>
>>>>>>>      https://docs.microsoft.com/en-us/cpp/c-runtime-library/compatibility?view=msvc-160 
>>>>>>>
>>>>>>>
>>>>>>> It may be possible to unset TZ when running non-Cygwin programs (possibly 
>>>>>>> behind a CYGWIN env var setting e.g. winnotz) by adding TZ= to 
>>>>>>> conv_envvars, and writing new helper functions env_tz_to_posix to call 
>>>>>>> tzset and env_tz_to_win32 to remove TZ in:
>>>>>>>
>>>>>>>      https://sourceware.org/git/?p=newlib-cygwin.git;f=winsup/cygwin/environ.cc;a=blob 
>>>>>>>
>>>>>>>
>>>>>>> What is the opinion on this from both Windows users and Cygwin patchers?
>>>>>>
>>>>>> I'm not convinced it's worth the trouble.  I haven't seen anyone argue 
>>>>>> that it's useful for Cygwin to set TZ, and I have seen an argument that 
>>>>>> it's harmful:
>>>>>>
>>>>>>    https://cygwin.com/pipermail/cygwin/2017-May/232675.html .
>>>>>>
>>>>>> So I prefer Keith's second suggestion:
>>>>>>
>>>>>>  >> - Cygwin shouldn't set TZ at all by default.
>>>>>
>>>>> It does so in default startup scripts
>>>>
>>>> Right, and I'm agreeing with Bruno (in the message cited above) that the 
>>>> default startup scripts should stop doing that.
>>>>
>>>>> to get the correct behaviour from Cygwin DLL and programs,
>>>>
>>>> Can you be more specific?  What goes wrong if TZ is not set?  I haven't seen 
>>>> any POSIX or Linux documentation that says it should be set, and I've just 
>>>> checked on two different Linux distros that it's not set by default.
> 
> See original message:
> 
>      https://sourceware.org/pipermail/cygwin/2012-January/199548.html
> 
> Looks like the tzdb metadata is not fully populated, including zone 
> abbreviations, though that could be patched to use <%z> like tzdata defaults 
> where countries use only a single local time zone and no name (other than 
> COUNTRY time, and maybe Winter Time or Summer Time, as in many European countries).
> 
>>> I would expect that date, ls, etc. would output UTC, or perhaps PST or EST, 
>>> depending on tzdata builds of Factory (tz -00/unset) and posixrules (Cygwin 
>>> PST, Debian EST) and use during system setup and startup, unless 
>>> /etc/timezone and/or /etc/localtime are set, and used during startup, often 
>>> by systemd, or login by profiles.
>>
>> No, you can 'unset TZ' and everything works fine.  Try it yourself.
> 
> It works incorrectly before 2007 because of DST rule changes.
 > [...]

You obviously know much more about this than I do.  I'm going to drop out of 
this discussion and leave it to people who know what they're talking about.

Ken

-- 
Problem reports:      https://cygwin.com/problems.html
FAQ:                  https://cygwin.com/faq/
Documentation:        https://cygwin.com/docs.html
Unsubscribe info:     https://cygwin.com/ml/#unsubscribe-simple