DKIM-Filter: OpenDKIM Filter v2.11.0 delorie.com 48G9ogin262530 Authentication-Results: delorie.com; dkim=pass (1024-bit key, unprotected) header.d=cygwin.com header.i=@cygwin.com header.a=rsa-sha256 header.s=default header.b=aJiik9VY X-Recipient: archive-cygwin AT delorie DOT com DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org EBAE73858417 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cygwin.com; s=default; t=1726480242; bh=E7ssmjAS80obz9m58oZgAMVtnLgxjdR7vgVZuKniMX4=; h=Subject:To:References:Date:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=aJiik9VYlSU5c10T8GfHUEwv3fOcGjjDIRmV6O2h6rw6qfcLUecJGOmq+tRbsGVuu rPMmfEuhhOSNdrjZGSR8SDbV/rUd3a+3C+UDKnD315EhsF/PL0xED2/eNIH2kgKGy+ 36EB9PMvgEljSystvcMos4LIn2Jbd3yKy+CeC7o4= X-Original-To: cygwin AT cygwin DOT com Delivered-To: cygwin AT cygwin DOT com DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 1D3953858D20 ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 1D3953858D20 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1726480190; cv=none; b=qvj4xAzwLjbD98/c+CAGYCAOktSJYMsPXZAcwADOW3XgLHL8UuJRQzNN2cjGTI+Gzyfwubw6pTcF1H+Hkaluc+cbQ5vDFuT95goM9Y15bL1Z5StoiTjuLX+UDsfPN7NjqM3vp4WHpIIfulqPbKepxAX1I5/r7+0Mz7bKwTvNKrI= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1726480190; c=relaxed/simple; bh=SXDBeRXc/MzyJnArrnmpZbZSki5AtFxzei0xwn4ZjPs=; h=Subject:To:From:Message-ID:Date:MIME-Version; b=ZiBvyrgr8Q4DiGU4C/DEPwnLKuc04mQ0WBgTzIXODlFDLxMhplpr1GQuQgMzvr50FLpb27YUYVeVITGz3kTsOtk6XWnsFCKKHINJRAvaeivUztoGlVHE28gj3bJ1AgbO+M/QG3GzPZmqXzu6zL1wJI5wYTFJkaT0x2NIsrzZUBk= ARC-Authentication-Results: i=1; server2.sourceware.org Subject: Re: readdir() returns inaccessible name if file was created with invalid UTF-8 To: cygwin AT cygwin DOT com References: <96f2253b-791b-b8a0-97dd-8d257eefb9b1 AT t-online DOT de> Message-ID: <6451a249-adcd-9c56-b76e-1b00886cea80@t-online.de> Date: Mon, 16 Sep 2024 11:49:41 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 SeaMonkey/2.53.18.2 MIME-Version: 1.0 In-Reply-To: X-TOI-EXPURGATEID: 150726::1726480181-DB7FC5A6-AFE7B622/0/0 CLEAN NORMAL X-TOI-MSGID: 5f96d0f3-c3af-4324-8152-db6bae90ce13 X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00, FREEMAIL_FROM, KAM_DMARC_STATUS, KAM_NUMSUBJECT, NICE_REPLY_A, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: cygwin AT cygwin DOT com X-Mailman-Version: 2.1.30 Precedence: list List-Id: General Cygwin discussions and problem reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Christian Franke via Cygwin Reply-To: cygwin AT cygwin DOT com Cc: Christian Franke Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: cygwin-bounces~archive-cygwin=delorie DOT com AT cygwin DOT com Sender: "Cygwin" Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by delorie.com id 48G9ogin262530 Christian Franke via Cygwin wrote: > Thomas Wolff via Cygwin wrote: >> >> Am 15.09.2024 um 20:15 schrieb Thomas Wolff via Cygwin: >>> Am 15.09.2024 um 19:47 schrieb Christian Franke via Cygwin: >>>> If a file name contains an invalid (truncated) UTF-8 sequence, open() >>>> does not refuse to create the file. Later readdir() returns a >>>> different name which could not be used to access the file. >>>> >>>> Testcase with U+1F321 (Thermometer): >>>> >>>> $ uname -r >>>> 3.5.4-1.x86_64 >>>> >>>> $ printf $'\U0001F321' | od -A none -t x1 >>>>  f0 9f 8c a1 >>>> >>>> $ touch 'file1-'$'\xf0\x9f\x8c\xa1''.ext' >>>> >>>> $ touch 'file2-'$'\xf0\x9f\x8c''.ext' >>>> >>>> $ touch 'file3-'$'\xf0\x9f\x8c' >>>> >>>> $ ls -1 >>>> ls: cannot access 'file2-.?ext': No such file or directory >>>> ls: cannot access 'file3-': No such file or directory >>>> 'file1-'$'\360\237\214\241''.ext' >>>> file2-.?ext >>>> file3- >>> I don't reproduce this. > > Yes, sorry, the above 'ls' was actually aliased to 'ls --color=auto' > which needs to call stat(). Plain 'ls' does not, so the errors do not > occur then. > > >>> >>> While the file name gets mangled, all resulting file names are valid >>> and >>> listed: >>> In file2 the sequence is turned into U+17B3 but exchanged with the dot. >>> In file3 the same sequence is just dropped. >>> $ ls -1|cat >>> file1-🌡.ext >>> file2-.ឳext >>> file3- >>> >>> However, ls file2* fails, as does ls *. >> On the other hand, ls file3- fails too, so some mapping error occurs >> internally. >> Also, the files cannot be deleted from cygwin (need to use cmd). > > 'rm' using the original names works for file2-..., but not for file3-... > > $ rm -v 'file2-'$'\xf0\x9f\x8c''.ext' > removed 'file2-'$'\360\237\214''.ext' > > $ rm -v 'file3-'$'\xf0\x9f\x8c' > rm: cannot remove 'file3-'$'\360\237\214': No such file or directory > Further tests suggest that the problem only occurs with: - incomplete 4 byte UTF-8 sequences (Unicode above 16 bit) - complete but invalid 3 byte UTF-8 sequences which encode the UTF-16 'high surrogate' range (0xD800..0xDBFF). -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation: https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple