X-Spam-Check-By: sourceware.org To: cygwin AT cygwin DOT com From: Eric Blake Subject: Re: Escape colour codes Date: Wed, 4 Apr 2007 19:50:24 +0000 (UTC) Lines: 137 Message-ID: References: <4613A562 DOT 5090806 AT byu DOT net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit User-Agent: Loom/3.14 (http://gmane.org/) X-IsSubscribed: yes Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com Thorsten Kampe thorstenkampe.de> writes: > > I need a simple test case. > > It's easy to show that one Cygwin application (lftp) has a problem > with its prompt but it's rather lenghty to show that not lftp is the > culprit but readline making output to Windows Terminals (Cmd, 4NT, > Console, FAR manager, Poderosa) > > 1. Install lftp, start cygwin.bat > > 2. Start lftp. Type > set cmd:prompt "\[\e[1;36m\]>\[\e[m\] " Yep, I reproduced that. Thanks for the better details. > Conclusion: there is something wrong with lftp or readline or the > Terminal. The bug is in lftp. Read on. > 6. Insert > prompt1 "%{\e[1;36m%}>%{\e[m%} " > into your yafcrc and start yafc The bug is in yafc. > 9. Install Python and IPython (http://ipython.scipy.org/moin/) > set > prompt_in1 '\C_White[\#\C_White]\C_LightCyan>>> ' > prompt_out '\C_White[\N\C_White] ' > in your ipythonrc and start IPython. Type 1 [Enter] The bug is in IPython. > > 12. Remove the \001/\002 constructs in line 88 and 89 from > ColorANSI.py in Cygwin IPython > > from > Normal = '\001\033[0m\002' # Reset normal coloring > _base = '\001\033[%sm\002' # Template for all other colors > > to > > Normal = '\033[0m' # Reset normal coloring > _base = '\033[%sm' # Template for all other colors I'm not sure if this is correct. If you pass invisible characters to readline without marking them as such, using \1 and \2, then readline messes up the display width. In other words, I wonder if IPython is adding extra \1 somewhere else in the sequence of things, which is then resulting in the spurious \1 to the cmd terminal. > 13. Recompile yafc, comment out lines 167 and 172 in prompt.c > > case '{': /* begin non-printable character string */ > #ifdef HAVE_LIBREADLINE > // ins = "\001\001"; /* \001 + Actually, that should be: ins = "\001"; /* RL_PROMPT_START_IGNORE */ > RL_PROMPT_START_IGNORE */ > #endif > break; > case '}': /* end non-printable character string */ > #ifdef HAVE_LIBREADLINE > // ins = "\001\002"; /* \001 + RL_PROMPT_END_IGNORE */ And that should be: ins = "\002"; /* RL_PROMPT_END_IGNORE */ In other words, the extra \001 is what is resulting in the smiley faces. > I tried to modify the lftp source, too, but my C knowledge was not > sufficient > > char StartIgn[3], EndIgn[3]; > /* bash adds the extra \001 too, don't know why */ > StartIgn[0] = '\001'; > StartIgn[1] = RL_PROMPT_START_IGNORE; > StartIgn[2] = 0; > EndIgn[0] = '\001'; > EndIgn[1] = RL_PROMPT_END_IGNORE; > EndIgn[2] = 0; Rewrite that as: char StartIgn[2], EndIgn[2]; StartIgn[0] = RL_PROMPT_START_IGNORE; StartIgn[1] = '\0'; EndIgn[0] = RL_PROMPT_END_IGNORE; EndIgn[1] = '\0'; The bug in all three of these programs is that they are adding spurious \1 into the string passed to readline. When you call readline("\001\001invisible\001 \002plain"), then readline assumes that anything between the FIRST \001 and the \002 is invisible (ie. special to the terminal instead of literal output). So readline thinks that it should PRINT the invisible string "\001invisible\001" special to the terminal, followed by the visible string "plain". However, as you noticed, \001 is NOT special to the cmd.com terminal, and results in a smiley face, and readline is now thoroughly confused (it thinks it is waiting for input on position 6, but in reality it is waiting for input on position 8, because you printed literal characters while claiming they were invisible). Bash, on the other hand, DOES map \[ to the sequence '\001\001' inside of parse.y's decode_prompt_string(), BECAUSE it later calls expand_prompt_string() to get rid of the extra \001. It needs to do this so that it can support PS1='$(foo)' (the prompt is the expansion of command foo), and needed a way to tell \[ and \] in PS1 apart from literal \001 and \002 resulting from the expansion of other elements in the prompt string. When the prompt is finally expanded and ready to hand to readline, the extra \001 _used by bash_ is gone, leaving only the SINGLE \001 _used by readline_. In other words, the common bug in all three programs you mentioned is that they copied bash's escape sequences, but NOT bash's round of internal expansion, prior to calling readline. The comment in the lftp sources was rather revealing - if the coder didn't know why bash used an extra \001, they shouldn't have copied that. Meanwhile, I'm asking the upstream readline maintainer if there is any way to output a literal printing \001 (cmd.com hollow smiley) without having it be claimed as invisible (in case you really _wanted_ a smiley in your prompt), and the converse question of if there is any way to output a literal \002 (cmd.com solid smiley) as part of an invisible sequence (in case it is possible that your terminal can change its title bar to include a smiley, for example). Just because most other terminals (rxvt, xterm, ...) are tolerant of unknown control characters, and treat spurious invisible \001 as non-printing characters, doesn't mean that lftp, yafc, or IPython should assume that all terminals behave that way. -- Eric Blake volunteer cygwin readline maintainer -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/