#1664 `ls --color=auto` sometimes display the control chars

Known_bugs
closed
nobody
MSYS (75)
2012-08-13
2012-08-12
LiTuX
No

Well, it is strange that this `bug` only occurs in my certain folder. ls display `[00m` before the file name, so it is clearly that the Escape char (\033) is missing here, then the align failed for this line. But this `bug` does NOT exist in other folders.

Here is the screenshot. I'm using MinGW/MSYS under windows 8 RP en x64, the bash is running inside MinTTY (because it is sexy :D ). When the linewidth of MinTTY is large enough for three column of those long file names, the bug occurs.

Discussion

  • LiTuX

    LiTuX - 2012-08-12

    a screenshot on a fullscreen minTTY

     
  • Earnie Boyd

    Earnie Boyd - 2012-08-12

    I don't know that we'll be able to resolve this. If you're running in a sh in a console window does it exhibit the issue? You can start a console window via ``start sh --login -i'' to find out.

     
  • Earnie Boyd

    Earnie Boyd - 2012-08-12
    • milestone: 102880 --> 519164
    • status: open --> pending
     
  • LiTuX

    LiTuX - 2012-08-13

    I've tried to run it in bash under cmd, it seems that this issue does not exist in CMD. But you know, CMD is not a normal window which can be resized by using the mouse, we'll have to change the properties with a bigger screen buffer size and window size to simulate the full-screen MinTTY which can contain three column of these file names.

    I've also tried about using rxvt as the terminal emulator, the behavior of ls is just the same as using MinTTY: a `[00m` comes before the filename.

    So I guess it should be an issue about either ls.exe or the hook on the `stdout` of console applications, please check it out.

    BTW, the folder structure comes from libcstl project, a `git clone https://github.com/activesys/libcstl.git && cd libcstl.git && ./configure && make LDFLAGS=-no-undefined && cd src/.libs` will lead to this folder if you need to reproduce this issue with the same environment as I was in.

     
  • Earnie Boyd

    Earnie Boyd - 2012-08-13
    • milestone: 519164 --> Known_bugs
    • status: pending --> closed
     
  • Earnie Boyd

    Earnie Boyd - 2012-08-13

    We suggest using Console2[1] instead of mintty or rxvt. Problems exist with the PTY emulation that cause those terminals to not DTRT in all cases for native runtime (MSVCRT or equivalent dependent) applications.

    [1] https://sourceforge.net/projects/console/

     
  • Earnie Boyd

    Earnie Boyd - 2012-08-13

    BTW as an FYI, you can change the window size of the console window by going to Properties->Layout and changing the width.

     
  • LiTuX

    LiTuX - 2012-08-14

    I know how to change the window size of CMD, and I've tried Console2 years ago. Because my locale is zh-CN, Console2 wasn't so good when meeting multi-byte characters (Hum, MinTTY/rxvt are the same on multibyte :( ). I can get Console2 worked normally only if the cmd is seperated from its window, --hence I don't even know what advantage does Console2 have.

    There exist problems about MinTTY/rxvt, the biggest is it will buffer the output of a CLI application, hence most interactive cmd applications can not work inside mintty. But it is still the best `terminal emulator` under windows as I know. What's more, `ls.exe` is not an interactive application actually, even if MinTTY buffered its output, the Escape char should never be dropped.

     
  • Maximus5

    Maximus5 - 2012-08-14

    Give a try to ConEmu - another open source Windows console emulator.

    https://sourceforge.net/projects/conemu/

    Afaik, it works well with DBCS codepages. Also many features are supported, ANSI X3.64, tabs, palettes, and much more.

     
  • LiTuX

    LiTuX - 2012-08-15

    Wow, ConEmu is great! After some simple test I haven't found any problem in running cmd under ConEmu, thanks for the recommendation.

    But the bug is still there, --even if it is not about `ls`, there's still something wrong, in MinTTY or something else.

     
  • LiTuX

    LiTuX - 2012-08-16

    I may know the cause of this issue now, although it hasn't been verified.

    We've know that MinTTY and rxvt both use a buffer for the output, when the buffer is full (or some other condition?) will the contents flush onto the screen. Hence when ls print those file names with escape chars, perhaps only PART of an escape sequence is sent to the screen, e.g., a single `\033` at the end of the current buffer. Then this buffer flushed, and the following characters `[00m...` came to the very first of this buffer, hence they'll displayed directly on the screen.

    To verify this inference, we must know the buffer size of MinTTY, as well as how many escape sequences does ls added before the file names. If it can be verified that this issue is caused by the buffer size, the only way to fix it is to use a different way in hooking the output of a CLI application, perhaps.

    BTW, it seems that ConEmu does NOT have a buffer issue. (But I may found some other issues about ConEmu on multibyte et al., @conemumaximus )

     
  • Maximus5

    Maximus5 - 2012-08-16

    Just checked `ls --color=auto`. I realize, that `ls` outputs colors directly to windows console, it does NOT use ANSI escape codes. So, the possible garbage as "[00m" can not be problem of ConEmu or any other console.

    Provide redirected output of `dir /s > dir.txt` in your directory.

    PS. ConEmu has not open issues on MBCS, the only found "buffer" problem is DBCS related in Chinese version of Win. And it is api problem in fact.

     
  • Maximus5

    Maximus5 - 2012-08-16

    I mean, execute `dir /s > dir.txt` in your directory from cmd.exe, of course.

     
  • LiTuX

    LiTuX - 2012-08-17

    `ls.exe --color` DO use ANSI escape code for colors in bash, although cmd does not have such an `echo` for testing ANSI escape code like in bash's `echo -e`, it seems that there are NO ansi escape code support in cmd (see ref. [1] ). But `ls.exe --color > dir.txt` shows that ls was using ANSI escape code, so it might be msys-1.0.dll who does come converting on the colors.

    I don't know how to add another attachment now, so you may get the dir.txt in libcstl-2.1.0/src/.lib, which will reproduce the `[00m` issue, from here [2]. Note that this file list won't reproduce the issue directly (via cat), you may need to touch those files with the same name, make the MinTTY at least 125 chars in width, then ls --color. See the screenshot for example.

    --
    [1]. http://en.wikipedia.org/wiki/ANSI_escape_code#Windows_and_DOS
    [2]. https://skydrive.live.com/redir?resid=86603AD62B48A2!1313

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks