#187 wrong line numbers for tags in files with random ^Ms

closed-fixed
None
6
2007-09-05
2007-08-14
Ray
No

Hi, Ctags Developers,

I am so proud to be an Exuberant ctags user, and I have been using it for about 2 years. It is working so far so good. However, I found if the file includes some "^M" in it, in my case the header files, which is "0x0d" in windows/msdos system. The Ctags will tread it as a line break, and the following "#define SOME" will totally be affected.

For example
/*---------------------------------------*/
1 #include <stdio>
2
3 void main()^M
4 {
5 printf("Hello World!\n");
6 }
7
8 #define SOME
9 #define OOK
/*---------------------------------------*/

When we build the "tags" file by "ctags -R .", the tag
for SOME and OOK will be all WRONG!, because there is "^M" in line #3.

Please note, ^M in VIM is not just two characters, "^"+"M", it is <CTRL-V>+<CTRL-M>, in hex it is 0x0d.

When we deal with a large project with a lot of source codes, if someone put the ^M in their files from different systems, Ctags will get the wrong tag positions. To make this easier is just to make some condition in Ctags, if this is the MS-Windows system, then don't treat "^M" as a line break, and count it as a line number, even though it should be counted in MAC OS.

Thanks for your attention!
Ray,

Discussion

  • Ray

    Ray - 2007-08-15
    • priority: 5 --> 6
     
  • Elliott Hughes

    Elliott Hughes - 2007-08-19

    Logged In: YES
    user_id=1127237
    Originator: NO

    works for me with the current version of Ctags in subversion:

    lithium:~/Projects/ctags/trunk$ cat -nv Test/bug1773926.cpp
    1 #include <cstdio>
    2
    3 void main()^M
    4 {
    5 printf("Hello World!\n");
    6 }
    7
    8 #define SOME
    9 #define OOK
    lithium:~/Projects/ctags/trunk$ ./dctags -n -f - Test/bug1773926.cpp
    OOK Test/bug1773926.cpp 9;" d file:
    SOME Test/bug1773926.cpp 8;" d file:
    main Test/bug1773926.cpp 3;" f
    lithium:~/Projects/ctags/trunk$

    works with ctags 5.6, too:

    lithium:~/Projects/ctags/tags/Ctags-5_6$ ./ctags -n -f - ../../trunk/Test/bug1773926.cpp
    OOK ../../trunk/Test/bug1773926.cpp 9;" d file:
    SOME ../../trunk/Test/bug1773926.cpp 8;" d file:
    main ../../trunk/Test/bug1773926.cpp 3;" f
    lithium:~/Projects/ctags/tags/Ctags-5_6$

    the relevant code doesn't seem to have changed since Ctags 5.1, which is the oldest i have available.

     
  • Elliott Hughes

    Elliott Hughes - 2007-08-19
    • assigned_to: nobody --> elliotth
     
  • Ray

    Ray - 2007-08-25

    Logged In: YES
    user_id=1867368
    Originator: YES

    As I emphasized, if you just simply copied the code I provided, then
    everything works fine, but, when you use Vim editor, in windows system
    to insert some special characters, like
    , then the problem will
    appear,

    I have tried this many times, and it is absolutely reproduceable.

     
  • Elliott Hughes

    Elliott Hughes - 2007-08-25

    Logged In: YES
    user_id=1127237
    Originator: NO

    the output i pasted below was "cat -v", not "cat". that file *does* have a control-M in it.

     
  • Ray

    Ray - 2007-08-28

    test file for your testing,

     
  • Ray

    Ray - 2007-08-28

    Logged In: YES
    user_id=1867368
    Originator: YES

    I uploaded a file for your testing, my environment is windows xp, please use your ctags.exe to see if this error can be produced or not, thanks!
    File Added: test.c

     
  • Elliott Hughes

    Elliott Hughes - 2007-08-29

    Logged In: YES
    user_id=1127237
    Originator: NO

    good idea. can you show me the command-line you're using, and attach the resulting tags file too, please? also, the output of "ctags --version" so i can see exactly what you're running.

     
  • Ray

    Ray - 2007-08-31

    Logged In: YES
    user_id=1867368
    Originator: YES

    Per your request,

    Command Line:
    -------------
    ctags.exe -R .

    Version Info:
    --------------------
    Exuberant Ctags 5.6, Copyright (C) 1996-2004 Darren Hiebert
    Compiled: Aug 13 2007, 22:41:24
    Addresses: <dhiebert@users.sourceforge.net>, http://ctags.sourceforge.net
    Optional compiled features: +win32, +internal-sort

     
  • Ray

    Ray - 2007-08-31

    Logged In: YES
    user_id=1867368
    Originator: YES

    And I also tried the original build

    Exuberant Ctags 5.6, Copyright (C) 1996-2004 Darren Hiebert
    Compiled: Jul 30 2006, 16:12:20
    Addresses: <dhiebert@users.sourceforge.net>, http://ctags.sourceforge.net
    Optional compiled features: +win32, +regex, +internal-sort

     
  • Elliott Hughes

    Elliott Hughes - 2007-09-02

    Logged In: YES
    user_id=1127237
    Originator: NO

    here's the content of the submitted file:

    lithium:~/Projects/ctags/trunk$ cat -vn ~/Desktop/test.c
    1 #include <stdio.h>^M
    2 ^M
    3 #define ERROR_HAPPENED^M 50 ^M^M^M
    4 #define OK 2^M
    5 #define NEXT_DEFINE 3^M
    6 ^M
    7 int main(int argc, char* argv[])^M
    8 {^M
    9 printf("Hello world\n");^M
    10 return 0;^M
    11 }^M
    12 ^M
    lithium:~/Projects/ctags/trunk$

    and here are the generated tags:

    lithium:~/Projects/ctags/trunk$ cat tags
    !_TAG_FILE_FORMAT 2 /extended format; --format=1 will not append ;" to lines/
    !_TAG_FILE_SORTED 1 /0=unsorted, 1=sorted, 2=foldcase/
    !_TAG_PROGRAM_AUTHOR Darren Hiebert /dhiebert@users.sourceforge.net/
    !_TAG_PROGRAM_NAME Exuberant Ctags //
    !_TAG_PROGRAM_URL http://ctags.sourceforge.net /official site/
    !_TAG_PROGRAM_VERSION 5.6b1 //
    ERROR_HAPPENED /home/elliotth/Desktop/test.c 3;" d file:
    NEXT_DEFINE /home/elliotth/Desktop/test.c 8;" d file:
    OK /home/elliotth/Desktop/test.c 7;" d file:
    main /home/elliotth/Desktop/test.c 10;" f
    lithium:~/Projects/ctags/trunk$

    the problem is caused by line 3, which contains a ^M in the middle of the line, and multiple ^Ms at the end of the line.

    i'm not sure we can fix this cleanly. i've never seen a Mac OS 9 file (just '\r's) in real life, but then i'd not seen a file as random as the attached "test.c" before, either. i'm going to reassign this to Darren because he'll have to decide.

     
  • Elliott Hughes

    Elliott Hughes - 2007-09-02
    • assigned_to: elliotth --> dhiebert
    • summary: wrong number in define, when 0x0d find --> wrong line numbers for tags in files with random ^Ms
     
  • Ray

    Ray - 2007-09-02

    Logged In: YES
    user_id=1867368
    Originator: YES

    Thanks to your effective responses, and hopefully this problem is going to be completely fixed, because I found it in my projects files, of course written by someone else, and I got a little doubt about ctags until I found this was caused by ^Ms.

     
  • Darren Hiebert

    Darren Hiebert - 2007-09-05
    • status: open --> closed-fixed
     
  • Darren Hiebert

    Darren Hiebert - 2007-09-05

    Logged In: YES
    user_id=38016
    Originator: NO

    I changed the newline canonicalization to process a lone CR as a newline only when running on MacOS 9.
    File Added: bug1773926.diff

     
  • Darren Hiebert

    Darren Hiebert - 2007-09-05

    Patch to correct bug

     

Log in to post a comment.