From: <no...@so...> - 2002-01-02 19:46:32
|
Bugs item #453466, was opened at 2001-08-20 12:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=104664&aid=453466&group_id=4664 Category: None Group: None Status: Closed Resolution: None Priority: 5 Submitted By: Satya Nemana (satyan) Assigned to: Hans-Bernhard Broeker (broeker) Summary: top lines invisible but still selectable Initial Comment: Environment: Windows 2K Professional 1-2 Pentium CPU on Dell Systems. Using CSCOPE 15.3 built with GNU gcc/g++ tools downloaded from CYGWIN 5.0 for Windows. If I invoke cscope session on the same 15.3 src directory and search for 'getline', then it displays all lines that contain 'getline' and I can select any of them for example 0 through 9 and get into vi. Functionally it works OK. But some of the lines in the top part of cscope window are not visible. In this case on cscope15.3/src, if I search for getline, then the lines 0 to 6 are not visible. I have added a snapshot of the screen dump as an enclosure with this. This problem is not only for searching as illustrated above, but also occurs sometimes if I try to do "Find this C symbol:", "Find this global definition:" etc. Typically it happens, when the resulting buffers displayed in top cscope window overflows beyond certian large buffer lengths. Selecting 0 to 6 in above example, brings up the vi session. It means functioanally it works. Only the displaying lines are missing. Please note that these top invisible 6 lines appear as blabk lines, which means the 1st visible line (which is 7) is located in 8th row and the all the 7 top rows are blank lines. I don't know if it happens on Solaris with our version. I would like to check it and let you know. ---------------------------------------------------------------------- >Comment By: Hans-Bernhard Broeker (broeker) Date: 2002-01-02 11:46 Message: Logged In: YES user_id=27517 I have a serious problem with accepting this usage/behaviour of Cygwin you describe as "correct" or "valid" --- it's actually a blatant violation of every existing standardization of C file I/O I've ever seen. Cygwin *is* trying to be overly clever, making life miserable for perfectly sane applications in order to "fix" utterly broken ones from the outside. IMHO, any C runtime environment that has an fopen(name,"r") open the file in binary mode is fouled up beyond all repair. This way of calling fopen() is the one and only method of a program to request a text-mode open. Once that's broken, there's no other valid method to achieve text-mode left. > cscope needs to read source files in text mode And that's exactly what it does, as far as the source can influence this: all source files are read via fopen(filename, "r"). If that doesn't result in text mode actually being used, that's a bug in the C runtime. You're correct in guessing that DJGPP doesn't have text/binary mounts --- nor does it ever need them. Instead, the programs are extended to use correct opening modes (adding "b" flags where appropriate) during porting from native Unix to DJGPP. That works perfectly in almost every single case we've ever come across. The only exception from that rule was texinfo, which is somewhat similar to cscope in that it stores file positions inside text-mode .info files. This broke the index search in 'info' whenever someone edited generated files with a DOS text editor. The core of the design error I see here is that *if* an override for the default file opening mode is needed at all, it should be specified by the individual application, but never as a system-wide setting. That's exactly what the "b" flag in fopen() is for, after all. Only the application can ever know whether it's correct to open a given file in text or binary mode, at a given point of its execution. Even a manifestly text-mode file can have to be opened in binary mode by some programs, and then maybe only in particular situations. Cygwin is trying to cure a symptom instead of the disease: badly ported Unix programs that don't have all the "b" flags they need. No correctly written program can ever gain from this feature, and some --- including cscope --- are actually harmed by it. In a nutshell: binary mounts are the root of this problem, not cscope's file I/O code. ---------------------------------------------------------------------- Comment By: Jason Tishler (jlt63) Date: 2002-01-02 10:30 Message: Logged In: YES user_id=86216 > This problem is really a non-issue, and has been for a > while. Once again, I respectfully disagree. I just tried cscope CVS against Cygwin CVS (last updated on 2001-12-28) and the problem remains. cscope does not deal appropriately when the source files are in text mode under a binary mount -- "invisible" lines are displayed. > As far as experiments and documentation could > confirm, it was caused by a bug in a particular release of > Cygwin that incorrectly activated textmode writing even in > cases where the source *explicitly* had requested binary > mode (--> fopen "wb" mode). Cygwin has always respected the fopen()'s "wb" mode. The issue here is on the read side -- not write. cscope needs to read source files in text mode (at least for Cygwin) just in case they are in text mode and the user has requested binary mounts. > The only thing Cygwin has to do to behave correctly is > respect the "b" mode flags as provided by cscope. It does as indicated above. > No automatism on Cygwin's side is needed --- if it has any > effect at all, it could only be to the worse. Once again, the above is not true. Linking cscope against Cygwin's automode.o solves the problem (albeit in a quick and dirty way). However, I would prefer that it is solved in the source. For example, Cygwin's gcc and bash have been changed in *their* source so that they can tolerate source files with CRNL line endings. The unfortunate fact of life is that binary versus text mode issues must be dealt with in any program ported to the Cygwin platform. > Cscope > already works quite nicely on DJGPP, the DOS port of the GNU > toolchain, which effectively proves that all the binary mode > issues are correctly handled in the source. If any problems > remain, they almost certainly are in the runtime > environment. I don't buy the above argument. I'm not familiar with DJGPP, but I presume that it does not have the concept of text/binary mounts. Since fopen() defaults to text mode, this is most likely why DJGPP does not have this problem. On the other hand, Cygwin has a mount table that among other things indicates the default handling for fopen() (and open()). Hence, just because cscope supports DJGPP, it does not necessarily following that it will automatically support Cygwin. Using binary mounts with text mode files is a validate and common Cygwin configuration. If supporting the Cygwin platform is important to the cscope developers, then I encourage them to properly handle this case. If not, then I apologize for wasting your time. ---------------------------------------------------------------------- Comment By: Hans-Bernhard Broeker (broeker) Date: 2002-01-02 08:23 Message: Logged In: YES user_id=27517 This problem is really a non-issue, and has been for a while. As far as experiments and documentation could confirm, it was caused by a bug in a particular release of Cygwin that incorrectly activated textmode writing even in cases where the source *explicitly* had requested binary mode (--> fopen "wb" mode). The only thing Cygwin has to do to behave correctly is respect the "b" mode flags as provided by cscope. No automatism on Cygwin's side is needed --- if it has any effect at all, it could only be to the worse. Cscope already works quite nicely on DJGPP, the DOS port of the GNU toolchain, which effectively proves that all the binary mode issues are correctly handled in the source. If any problems remain, they almost certainly are in the runtime environment. ---------------------------------------------------------------------- Comment By: Jason Tishler (jlt63) Date: 2002-01-02 07:21 Message: Logged In: YES user_id=86216 [Due to overly aggressive spam filtering, your response ended up in my spam folder. So, I didn't notice it until today. I have added a procmail recipe to rectify this problem. Sorry, for the sluggish response time...] > I seriously doubt that any automatic tool will help, here. Hmm...I have empirical evidence to the contrary. > The problem is that some files, although they *look* like > text files, will be completely broken if the automatic > decides that they can be stored in text mode. The issue is > that there are byte indices for fseek()ing into the file > stored in the file itself. Storing the file in text mode > would break those almost certainly. The above is not the semantics of linking with automode.o which is to read in text mode but write in binary mode. So, the above cannot happen. Actually, the opposite, storing the file in binary mode (instead of text mode), can. Hence, tools like cscope would be fine, but ones like MS's notepad wouldn't. > Cscope is quite able to deal with text files, by now Obviously, the above is the best solution and I would be very pleased if the current cscope CVS source has these changes (I haven't checked yet.) I didn't suggest this previously because I wasn't sure if the cscope developers would be willing to go through the trouble. > --- it was Cygwin trying to be overly smart that caused > these problems in the first place. Sorry, I disagree with the above. Cygwin's tar will never convert files from binary to text mode -- even with text mode mounts. My WAG is that satyan untarred the cscope tarball with WinZip or another Win32 tool and not a Cygwin one. ---------------------------------------------------------------------- Comment By: Hans-Bernhard Broeker (broeker) Date: 2001-10-31 05:02 Message: Logged In: YES user_id=27517 I seriously doubt that any automatic tool will help, here. The problem is that some files, although they *look* like text files, will be completely broken if the automatic decides that they can be stored in text mode. The issue is that there are byte indices for fseek()ing into the file stored in the file itself. Storing the file in text mode would break those almost certainly. Cscope is quite able to deal with text files, by now --- it was Cygwin trying to be overly smart that caused these problems in the first place. ---------------------------------------------------------------------- Comment By: Jason Tishler (jlt63) Date: 2001-10-31 04:50 Message: Logged In: YES user_id=86216 > If I invoke cscope session on the same 15.3 src > directory and search for 'getline', then it displays > all lines that contain 'getline' and I can select any > of them for example 1 through 9 and get into vi. I believe that a good (i.e., possibly best) solution to the above is to link cscope with /usr/lib/automode.o. This will cause text mode files to automatically be coverted to binary mode on input, but cscope will continue to write binary files on output. Hence, Cygwin cscope will now be able to deal with text mode files too. This is important in mixed environments where some are using MS VisualStudio and some cscope. It would be nice if cscope's configure under Cygwin would automatically set up to link against automode.o ... ---------------------------------------------------------------------- Comment By: Satya Nemana (satyan) Date: 2001-09-14 10:04 Message: Logged In: YES user_id=302967 Broeker, It looks like this problem is fixed, when I downlaoded the latest CYGWIN environment today, as suspected by you. I request you to close both of my bug reports based on this observation below, after more thorough testing as needed. These bugs do not occur anymore when I downloaded the latest CYGWIN environment with following version numbers. 1. GNU bash, version 2.05.0(8)-release (i686-pc-cygwin) 2. VIM - Vi IMproved 5.8 (2001 May 31, compiled Aug 31 2001 20:36:45) PS: I did not make any cscope source code changes for this. This fixed two problems that I reported earlier in this bugs report, which are as follows: 1. some of the top window lines were invisible. They appear OK if I convert files form DOS to UNIX mode ie. remove \r. 2. vi could not be invoked, if we don't explicitely define EDITOR=vi or CSCOPE_EDITOR=vi. Without defining them, I expected that vi should be automatically defaulted and invoked, but the problem was vi was executed OK, but it exited with some error and its display was cleared immediately by the cscope screen refresh. With this, there is no need to define the environment variables like EDITOR or CSCOPE_EDITOR, which then automatically defaults to vi and then vi works well. Thanks, Satya Nemana. ---------------------------------------------------------------------- Comment By: Satya Nemana (satyan) Date: 2001-08-22 17:05 Message: Logged In: YES user_id=302967 I take this as an update. II added replies to all your questsions below. I was not aware of this magic done by Cygwin internally, which I tend to agree upon. But onething is for sure that this conversion from dos to unix mode, brings up the invisible lines to display, makeing them visible again. It works like a temporary fix. I need to write a little a script to do this conversion before I start using the cscope on Cygwin on Windows. Surprisingly, I noticed that if I "Find a C symbol" for "main" in cscope src using cscope, then main.c dislays visible lines, eventhough main.c is in DOS mode ie \r\n at EOL. So my earlier theory of not nailing down the problem to DOS/UNIX ormat being responsible for invisible lines is not correct, as this "main" case makes an exception already. And also, cscope.out is not fully in DOS/UNIX mode, whether I run it with -k or without -k or single files or many files, because some lines in cscope.out terminate with \n and some others with \r\n as follows: 0000000 c s c o p e 1 5 $ H O M E / 0000020 s a t y a / c s c o p e 1 5 . 3 0000040 / c s c o p e - 1 5 . 3 / s r c 0000060 0 0000100 0 0 0 4 5 1 0 3 4 \n \t @ a l l o 0000120 c . c \n \n 3 5 002 \n \t ~ < 241 d i 0000140 o . h \n > \r \n \n 3 6 002 \n \t ~ < 0000160 241 r 232 g . h \n > \r \n \n 3 7 002 \n 0000200 \t ~ " l i b 277 r y . h \n " \r \n \n 0000220 3 8 002 \n \t ~ " g l o b 256 . h \n 0000240 " \r \n \n 4 0 030 005 c 332 241 \n \t g mount options for cygwin already show up as binmode as follows: $ mount C:\cygwin\bin on /usr/bin type system (binmode) C:\cygwin\lib on /usr/lib type system (binmode) C:\cygwin on / type system (binmode) c: on /cygdrive/c type user (binmode,noumount) e: on /cygdrive/e type user (binmode,noumount) j: on /cygdrive/j type user (binmode,noumount) r: on /cygdrive/r type user (binmode,noumount) s: on /cygdrive/s type user (binmode,noumount) z: on /cygdrive/z type user (binmode,noumount) I agree with you that tar untars all files and puts them into dos mode by default, eventhough the real files inside the tar, as I could see on a solaris system, do not contain \r\n. It is the cygwin that does some magic to make the files appear or covert into dos mode by tar command. ---------------------------------------------------------------------- Comment By: Hans-Bernhard Broeker (broeker) Date: 2001-08-22 04:05 Message: Logged In: YES user_id=27517 You shouldn't dismiss Cygwin's role so quickly. Guess what runtime environment your 'tar' and your 'od' are using --- yes, it's the same Cygwin that your freshly compiled cscope uses, probably. If it weren't for over-smart Cygwin, the files extracted by "tar" wouldn't have been in DOS textfile format, to begin with. And the way cscope opens its files, it *should* automatically cut out the CR in DOS-style linebreaks as it reads a source file. That's how text mode for fopen() is defined. If it doesn't, that's a grave misbehaviour of Cygwin. There should never be DOS-style linebreaks in cscope.out (It's written in binary mode), but from the symptoms you describe, there are. And that's the real problem. Please check your 'mount' options and the Cygwin docs about this. And check the generated cscope.out file if you run cscope -k upon a single source file that has DOS-style CRLF linebreaks. Does cscope.out have CRLF ones, too? ---------------------------------------------------------------------- Comment By: Satya Nemana (satyan) Date: 2001-08-21 12:46 Message: Logged In: YES user_id=302967 No. This is nothing to do with CYGWIN. It is just the file format. I could see that from the octal file dump using the od command, as illsutrated below. With this octal dump, I can prove that if the file contains \r\n at EOL, then this problem occurs. When I untarred your cscope15.3.tar.gz, these files like src/command.c are created in DOS mode. Once I converted into unix mode by replacing \r\n into \n at EOL, then the problem went away. Another way of doing this conversion is that vim support :set fileformat=dos and when I quit with :wq, the file will be saved in unix mode. Before conversion: $ od -c ccommand.c 0000000 / * = = = = = = = = = = = = = = 0000020 = = = = = = = = = = = = = = = = * 0000100 = = = = = = = = = = = = = \r \n 0000120 C o p y r i g h t ( c ) 1 9 After conversion: $ od -c command.c 0000000 / * = = = = = = = = = = = = = = 0000020 = = = = = = = = = = = = = = = = * 0000100 = = = = = = = = = = = = = \n C 0000120 o p y r i g h t ( c ) 1 9 9 I may be wrong, but I still think, this could be fixed in cscope.exe by trimming the last but one char, if it finds \r\n at EOL. I will take a look into it and send the fix. As far as vim.exe is concerned, it is smart enough to distinguish between dos and unix modes and make vi work sameway, so that user do not see these \r\n specially. It displays the file mode at the bottom line of vi session. It also provides conversions by the :set command. But all this is irrelavant to the current problem. cscope can not display the lines that end with \r\n, which can be fixed in cscope, I believe. Sorry, if I am misleading by any chance. ---------------------------------------------------------------------- Comment By: Hans-Bernhard Broeker (broeker) Date: 2001-08-21 11:37 Message: Logged In: YES user_id=27517 I suspect the real problem is that CYGWIN is trying to be overly clever, here. cscope is supposed to already deal with this issue correctly --- all reads of filenames are done in text mode, but all files cscopes writes (including those of the cscope.out file) are in binary mode. Without those changes, porting cscope to DJGPP would never have been possible. But IIRC, Cygwin has a really boneheaded idea that it should be doing DOS<->UNIX textfile conversion behind the scenes, too. For a program carefully ported to non-Unix filesystems already, this transformation breaks it instead of fixing anything. The solution might be to force Cygwin to interpret these sourcefiles as binary mountpoints. ---------------------------------------------------------------------- Comment By: Satya Nemana (satyan) Date: 2001-08-21 11:25 Message: Logged In: YES user_id=302967 Broeker, Congrats. you nailed the problem right on its head. All the cscope sources came up in DOS mode with each line ending with \r\n in stead of a just \n whereas system headers are not in dos format. If the file is in DOS mode, then cscope shows up invisible or blank lines. In stead of expecting the user to convert all files into non-dos mode, it is better for cscope to treat them in same way, by fixing it in cscope.exe. In my example, what ever are displayed are system headers, which are not in DOS mode (line terminates with a \n) and hence these lines are visible. For example, I could not see the command.c entries for getline search. I could see them again, as soon as I removed the \r before \n. Once I converted the command.c file into non-dos mode (as indicated below) by removing the \r before \n, then these lines became visible. I normally use vim.exe, which comes with GNU in CYGWIN, and is linked to vi.exe, so that vi is same as vim. To convert into dos mode actually, I copied stdio.h into a local command.c and did a .r oldcommand.c and removed the actual stdio.h related lines, so that file format is preserved. Summary is that this can be fixed in cscope somehow by treating \r\n specially and converting into \n. Please let me know, if I can fix it here and send it for your review. I will wait for your suggestion. Thanks in advance. ---------------------------------------------------------------------- Comment By: Hans-Bernhard Broeker (broeker) Date: 2001-08-21 03:25 Message: Logged In: YES user_id=27517 More likely than not, this is a Curses-related problem. I had lots of problems with the DOS version of curses, too. Does the display come up correctly again if you force a redraw (by typing Ctrl-L)? Did you try it with a less more traditional Screen size? The screenshot you uploaded seems to be from a session with considerably more than 25 lines. And yet another idea: reproducing your example search for symbol "getline", I only find 7 references (numbered 0 to 6), all contained in the cscope sources. The ones still visible in your screenshot are from system headers. This *may* be part of the problem, or a sign of some difference between local source files and system headers that causes it. It just might be a DOS-style line endings problem, e.g. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=104664&aid=453466&group_id=4664 |