I had an idea that might make this filename thing easier.
we should move the uppercasing of tokens from the tokenizer to the parser.
We should do it in the function that converts to tokens to pcode. the logic
1.) upercase the tokens
2.) check if it is a keyword, if it is keep it uppercased and proceed if not
proceed with the non-uppercased token.
3.) we would also have to check to make sure that it is not inside a quote
This will simplyfy all future commands for filenames and other things that
may pop up...
From: Mick Brooks
To: Vincent LaBella
Cc: glx-devel@...; glx-general@...
Sent: 1/15/2004 11:06 AM
Subject: Re: [Glx-general] Linux case insensitivity [patch]
On Thu, Jan 15, 2004 at 10:28:58AM -0500, Vincent LaBella wrote:
> One should allow for quoted and unquoted filenames. If we mandate
> filenames need to be quoted then this will break a lot of peoples
> The shell mandates that filenames with spaces and other special
> be quoted so we should following this trend.
> But by all means allow for quoted filenames, but we should try to
> unquoted filenames not to be uppercased as well.
I think that most of this should all just work. If we allow for quoted
filenames, we do it in such a way that the quotes are left in the text
of the token. Then we could have a single utility function that
prepares the filename - if it is enclosed in quotes, strip them and
return the filename. If it is not enclosed in quotes, then the function
does what is currently done, that is return the lowercase version of the
filename. Very little else would need doing, we don't upset old working
scripts and we allow quoted filenames.
Anybody adding new functionality just needs to remember to call the
prepare_filename function on the filename before using it...... or maybe
the tokenizer should prepare the filename as it reads the file? If that
was wanted, I'd need to know if there were any other GLE entities that
can be quoted and aren't filenames.
In order to allow non-quoted case-sensitive filenames, I think it would
need more work - the tokenizer would need to know about context when
finding a filename and starts to become a parser. But then again, it
only needs one extra state variable... maybe it's not too tough
- sorry, I'm rambling.
> I would much prefer the boost tokenizer. If you could get it to work
> would be fantastic! This would alow us to cut out some code...
Well, I need to get my head fully around what the old tokenizer does,
but so far I have an idea how the boost one might be dropped in.