Re: [Nasm-devel] [nasm:master] BR30730640: Restore preprocessor token concatenation rules

[anonymous c. <nas...@us...>]

> The former candidates for concatenation were (in terms of RE)
>
> expand_smacro
>    [(TOK_ID|TOK_PREPROC_ID)][(TOK_ID|TOK_PREPROC_ID|TOK_NUMBER)]
>
> expand_mmac_params
>    [(TOK_ID|TOK_NUMBER|TOK_FLOAT)][(TOK_ID|TOK_NUMBER|TOK_FLOAT|TOK_OTHER)]

The above permit concatenations that result in invalid
tokens, and therefore need to be tightened. Try these
rules instead:

- whitespace(s)/tab(s) with whitespace(s)/tab(s)
- identifiers with either identifiers or numbers or $/$$
- preprocessor identifiers or numbers with numbers
- $ with identifiers or numbers that don't begin with $

I don't know enough about TOK_FLOAT to comment.