#48 Delay categorization of decimals

[KrisDS]

There is an inherent ambiguity in inputs such as "1,2" which Koopa is having trouble with. Such an example can either be a decimal value (1.2, but using a comma separator for the decimals), or it can be a list two integers (1 and 2).

Deciding whether something is a decimal or just a list of integers is something which is best done during parsing, in a similar way to deciding whether something is a keyword or not. Unfortunately this decision is currently taken in the tokenizer stages.

So the task here is to remove this categorization from the tokenizer stage, and add it to the parser instead.

[KrisDS]

Should be fixed in [r165].