[antlr4, maven testing] Fix for #4569: predefined.tokens interfering with testing.

[kaby76]

This is a fix for #4569.

The file predefined.tokens, introduced in #4424, breaks Maven testing of the whole repo, which is accomplished by typing mvn clean test in the root directory. Not only does this test each grammar, but it also tests the root pom.xml. Users still test using Maven. So, we are going to reintroduce Maven testing, but in a more "incremental" way.

In this PR, the predefined.tokens file is replaced with an ordered list of tokens in the grammar. I would recommend the file be removed, but it can be kept for reference; the option to include the file is still there, but commented out. In addition, I noticed an error in the file: TOKEN_REF is listed twice.

grammars-v4/antlr/antlr4/predefined.tokens

Lines 9 to 10 in 1dc19f1

	TOKEN_REF = 12
	TOKEN_REF = 13

Mike's antlr-ng contains the error as well. https://github.com/antlr-ng/antlr-ng/blob/54a9460f1cdcdb1aa0b7f90420a9f0dab820351a/src/grammars/predefined.tokens#L9-L10
I changed the file to remove the duplicate. The order of token types is not a requirement for most tools that use this grammar, but antlr-ng does. This change works at least for the Antlr4 Tool because the token vocabulary is imported prior to processing the grammar. https://github.com/antlr/antlr4/blob/6a15cc32dea4e1ec4aa2abbdc90e1b54cf288c58/tool/src/org/antlr/v4/tool/Grammar.java#L360-L362 I don't know whether this works in antlr-ng because the code deviates from that of Antlr4 Tool. https://github.com/antlr-ng/antlr-ng/blob/54a9460f1cdcdb1aa0b7f90420a9f0dab820351a/src/tool/Grammar.ts#L269-L273 tool.process() was moved.

mvn clean test at root now works.

[teverett]

@kaby76 thanks