lundi 18 septembre 2017

Efficiently handle tokens in recursive descent parser

I implemented a recursive descent parser. Over the time the number of tokens grew, and now I have about 190 tokens which are all checked in my getNextToken() method each like:

...
else if CHECK_TOKEN("fill-color", z, TOKEN_FILLCOLOR)
else if CHECK_TOKEN("fill-outline-color", z, TOKEN_FILLOUTLINECOLOR)
...

As you can imagine, this slows down the tokenization. Currently the tokenizer is implemented in C. I can imagine to convert the code to C++11 and maybe use a map and lambda functions in order to compare just once.

But I'm rather interested in a proper "compile architecture" solution to such problem. What is a correct way to implement a recursive descent parser so that it efficiently handles hundreds of tokens?

Aucun commentaire:

Enregistrer un commentaire