Home > other >  Can't make bison to identify a token - always `$undefined`
Can't make bison to identify a token - always `$undefined`

Time:08-26

So I'm currently making a parser with Flex-Bison for python. The lexer part has been completed quite quickly. Here are some parts of the lexer that might be relevant for the current question :

%{
#include <stdio.h>

// many define
#define TOKEN_IDENTIFIER 259
// many define
#define TOKEN_WHITE 294
// many define
#define TOKEN_KEYWORD_FROM 306
// many define
#define TOKEN_KEYWORD_IMPORT 311
// many define

int displayToken(int);
void yyerror(char *sp);

numbers         ((0|[1-9][0-9]*)(\.[0-9] )?)
identifiers     ([a-zA-Z\_][0-9a-zA-Z\_]*)
identation      (^\t )
whites          (\ |\t|\r|\n) 

// many lines
"from"          { return displayToken(TOKEN_KEYWORD_FROM); }
// many lines
"import"        { return displayToken(TOKEN_KEYWORD_IMPORT); }
// many lines

{identifiers}   { return displayToken(TOKEN_IDENTIFIER); }
{identation}    { return displayToken(TOKEN_IDENTATION); }
{whites}        { return displayToken(TOKEN_WHITE); }

// many lines
.               { yyerror("Unknown token!!!"); }
%%

int displayToken(int token) {

#ifdef DEBUG

// logic to help debug

#endif

    return token;
}

Now my parser only tries to parse lines such as :

import math
# or
from math import sqrt

So here is my code :

%{
    #pragma GCC diagnostic ignored "-Wimplicit-function-declaration"
    #include <stdio.h>
    #include <stdlib.h>

    int yylex();
%}

           
  • Related