token recognition error at: ‘°’ for degree symbol in ANTLR4.8 C++ target

  antlr, antlr4, c++, unicode

I’m getting

token recognition error at: '°' 

for degree symbol and I can’t figure out what to do about it.

In the lexer I have defined

Degree : ‘uC2B0’ | ‘u00F8’;

The .g4 files for lexer and parser are stored in UTF-8. The input file is stored in UTF-8 as well. When I look at the input file in a hex editor the degree sign is encoded as c2 b0.

Code to read in the input file and parse

std::ifstream is(inputFile);

antlr4::ANTLRInputStream antlrIs(is);
TagsLexer lexer(&antlrIs);
antlr4::CommonTokenStream tokens(&lexer);
TagsParser parser(&tokens);
TagsParser::TagsContext* tagsContext = parser.tags();

Source: Windows Questions C++

LEAVE A COMMENT