newlang::Scanner Class Reference

#include <lexer.h>

Inheritance diagram for newlang::Scanner:

Public Member Functions
	Scanner (std::istream *arg_yyin=&std::cin, std::ostream *arg_yyout=&std::cout, SourceType source=nullptr)
virtual	~Scanner ()
void	ApplyDiags (DiagPtr diag=nullptr)
virtual parser::token_type	lex (TermPtr *yylval, parser::location_type *yylloc)
void	set_debug (bool b)
void	LexerError (const char *msg) override

Static Public Member Functions
static BlockType	ParseLexem (const std::string str)

Public Attributes
SourceType	source_base
SourceType	source_string
parser::location_type *	m_loc
std::string	buffer
SourceType	m_macro_data
std::istringstream *	m_macro_iss
parser::location_type	m_macro_loc
size_t	m_macro_count
size_t	m_macro_del
TermPtr	m_macro_body
bool	m_ignore_space
bool	m_ignore_indent
bool	m_ignore_comment
bool	m_ignore_crlf
int	m_bracket_depth

Detailed Description

Scanner is a derived class to add some extra function to the scanner class. Flex itself creates a class named yyFlexLexer, which is renamed using macros to ExampleFlexLexer. However we change the context of the generated yylex() function to be contained within the Scanner class. This is required because the yylex() defined in ExampleFlexLexer has no parameters.

Definition at line 36 of file lexer.h.

Constructor & Destructor Documentation

Scanner()

newlang::Scanner::Scanner	(	std::istream *	arg_yyin = &std::cin,
		std::ostream *	arg_yyout = &std::cout,
		SourceType	source = nullptr )

Create a new scanner object. The streams arg_yyin and arg_yyout default to cin and cout, but that assignment is only made when initializing in yylex().

Definition at line 6 of file lexer.cpp.

~Scanner()

newlang::Scanner::~Scanner ( )

virtual

Required for virtual functions

Definition at line 15 of file lexer.cpp.

Member Function Documentation

ParseLexem()

BlockType newlang::Scanner::ParseLexem ( const std::string str )

static

Definition at line 18 of file lexer.cpp.

References lex(), and result.

ApplyDiags()

void newlang::Scanner::ApplyDiags

(

DiagPtr

diag = nullptr

)

Definition at line 33 of file lexer.cpp.

References m_ignore_comment, m_ignore_crlf, m_ignore_indent, and m_ignore_space.

Referenced by newlang::Parser::Parse().

lex()

virtual parser::token_type newlang::Scanner::lex ( TermPtr * yylval, parser::location_type * yylloc )

virtual

This is the main lexing function. It is generated by flex according to the macro declaration YY_DECL above. The generated bison parser then calls this virtual function to fetch new tokens.

Referenced by newlang::Parser::GetNextToken(), and ParseLexem().

set_debug()

void newlang::Scanner::set_debug

(

bool

b

)

Enable debug output (via arg_yyout) if compiled into the scanner.

LexerError()

void newlang::Scanner::LexerError ( const char * msg )

inlineoverride

Definition at line 89 of file lexer.h.

References buffer, LOG_RUNTIME, m_loc, and newlang::ParserMessage().

Member Data Documentation

source_base

SourceType newlang::Scanner::source_base

Definition at line 48 of file lexer.h.

Referenced by newlang::Parser::GetNextToken().

source_string

SourceType newlang::Scanner::source_string

Definition at line 49 of file lexer.h.

Referenced by newlang::Parser::AstAddTerm(), newlang::ExpandTermMacro(), and newlang::Parser::GetNextToken().

m_loc

parser::location_type* newlang::Scanner::m_loc

Definition at line 50 of file lexer.h.

Referenced by newlang::ExpandTermMacro(), newlang::Parser::GetNextToken(), and LexerError().

buffer

std::string newlang::Scanner::buffer

Definition at line 51 of file lexer.h.

Referenced by LexerError().

m_macro_data

SourceType newlang::Scanner::m_macro_data

Definition at line 60 of file lexer.h.

m_macro_iss

std::istringstream* newlang::Scanner::m_macro_iss

Definition at line 61 of file lexer.h.

Referenced by newlang::ExpandTermMacro(), and newlang::Parser::GetNextToken().

m_macro_loc

parser::location_type newlang::Scanner::m_macro_loc

Definition at line 62 of file lexer.h.

Referenced by newlang::ExpandTermMacro(), and newlang::Parser::GetNextToken().

m_macro_count

size_t newlang::Scanner::m_macro_count

Definition at line 64 of file lexer.h.

Referenced by newlang::Parser::GetNextToken().

m_macro_del

size_t newlang::Scanner::m_macro_del

Definition at line 65 of file lexer.h.

Referenced by newlang::Parser::GetNextToken().

m_macro_body

TermPtr newlang::Scanner::m_macro_body

Definition at line 66 of file lexer.h.

Referenced by newlang::Parser::GetNextToken().

m_ignore_space

bool newlang::Scanner::m_ignore_space

Definition at line 72 of file lexer.h.

Referenced by ApplyDiags().

m_ignore_indent

bool newlang::Scanner::m_ignore_indent

Definition at line 73 of file lexer.h.

Referenced by ApplyDiags().

m_ignore_comment

bool newlang::Scanner::m_ignore_comment

Definition at line 74 of file lexer.h.

Referenced by ApplyDiags().

m_ignore_crlf

bool newlang::Scanner::m_ignore_crlf

Definition at line 75 of file lexer.h.

Referenced by ApplyDiags().

m_bracket_depth

int newlang::Scanner::m_bracket_depth

Definition at line 76 of file lexer.h.

---

The documentation for this class was generated from the following files:

• lib/lexer.h
• lib/lexer.cpp