Class TScanner (unit FPScanner)

Inherits from

TComponent

decralation of the main scanner component

Constructors


constructor Create(AOwner: TComponent);

Public-Deklarationen

known white spaces this is create constructor of the scanner.



Functions

procedure Analyze;

destroy the scanner component

this is the main analysis method


destructor Destroy;

create the scanner component

this is destructor of the scanner.


procedure EASwitch0( ch: Char );

actual column within source file

the following methods are characterizing the internal state machine process a char if the state machine has the state 0 state 0 is the starting state


procedure EASwitch1( ch: Char );

process a char at state 0 ( start

process a char if the state machine has the state 1.


procedure EASwitch11( ch: Char );

process a char at state 9

process a char if the state machine has the state 11 in this state the state machines tries to read a comment.


procedure EASwitch13( ch: Char );

process a char at state 11

process a char if the state machine has the state 13 in this state a zero is already read and it is possible that it is an integer, a real or a hex number


procedure EASwitch14( ch: Char );

process a char at state 13

process a char if the state machine has the state 14 in this state a hex number is read


procedure EASwitch16( ch: Char );

process a char at state 14

process a char if the state machine has the state 14 in this state a hex number is read


procedure EASwitch3( ch: Char );

process a char at state 1

process a char if the state machine has the state 3.


procedure EASwitch5( ch: Char );

process a char at state 3

process a char if the state machine has the state 5.


procedure EASwitch7( ch: Char );

process a char at state 5

process a char if the state machine has the state 7 in this state string enclosed in ' is read


procedure EASwitch9( ch: Char );

process a char at state 7

process a char if the state machine has the state 9 in this state string enclosed in " is read


function GetCount: Integer;

process a char at state 16

get the number of read token


function GetSpecialChars: string;

read method for property Token

internal the specail char are stored in a set of char this method converts the set of char into a string


function GetToken( Index: Integer ) : TToken;

read method for property Count

get the already read token at index Index


function IsKeyword( Value: string ) : Boolean;

read method for property SpecialChars

this method tests if a string is a keyword.


procedure ProcessChar;

test if a string is a keyword

this method reads a char from the source file and adds it to the actual token text


procedure ReadChFromFile( var ch: Char );

process the available char

read a new char from the input file the char #10 is used as global linefeed; MAC file has only #13 as linefeed, havn't they? Sorry! this procedure count the actual row and colum of the input file.


procedure ReadToken;

read a char from file

read the next token with a state machine


procedure SetKeywords( Value: TStringList );

read the next token

this method seems to be unnecessary, but it is very important for the correct work of TStringList in the object inspector


procedure SetSpecialChars( Value: string );

write method for property Keywords

the user defines the special char in a string.


Properties

property AllowIdentifier : Boolean

Published-Deklarationen

property CommentLine : string


property Filename : string


property Keywords : TStringList


property KeywordsCaseSensitive : Boolean


property SpecialChars : string


property Count : Integer

analyze the input file

property Token : TToken

number of found token

Events

event OnTokenRead : TTokenEvent


Variables

CommentIndex : Integer;

user defined event if a new token is read

EAColumn : Integer;

Row of the first token char

EARow : Integer;

recognized token

EAState : Byte;

list of read token

EAText : string;

condition of the state machine

EAToken : TTokenType;

string that belongs to a token

FAllowIdentifier : Boolean;

next not processed char in source file

FCaseSensitive : Boolean;

allow identifiers

FCharacters : TCharSet;

detecting keywords case sensitive

FCommentLine : string;

allowed special chars

FFilename : string;

Private-Decarations

FKeywords : TStringList;

introducing comment chars

FOnTokenRead : TTokenEvent;

list of defined keywords

NextChar : Char;

Column of the first token char

SourceFile : TFile;

index for comment recognition

SourceFileX : Integer;

actual row within source file

SourceFileY : Integer;

file for lexical analysis

TokenList : TList;

filename of source file


Constructors


constructor Create(AOwner: TComponent);

Public-Deklarationen

known white spaces this is create constructor of the scanner. no changes to the inherited create, only initialization of internal and external variables


Functions


procedure Analyze;

destroy the scanner component

this is the main analysis method


destructor Destroy;

create the scanner component

this is destructor of the scanner. it is neccessary to free the internal dynamic data structures


procedure EASwitch0( ch: Char );

actual column within source file

the following methods are characterizing the internal state machine process a char if the state machine has the state 0 state 0 is the starting state


procedure EASwitch1( ch: Char );

process a char at state 0 ( start

process a char if the state machine has the state 1. in this state the state machines tries to read an identifier. an identifier consists of a leading char and any following number or char


procedure EASwitch11( ch: Char );

process a char at state 9

process a char if the state machine has the state 11 in this state the state machines tries to read a comment. a comment begins with the introducing user defined comment string. the first char of the comment is already read by the state 0. it is possible that the first char of the comment string is a special char. the state machine can handle this in state 11.


procedure EASwitch13( ch: Char );

process a char at state 11

process a char if the state machine has the state 13 in this state a zero is already read and it is possible that it is an integer, a real or a hex number


procedure EASwitch14( ch: Char );

process a char at state 13

process a char if the state machine has the state 14 in this state a hex number is read


procedure EASwitch16( ch: Char );

process a char at state 14

process a char if the state machine has the state 14 in this state a hex number is read


procedure EASwitch3( ch: Char );

process a char at state 1

process a char if the state machine has the state 3. in this state a integer or a floating point number is read


procedure EASwitch5( ch: Char );

process a char at state 3

process a char if the state machine has the state 5. in this state floating point number is read


procedure EASwitch7( ch: Char );

process a char at state 5

process a char if the state machine has the state 7 in this state string enclosed in ' is read


procedure EASwitch9( ch: Char );

process a char at state 7

process a char if the state machine has the state 9 in this state string enclosed in " is read


function GetCount: Integer;

process a char at state 16

get the number of read token


function GetSpecialChars: string;

read method for property Token

internal the specail char are stored in a set of char this method converts the set of char into a string


function GetToken( Index: Integer ) : TToken;

read method for property Count

get the already read token at index Index


function IsKeyword( Value: string ) : Boolean;

read method for property SpecialChars

this method tests if a string is a keyword. the keywords are defined in the list Keywords


procedure ProcessChar;

test if a string is a keyword

this method reads a char from the source file and adds it to the actual token text


procedure ReadChFromFile( var ch: Char );

process the available char

read a new char from the input file the char #10 is used as global linefeed; MAC file has only #13 as linefeed, havn't they? Sorry! this procedure count the actual row and colum of the input file.


procedure ReadToken;

read a char from file

read the next token with a state machine


procedure SetKeywords( Value: TStringList );

read the next token

this method seems to be unnecessary, but it is very important for the correct work of TStringList in the object inspector


procedure SetSpecialChars( Value: string );

write method for property Keywords

the user defines the special char in a string. this string must be converted into a set of char. working with a set of char is much easier, but i do not want to implement a new property editor for die usage with the object inspector


Properties


property AllowIdentifier : Boolean

Published-Deklarationen


property CommentLine : string


property Filename : string


property Keywords : TStringList


property KeywordsCaseSensitive : Boolean


property SpecialChars : string


property Count : Integer

analyze the input file


property Token : TToken

number of found token


Events


event OnTokenRead : TTokenEvent


Variables


CommentIndex : Integer;

user defined event if a new token is read


EAColumn : Integer;

Row of the first token char


EARow : Integer;

recognized token


EAState : Byte;

list of read token


EAText : string;

condition of the state machine


EAToken : TTokenType;

string that belongs to a token


FAllowIdentifier : Boolean;

next not processed char in source file


FCaseSensitive : Boolean;

allow identifiers


FCharacters : TCharSet;

detecting keywords case sensitive


FCommentLine : string;

allowed special chars


FFilename : string;

Private-Decarations


FKeywords : TStringList;

introducing comment chars


FOnTokenRead : TTokenEvent;

list of defined keywords


NextChar : Char;

Column of the first token char


SourceFile : TFile;

index for comment recognition


SourceFileX : Integer;

actual row within source file


SourceFileY : Integer;

file for lexical analysis


TokenList : TList;

filename of source file