<< Click to Display Table of Contents >> Home  mIoTa BASIC > mIoTa BASIC Language Reference > Lexical Elements > Tokens

A token is the smallest element of a mIoTa BASIC program that the compiler can recognize. The parser separates tokens from the input stream by creating the longest token possible using the input characters in a left to right scan.

 

mIoTa BASIC recognizes the following kinds of tokens:

 

•Literals

 

•Separators

 

•Identifiers

 

•Operators

 

•Commands and Functions

 

 

Token Extraction Example

 

Here is an example of token extraction. Take a look at the following example code sequence ..

 

 

Note that end_of_time would be parsed as a single identifier, rather than as the keyword END followed by the identifier _of_time because there is no intervening space.

The compiler would parse it as the following four tokens: