Tokens are the minimal chunk of program that have meaning
to the compiler – the smallest meaningful symbols in the
language.
TOKENS
Token type Description Examples
Keywords
Words with special meaning
to the compiler
int, double, for, auto
Identifiers Names of things that are
not built into the language
cout, std, x, myFunction
Literals Basic constant values
whose value is specified
directly in the source code
“Hello, world!”, 24.3, 0, ‟c‟
Operators Mathematical or logical
operations
+, -, &&, %, <<
Punctuation/Separators Punctuation defining the
structure of a program
{ } ( ) , ;
Whitespace Spaces of various sorts;
ignored by the compiler
Spaces, tabs, newlines,
comments
6