In our previous post, we had talked about the character set used in C++. We will next discuss tokens in C++ and their various types.

Let us now find out what a token is.

DEFINITION: A token is the smallest identifiable and individual unit in any program. It is formed when a group of characters is combined together.

There are five types of tokens in C++ language:

  1. Keywords
  2. Variables/Identifiers
  3. Constants/Literals
  4. Punctuators
  5. Operators

1. Keywords

Keywords are a set of words which are used for special purposes while writing a program.

They are also known as reserved words. Their meaning is predefined.

Few examples are: for, while, switch, break, goto, and etc.

2. Variables/Identifiers

An identifier is a name given to a data item in any program.  The value of this identifier can change which is why it can also be called a variable.

For example: int num=3;

Here ‘num’ is an identifier that stores a value 3.

Rules for naming identifiers:

  1. They can be any combination of letters and digits.
  2. An underscore _ can also be used in naming an identifier. No other special character can be used.
  3.  But the first character must be a letter or an underscore _  i.e. identifiers can not begin with a digit.
  4. C++ is a case-sensitive language. So upper and lower case characters are different.
  5. No keyword can be used as an identifier.

Examples :

  • Valid Identifiers : num, NUM, _inPut, my_name etc
  • Invalid Identifiers : 1_num (begins with a digit) , break(keyword) , special character)

3. Literals/Constants

As the name suggests, literals are those data items whose value cannot be changed during the program execution.

They are of four types:

  1. Integer Literals
  2. Floating literals
  3. Character Literals
  4. String Literals

Integer Literals 

These are integers and they do not have any decimal/fractional part. They are of 3 types:

a) Decimal(Base 10) : Sequence of digits that does not begins with 0(zero). Eg : 798, -32, 100 etc

b)Octal (Base 8): Sequence of digits that begin with 0. Eg: 012,056

c) Hexadecimal (Base 16): Sequence of numbers beginning with 0x/0X. Eg: 0XA

Floating Literals

These are numbers that have a decimal part. They are also called real constants.They can be signed and unsigned. For eg : 127.32, -12.9 etc

Character Literals

A single character enclosed in single quotes constitutes a character literal. For example : ‘g’ , ‘z’ etc

We can also represent non-graphic characters (can’t be typed using a keyboard). This is done by using escape sequences.

An escape sequence consists of a backslash followed by a single character.

For example: \n is an escape sequence for newline.

String literals

A string is a group of characters. When a string is enclosed in double quotes, it becomes a string literal.

For example: “hello” is a string literal.

Strings in C++ are automatically appended with a ‘\0’ character to signify its end. So in the above example, a size of  “hello” is 6.

4. Punctuators

They are mostly used for formatting statements and expressions.

Examples are  [ ]  { }   ( )  ,  ;  :  * = etc

5. Operators

They are used for computations on data items. They are of following types:

  • Arithmetic Operators : +,  -,  *,  /
  • Increment/Decrement Operators : ++,–
  • Relational Operators: ==, <, >, <=, >=, !=
  • Logical operators : &&, ||, !
  • Conditional Operators :   ?:

This is all about the various tokens in C++. Let us know your queries on this topic in comments below.

Tags: , , , ,

Leave A Response