Bitwise Operators in

About Bitwise Operators

Down at the hardware level, transistors can only be on or off: two states that we traditionally represent with 1 and 0. These are the binary digits, abbreviated as bits. Awareness of bits and binary is particularly important for systems programmers working in low-level languages.

However, for most of the history of computing the programming priority has been to find increasingly sophisticated ways to abstract away this binary reality. In Python (and many other high-level programming languages), we work with int, float, string and other defined types, up to and including audio and video formats. We let the Python internals take care of (eventually) translating everything to bits.

Nevertheless, using bitwise-operators and bitwise operations can sometimes have significant advantages in speed and memory efficiency, even in a high-level language like Python.

Entering and Displaying Binary Numbers

Unsurprisingly, Python interacts with the user using decimal numbers, but a programmer can override this default. In fact, Python will readily accept an int in binary, hexadecimal, or octal format, and will happily perform mathematical operations between them. For more details, you can review the binary-octal-hexadecimal concept.

Binary numbers are entered with a 0b prefix, just as 0x can be used for hexadecimal (hex numbers are a concise way to represent groups of 4 bits), and oct can be used for octal numbers.

There are multiple ways to convert integers to binary strings, varying in whether they include the 0b prefix and whether they support left-padding with zeros:

# Binary entry.
>>> 0b10111
23

# Converting an int display to binary string, with prefix.
>>> bin(23)  
'0b10111'

>>> number = 23

# Binary without prefix, padded to 8 digits.
>>> format(number, '08b')  
'00010111'

# Same format, but using an f-string.
>>> f"{number} in decimal is {number:08b} in binary and {number:x} in hex" 
'23 in decimal is 00010111 in binary and 17 in hex'

Bitwise Logic

In the bools concept, we discussed the logical operators and, or and not used with Boolean (True and False) values. The same logic rules apply when working with bits.

However, the bitwise equivalents of the logical operators & (and), | (or), ~ (not), and ^ (XOR), are applied to each bit in a binary representation, treating 1 as True ("on") and 0 as False ("off"). An example with the bitwise & might make this clearer:

>>> x = 0b01100110
>>> y = 0b00101010

>>> format(x & y, '08b')
'00100010'

Only positions with a 1 in both the input numbers are set to 1 in the output.

Bitwise & is commonly used as a way to isolate single bits in a compacted set of True/False values, such as user-configurable settings in an app. This enables the value of individual bits to control program logic:

>>> number = 0b0110
>>> number & 0b0001 > 0
False

>>> number & 0b0010 > 0
True

For a bitwise | (or), a 1 is set in the output if there is a 1 in either of the inputs:

>>> x = 0b01100110
>>> y = 0b00101010

>>> format(x | y, '08b')
'01101110'

With the ^ operator for bitwise exclusive or (xor), a 1 is set if it appears in either of the inputs but not both inputs. This symbol might seem familiar from the sets concept, where it is used for set symmetric difference, which is the same as xor applied to sets. If xor ^ seems strange, be aware that this is by far the most common operation in cryptography.

>>> x = 0b01100110
>>> y = 0b00101010

>>> format(x ^ y, '08b')
'01001100'

Finally, there is the ~ operator (the tilde character), which is a bitwise not that takes a single input and inverts all the bits, which might not be the result you were expecting! Each 1 in the representation changes to 0, and vice versa. See the section below for details.

Negative Numbers and Binary Representation

In decimal representation, we distinguish positive and negative numbers by using a + or - sign to the left of the digits. Using these symbols at a binary level proved inefficient for digital computing and raised the problem that +0 is not the same as -0.

Rather than using - and +, all modern computers use a twos-complement representation for negative numbers, right down to the silicon chip level. This means that all bits are inverted and a number is interpreted as negative if the left-most bit (also termed the "most significant bit", or MSB) is a 1. Positive numbers have an MSB of 0. This representation has the advantage of only having one version of zero, so that the programmer doesn't have to manage -0 and +0.

This way of representing negative and positive numbers adds a complication for Python: there are no finite-integer concepts like int32 or int64 internally in the core language. In 'modern' Python, ints are of unlimited size (limited only by hardware capacity), and a negative or bit-inverted number has a (theoretically) infinite number of 1's to the left, just as a positive number has unlimited 0's.

This makes it difficult to give a useful example of bitwise not:

>>> x = 0b01100110
>>> format(x, '08b')
'01100110'

# This is a negative binary (not twos-complement display).
>>> format(~x, '08b')
'-1100111'  

 # Decimal representation.
>>> x
102

# Using the Bitwise not, with an unintuitive result.
>>> ~x
-103

This is not the 0b10011001 we would see in languages with fixed-size integers.

The ~ operator only works as expected with unsigned byte or integer types, or with fixed-sized integer types. These numeric types are supported in third-party packages such as NumPy, pandas, and sympy but not in core Python.

In practice, Python programmers quite often use the shift operators described below and & | ^ with positive numbers only. Bitwise operations with negative numbers are much less common. One technique is to add 2**32 (or 1 << 32) to a negative value to make an int unsigned, but this gets difficult to manage. Another strategy is to work with the ctypes module, and use c-style integer types, but this is equally unwieldy.

Shift operators

The left-shift operator x << y simply moves all the bits in x by y places to the left, filling the new gaps with zeros. Note that this is arithmetically identical to multiplying a number by 2**y.

The right-shift operator x >> y does the opposite. This is arithmetically identical to integer division x // 2**y.

Keep in mind the previous section on negative numbers and their pitfalls when shifting.

>>> x = 8
>>> format(x, '08b')
'00001000'

# A left bit shift. 
>>> x << 2  
32

>>> format(x << 2, '08b')
'00100000'

# A right bit shift. 
>>> format(x >> 2, '08b')
'00000010'