How To Convert Hexadecimal To Bytes In Python

Both hexadecimal and bytes are common terminology in programming languages like Python. Hexadecimal is a base-16 number system that expresses numbers with 16 unique digits. (0-9, A-F).

Bytes, on the other hand, are digital information units composed of eight bits.

When converting hexadecimal to bytes, we need to understand that each pair of hexadecimal digits equals one byte.

Simply divide the hexadecimal string into two-character pairs and turn each pair into its matching byte value to convert a hexadecimal value to a byte value.

Why do we need to convert hexadecimal to bytes in python

Here are some reasons why one might need to convert hexadecimal to bytes in Python:

• Working with low-level data formats: Data in computer systems is frequently expressed in bytes. You may need to convert hexadecimal representations to bytes if you need to interact with data in a low-level format, such as when reading or writing binary files.
• Network protocols:Many network protocols employ binary formats, which are denoted in hexadecimal notation. You may need to convert hexadecimal data to bytes while working with network protocols in Python.
• Encryption and hashing: Bytes are used as inputs or outputs in several cryptographic techniques. You may need to convert hexadecimal data to bytes while dealing with encryption or hashing techniques in Python.
• Debugging and testing: When debugging or testing binary data-related code, it might be useful to convert hexadecimal data to bytes for better examination and comparison.
• Data visualization: You could possibly be able to produce representations that depict data in a more understandable manner by converting hexadecimal data to bytes.

Ways to convert hexadecimal to bytes in python

There are various techniques to convert hexadecimal to bytes in Python. Here are several possibilities:

• Using the built-in bytes.fromhex() function
• Using the binascii module
• Using the bytearray.fromhex() function
• Using a loop to convert each pair of hexadecimal digits to a byte

Approach 1: Using the built-in bytes.fromhex() function

A string of hexadecimal digits can be transformed into a bytes object in Python by using the built-in method bytes.fromhex(). The bytes object produced by this function, which accepts a string of hexadecimal numbers, contains the binary data that the hexadecimal string represents.

# Define a hexadecimal string
hex_string = "48656c6c6f20576f726c64"  # "Hello World" in hexadecimal

# Convert the hexadecimal string to bytes
byte_string = bytes.fromhex(hex_string)

# Print the byte string and its type
print(byte_string)
print(type(byte_string))

Output:

b'Hello World'
<class 'bytes'>

Code Explanation:

• We define a hexadecimal string hex_string that contains the ASCII representation of the phrase “Hello World”.
• We use the bytes.fromhex() function to convert the hexadecimal string to bytes and store the result in the byte_string variable.
• We print the byte string and its type using the print() function.
• The output shows that the byte string has been successfully converted from hexadecimal to ASCII-encoded bytes, and its type is bytes.

Approach 2: Using the binascii module

The unhexlify() function of the Python binascii module offers an easy way to convert hexadecimal strings into bytes. This function returns a bytes object after receiving a string of hexadecimal digits as input.

import binascii

# Define a hexadecimal string
hex_string = "48656c6c6f20576f726c64"  # "Hello World" in hexadecimal

# Convert the hexadecimal string to bytes using binascii
byte_string = binascii.unhexlify(hex_string)

# Print the byte string and its type
print(byte_string)
print(type(byte_string))

Output:

b'Hello World'
<class 'bytes'>

Code Explanation:

• We import the binascii module, which provides several functions for working with binary data.
• We define a hexadecimal string hex_string that contains the ASCII representation of the phrase “Hello World”.
• We use the binascii.unhexlify() function to convert the hexadecimal string to bytes and store the result in the byte_string variable.
• We print the byte string and its type using the print() function.
• The output shows that the byte string has been successfully converted from hexadecimal to ASCII-encoded bytes, and its type is bytes.

Approach 3: Using the bytearray.fromhex() function

The Python method bytearray.fromhex() is used to translate a hexadecimal string into a bytes object. This method returns a byte array containing the corresponding bytes after receiving a string of hexadecimal digits as an input.

# Define a hexadecimal string
hex_string = "48656c6c6f20576f726c64"  # "Hello World" in hexadecimal

# Convert the hexadecimal string to bytes using bytearray
byte_string = bytearray.fromhex(hex_string)

# Print the byte string and its type
print(byte_string)
print(type(byte_string))

Output:

bytearray(b'Hello World')
<class 'bytearray'>

Code Explanation:

• We define a hexadecimal string hex_string that contains the ASCII representation of the phrase “Hello World”.
• We use the bytearray.fromhex() function to convert the hexadecimal string to a mutable byte array object and store the result in the byte_string variable.
• We print the byte string and its type using the print() function.
• The output shows that the byte string has been successfully converted from hexadecimal to ASCII-encoded bytes, and its type is bytearray.

Approach 4: Using a loop to convert each pair of hexadecimal digits to a byte

Hexadecimal numbers are frequently used in Python to represent binary data in a more human-readable way. A loop that iterates over each pair of hexadecimal digits and transforms them to a byte using the built-in int() function is one technique to convert a hexadecimal string to bytes.

This technique necessitates manual conversion implementation, but it allows developers greater control over the conversion and can be beneficial in cases when built-in functions are unavailable.

# Define a hexadecimal string
hex_string = "48656c6c6f20576f726c64"  # "Hello World" in hexadecimal

# Initialize an empty byte array
byte_array = bytearray()

# Convert the hexadecimal string to bytes using a loop
for i in range(0, len(hex_string), 2):
    byte = int(hex_string[i:i+2], 16)
    byte_array.append(byte)

# Print the byte array and its type
print(byte_array)
print(type(byte_array))

Output:

bytearray(b'Hello World')
<class 'bytearray'>

Code Explanation:

• We define a hexadecimal string hex_string that contains the ASCII representation of the phrase “Hello World”.
• We initialize an empty bytearray object called byte_array.
• We use a for loop to iterate over the string hex_string in increments of two characters at a time.
• For each pair of hexadecimal digits, we convert it to an integer using the built-in int() function with a base of 16 (which indicates hexadecimal).
• We append the resulting integer to the byte_array object using the append() method.
• We print the resulting byte_array object and its type using the print() function.
• The output shows that the byte array has been successfully converted from hexadecimal to ASCII-encoded bytes, and its type is bytearray.

Best Approach

When we speak about the best approach for converting hexadecimal to bytes in python, no doubt that bytes.fromhex() is the most widely and best approach for the following reasons:

• Simplicity: The bytes.fromhex() function is a built-in method in Python that can be used to convert a hexadecimal string to a bytes object in a single line of code, without the need for loops or external modules.
• Performance: The bytes.fromhex() function is optimized for performance and can convert large hexadecimal strings to bytes quickly and efficiently.
• Error handling: The bytes.fromhex() function automatically handles common errors that can occur when converting hexadecimal strings, such as invalid characters or odd-length strings.
• Compatibility: The bytes.fromhex() function is available in all versions of Python 3, and can also be used in Python 2.7 with the addition of an import statement.

Sample Problem

Sample Problem 1:

How can you convert a hexadecimal string representation of a movie’s runtime into bytes using the bytes.fromhex() function?

Solution:

• The hex_str variable is defined as a string representing the movie’s runtime in hexadecimal format.
• The replace() method is used to remove the colon separator from the hexadecimal string, since the bytes.fromhex() function expects a string of pairs of hexadecimal digits.
• The bytes.fromhex() function is then used to convert the modified hexadecimal string to bytes.
• The resulting byte data is stored in the byte_data variable.
• Finally, the print() function is used to display the byte data.

Code

# Define a hexadecimal string representing the movie's runtime
hex_str = '01:54:36'

# Remove the colon separator from the string
hex_str = hex_str.replace(':', '')

# Convert the hexadecimal string to bytes using the bytes.fromhex() function
byte_data = bytes.fromhex(hex_str)

# Print the byte data
print(byte_data)

Output:

b'\x01T6'

Sample problem 2:

What is the most efficient way to convert a list of hexadecimal strings representing the episode titles of a TV series into bytes using the binascii module?

Solution:

• The binascii module is imported to access the unhexlify() function.
• The hex_list variable is defined as a list of hexadecimal strings representing episode titles of a TV series.
• A list comprehension is used to iterate over the elements of hex_list and convert each hexadecimal string to bytes using the binascii.unhexlify() function.
• The resulting byte data is stored in the byte_list variable.
• Finally, the print() function is used to display the byte data.

Code:

import binascii

# Define a list of hexadecimal strings representing episode titles
hex_list = ['54686520536176696e672057686f',
            '20476f742041772049742077656e742c20',
            '4c696b652041204d6f7669652c20',
            '416e64204d616472696420416e20',
            '41776172656e657373205265666c656374696f6e']

# Convert each hexadecimal string to bytes using the binascii.unhexlify() function
byte_list = [binascii.unhexlify(h) for h in hex_list]

# Print the byte data
print(byte_list)

Output:

[b'The Saving Who', b' Got Aw It went, ', b'Like A Movie, ', b'And Madrid An ', b'Awareness Reflection']

Sample Problem 3:

How can you use the bytearray.fromhex() function to convert a hexadecimal string representing a music track’s file size into bytes?

Solution:

• The hexadecimal string 03A8 represents the file size of a music track in hexadecimal format.
• The bytearray.fromhex() function is used to convert the hexadecimal string to a byte array.
• The resulting byte array is stored in the byte_arr variable.
• Finally, the print() function is used to display the byte array.

Code

# Define a hexadecimal string representing the file size of a music track
hex_str = '03A8'

# Convert the hexadecimal string to a byte array using the bytearray.fromhex() function
byte_arr = bytearray.fromhex(hex_str)

# Print the byte array
print(byte_arr)

Output:

bytearray(b'\x03\xa8')

Sample Problem 4:

Can you use a loop to convert each pair of hexadecimal digits to a byte to decode a book title that has been encoded as a hexadecimal string?

Solution:

• The hexadecimal string 54686520446976696e652046696e67657273 represents a book title encoded as a hexadecimal string.
• The range() function is used to iterate over each pair of hexadecimal digits in the string.
• The int() function is used to convert each pair of hexadecimal digits to a decimal integer.
• The resulting decimal integers are stored in a list called byte_list.
• The bytes() function is used to convert the byte list to a byte array.
• The decode() method is used to decode the byte array using the UTF-8 encoding to obtain the original book title.
• The resulting book title is stored in the book_title variable.
• Finally, the print() function is used to display the decoded book title.

Code:

# Define a hexadecimal string representing a book title
hex_str = '54686520446976696e652046696e67657273'

# Convert the hexadecimal string to a list of bytes
byte_list = [int(hex_str[i:i+2], 16) for i in range(0, len(hex_str), 2)]

# Convert the byte list to a byte array
byte_arr = bytes(byte_list)

# Decode the byte array using the UTF-8 encoding to obtain the original book title
book_title = byte_arr.decode('utf-8')

# Print the decoded book title
print(book_title)

Output:

The Divine Fingers

Conclusion

Finally, converting hexadecimal to bytes is a typical programming activity, especially when working with low-level data formats, network protocols, encryption and hashing, debugging and testing, and data visualization.

Python has numerous built-in methods for converting hexadecimal to bytes, including bytes.fromhex(), the binascii module, bytearray.fromhex(), and loops to convert each pair of hexadecimal digits to a byte.

24×7 Live Python Homework Help From Expert Tutors! Get Help Now

Author:

Balaraj Gulabal