How to read a binary file in Python

To read binary files in Python, you must handle non-text data like images or executables. This process uses specific modes and methods to interpret raw byte streams correctly.

In this article, you'll learn key techniques to read binary data with functions like open() in 'rb' mode. We'll cover practical tips, real-world applications, and common debugging advice.

Reading a binary file using open() with 'rb' mode

with open('sample.bin', 'rb') as file:
binary_data = file.read()
print(f"Read {len(binary_data)} bytes")
print(binary_data[:10]) # Display first 10 bytes--OUTPUT--Read 1024 bytes
b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09'

The key to this operation is the 'rb' mode in the open() function. The 'b' specifies binary mode, telling Python to read the file's raw bytes without trying to decode them as text. This is crucial because it prevents UnicodeDecodeError exceptions that you'd get with non-text files like images or executables.

When you call file.read(), the function returns a bytes object instead of a regular string. That's why the output is prefixed with a b. This object gives you direct access to the file's underlying byte sequence for further processing.

Basic binary file handling techniques

While reading the whole file works, you can handle large binary data more efficiently by reading in chunks, using direct buffer operations, or memory-mapping the file.

Reading binary files in chunks with read()

with open('sample.bin', 'rb') as file:
chunk1 = file.read(4) # Read first 4 bytes
chunk2 = file.read(4) # Read next 4 bytes
print(f"First chunk: {chunk1}")
print(f"Second chunk: {chunk2}")--OUTPUT--First chunk: b'\x00\x01\x02\x03'
Second chunk: b'\x04\x05\x06\x07'

Reading a file in chunks is ideal for managing memory, especially with large files. By passing an integer to file.read(), you specify exactly how many bytes to retrieve. Each call to read() advances the file pointer, so subsequent reads pick up where the last one left off.

• The first file.read(4) call reads the initial four bytes.
• The second call reads the next four bytes, not the same ones.

This method gives you precise control over data processing without overwhelming your system's resources.

Using readinto() for direct buffer reading

import array

with open('sample.bin', 'rb') as file:
buffer = array.array('B', [0] * 10) # Create a byte array
bytes_read = file.readinto(buffer) # Read directly into buffer
print(f"Bytes read: {bytes_read}")
print(f"Buffer content: {buffer}")--OUTPUT--Bytes read: 10
Buffer content: array('B', [0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

The readinto() method provides a more memory-efficient alternative to read(). Instead of creating a new bytes object, it populates a pre-allocated, mutable buffer you provide. This avoids unnecessary memory allocation, which is a significant advantage in performance-sensitive applications.

• You first create a buffer, like a bytearray or an array.array.
• readinto() then fills this buffer with data directly from the file.
• The method returns the number of bytes it successfully read into the buffer.

Memory-mapped binary files with mmap

import mmap
import os

with open('sample.bin', 'rb') as file:
with mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as mmapped_file:
print(f"File size: {len(mmapped_file)} bytes")
print(f"First 5 bytes: {mmapped_file[:5]}")--OUTPUT--File size: 1024 bytes
First 5 bytes: b'\x00\x01\x02\x03\x04'

Memory mapping with Python's mmap module is a powerful technique for handling large files. It's highly efficient because it doesn't load the entire file into RAM. Instead, it maps the file on disk directly into your process's memory space, letting the operating system manage data access on demand.

• The mmap.mmap() function creates a memory-mapped object from the file.
• This object behaves much like a bytes object, so you can use familiar operations like slicing (e.g., mmapped_file[:5]) to read specific byte ranges without reading the whole file first.

Advanced binary file operations

With the basics of reading bytes covered, you can now use powerful modules to parse that binary data into structured information or process it directly in memory.

Parsing binary data with struct

import struct

with open('numbers.bin', 'rb') as file:
# Read 2 integers (8 bytes) in big-endian format
data = file.read(8)
numbers = struct.unpack('>ii', data)
print(f"Unpacked integers: {numbers}")--OUTPUT--Unpacked integers: (16909060, 84281096)

The struct module is your go-to for converting raw bytes into specific data types. You use the struct.unpack() function with a format string that defines the structure of the binary data. This function applies the format to a byte string, returning the data as a Python tuple.

• The format string '>ii' tells Python how to interpret the 8 bytes you read.
• > sets the byte order to big-endian, which dictates how multi-byte numbers are stored.
• ii specifies that you're unpacking two consecutive integers.

Reading binary files with NumPy

import numpy as np

# Read binary file directly into a NumPy array
data = np.fromfile('matrix.bin', dtype=np.float32)
# Reshape if it represents a matrix (e.g., 3x3)
matrix = data.reshape((3, 3))
print(matrix)--OUTPUT--[[1.1 2.2 3.3]
[4.4 5.5 6.6]
[7.7 8.8 9.9]]

For numerical data, NumPy offers a highly efficient way to read binary files. The np.fromfile() function reads data directly into a NumPy array, making it exceptionally fast for large datasets common in scientific computing and machine learning.

• You must specify the dtype, which tells NumPy how to interpret the raw bytes—for example, as 32-bit floats.
• Once loaded, you can use methods like reshape() to structure the one-dimensional array into a meaningful format, like a matrix.

Using the io module for binary data

import io

# Create a binary stream from bytes
binary_data = b'\x00\x01\x02\x03\x04\x05'
binary_stream = io.BytesIO(binary_data)

print(binary_stream.read(2))
binary_stream.seek(4) # Move to position 4
print(binary_stream.read(2))--OUTPUT--b'\x00\x01'
b'\x04\x05'

The io module lets you treat in-memory binary data as if it were a file. By wrapping a bytes object with io.BytesIO, you create an in-memory binary stream. This is useful when you need to process binary data from sources like network APIs without first saving it to disk.

• You can use familiar file methods like read() to get data from the stream.
• The seek() method allows you to move the read pointer to any position, giving you random access to the bytes instead of just reading them sequentially.

Move faster with Replit

Replit is an AI-powered development platform where all Python dependencies come pre-installed, letting you skip setup and start coding instantly.

While the techniques in this article are powerful, Agent 4 helps you move from individual functions to building complete applications. Instead of piecing code together, you can describe the tool you want to build. For example, you could ask it to create:

• A custom file parser that uses struct.unpack() to interpret binary data from a sensor and display it in a readable table.
• A data utility that loads a large numerical dataset with np.fromfile() and reshapes it into a matrix for scientific analysis.
• An image property extractor that reads the header of a JPEG or PNG file to pull out its dimensions and color depth without loading the full image.

Simply describe your app, and Replit will write the code, test it, and fix issues automatically, all within your browser.

Common errors and challenges

When reading binary files, you'll likely encounter a few common pitfalls, from missing files to misinterpreted data.

Handling file not found errors with try-except

One of the most frequent issues is the FileNotFoundError. This error occurs when your script tries to open a file that doesn't exist at the specified path. Instead of letting your program crash, you can handle this gracefully using a try-except block. By wrapping your open() call in a try statement and catching the FileNotFoundError in an except block, you can provide a helpful message or attempt an alternative action.

Debugging endianness issues with struct

Endianness—the order in which bytes are arranged to form multi-byte numbers—is a common source of bugs when using the struct module. If you unpack data using the wrong byte order, you'll get nonsensical values. For example, data written on a big-endian system will be misinterpreted if you read it as little-endian.

• To fix this, always check the documentation for the binary format you're working with to confirm its endianness.
• When using struct.unpack(), explicitly set the byte order with > for big-endian or < for little-endian to ensure your data is interpreted correctly.

Avoiding string vs bytes confusion in binary mode

A fundamental concept that often trips people up is the difference between bytes and str objects. When you open a file in binary mode ('rb'), read operations return bytes objects, not strings. This means you can't perform string-specific operations on the data directly.

Similarly, if you try to write a regular string to a file opened in binary write mode ('wb'), you'll get a TypeError. You must first encode the string into bytes using a method like my_string.encode('utf-8'). Conversely, if you read bytes that you know represent text, you'll need to decode them back into a string with my_bytes.decode('utf-8').

Handling file not found errors with try-except

When Python can't find the file you're trying to open, it raises a FileNotFoundError and immediately stops execution. This is a frequent issue, often caused by a simple typo in the file path.

The code below demonstrates this exact scenario, showing how the program crashes when it can't locate the file.

# This code will crash when the file doesn't exist
file = open('missing_file.bin', 'rb')
binary_data = file.read()
file.close()
print(f"Read {len(binary_data)} bytes")

The script fails because it directly calls open() on a non-existent file. Without a safety net, the resulting FileNotFoundError halts everything. The next example demonstrates how to handle this situation gracefully.

# Using try-except to handle file not found errors
try:
with open('missing_file.bin', 'rb') as file:
binary_data = file.read()
print(f"Read {len(binary_data)} bytes")
except FileNotFoundError:
print("Error: File not found")
except PermissionError:
print("Error: Permission denied")

This solution wraps the file operation in a try block, allowing you to gracefully handle potential issues without crashing the program. It's a fundamental pattern for robust error handling, especially when file paths might be incorrect or permissions are uncertain.

• The except FileNotFoundError block catches the error if the file doesn't exist.
• You can chain multiple except blocks to handle different errors, like PermissionError if you lack access rights.

Debugging endianness issues with struct

Endianness—the order bytes are arranged in memory—is a frequent source of bugs. If you unpack data with the wrong byte order using struct.unpack(), you'll get nonsensical values. The code below shows what happens when the default endianness doesn't match the data's format.

import struct

with open('network_data.bin', 'rb') as file:
data = file.read(4)
value = struct.unpack('i', data)[0] # Using default endianness
print(f"Value: {value}") # May produce unexpected results

The format string 'i' defaults to the system's native byte order, which can misread data from another system. The corrected code below fixes this by explicitly defining the expected byte order for consistent results across platforms.

import struct

with open('network_data.bin', 'rb') as file:
data = file.read(4)
# Explicitly specify endianness ('>i' for big-endian, '<i' for little-endian)
value = struct.unpack('>i', data)[0]
print(f"Value: {value}")

This solution fixes the problem by explicitly telling struct.unpack() which byte order to use. The format string '>i' specifies big-endian, which is common for network data. By being explicit, you ensure your data is read correctly regardless of your machine's native format.

• Use '>' for big-endian (often called network byte order).
• Use '<' for little-endian.

Always be mindful of this when you're reading binary files created on a different system.

Avoiding string vs bytes confusion in binary mode

A frequent mix-up in binary file handling is the difference between str and bytes objects. When a file is opened in binary mode, Python strictly requires bytes. Attempting to write a regular string directly triggers a TypeError, as the following code demonstrates.

# This will raise a TypeError
with open('output.bin', 'wb') as file:
file.write("Hello, binary world!") # String, not bytes

The write() method, when used with a file opened in binary mode ('wb'), can't process a standard str. It expects raw bytes, and this mismatch is what triggers the TypeError. The corrected code below shows how to properly prepare your data.

# Correctly writing bytes to a binary file
with open('output.bin', 'wb') as file:
file.write(b"Hello, binary world!") # Bytes literal
# Alternative: file.write("Hello, binary world!".encode('utf-8'))

This solution works by providing the write() method with the bytes object it expects, which prevents a TypeError. When writing to a file in binary mode, you must convert strings to bytes first. You can do this by creating a bytes literal—prefixing the string with a b—or by using the encode() method. Remember this step anytime you open a file with a mode like 'wb'.

Real-world applications

Beyond debugging, these techniques are essential for real-world tasks like parsing image headers or processing sensor data from compressed archives.

Extracting dimensions from PNG images with struct.unpack()

By combining file seeking with struct.unpack(), you can efficiently parse the header of a PNG file to extract its width and height without processing the full image data.

with open('image.png', 'rb') as f:
f.seek(16) # Skip PNG signature and chunk info
width_height = f.read(8)
import struct
width, height = struct.unpack('>II', width_height)
print(f"Image dimensions: {width}x{height} pixels")

This code efficiently reads a PNG's dimensions without loading the whole image. It uses f.seek(16) to jump directly to the metadata, bypassing the file's signature and header information. From there, f.read(8) grabs the exact 8 bytes containing the width and height.

• The struct.unpack() function then translates these raw bytes into usable numbers.
• The format string '>II' specifies how to interpret the data: as two big-endian (>) unsigned integers (II), which is standard for the PNG format.

Processing sensor data from ZIP archives using zipfile

You can combine the zipfile and struct modules to read and parse binary data directly from a compressed archive without first extracting the files.

import zipfile
import struct

with zipfile.ZipFile('data_archive.zip', 'r') as zip_ref:
with zip_ref.open('sensor_readings.bin') as bin_file:
# Read a series of timestamp-temperature pairs
for _ in range(3):
data = bin_file.read(8)
timestamp, temperature = struct.unpack('ff', data)
print(f"Time: {timestamp:.1f}s, Temperature: {temperature:.1f}°C")

This code processes a binary file directly from a ZIP archive, which avoids having to extract it to disk first. The zipfile.ZipFile context manager opens the archive, and then zip_ref.open() provides a file-like object for the binary data inside.

• The script iterates three times, reading 8-byte chunks from the in-memory file stream.
• Each chunk is unpacked by struct.unpack('ff', data) into two floating-point numbers, representing a timestamp and a temperature reading.

This is a memory-efficient way to handle compressed sensor logs or other structured binary data.

Get started with Replit

Put these techniques into practice with Replit Agent. You can ask it to “build a tool that reads the width and height from a PNG file header” or “write a script that loads floats from a binary file into a NumPy array.”

The Agent will write the code, test for errors, and help you deploy your application. Start building with Replit.