How to append to a CSV file in Python

To add new data to a CSV file in Python is a common requirement for log files and data updates. Python's modules let you open files in append mode, 'a', to add rows without an overwrite.

In this article, you'll learn several techniques to append data with different modules. You will find practical tips, see real-world applications, and get advice to debug common issues for a smooth workflow.

Using the csv module to append data

import csv

with open('data.csv', 'a', newline='') as file:
   writer = csv.writer(file)
   writer.writerow(['John', 'Doe', 30, 'Developer'])--OUTPUT--# No visible output, but a new row is added to data.csv

The key to this method is opening the file with newline=''. This argument is essential because it tells Python's CSV writer how to handle line endings, preventing extra blank rows from appearing in your file. It’s a small detail that ensures cross-platform compatibility.

Once the file is open, csv.writer(file) creates a writer object. You can then use its writerow() method to append a new list of data, which it automatically formats into a comma-separated line.

Standard CSV approaches

Beyond the basic writerow() method, Python provides more robust ways to append structured data or add multiple rows in a single operation.

Using csv.DictWriter for structured appending

import csv

with open('employees.csv', 'a', newline='') as file:
   fieldnames = ['first_name', 'last_name', 'age', 'position']
   writer = csv.DictWriter(file, fieldnames=fieldnames)
   writer.writerow({'first_name': 'Jane', 'last_name': 'Smith', 'age': 28, 'position': 'Manager'})--OUTPUT--# No visible output, but a new row is added to employees.csv

When your data has a clear structure, csv.DictWriter offers a more readable and robust approach. It lets you work with dictionaries instead of lists, making your code self-documenting since each value is paired with a descriptive key.

• The fieldnames list is crucial—it defines your CSV's column headers and ensures data is written in the correct order.
• When you call writer.writerow(), you pass a dictionary, and DictWriter automatically maps the values to the right columns based on the keys.

Appending multiple rows at once

import csv

new_data = [
   ['Alice', 'Johnson', 35, 'Designer'],
   ['Bob', 'Williams', 42, 'Analyst'],
   ['Carol', 'Brown', 31, 'Developer']
]
with open('team.csv', 'a', newline='') as file:
   csv.writer(file).writerows(new_data)--OUTPUT--# No visible output, but three new rows are added to team.csv

For bulk operations, the writerows() method is your go-to. It's designed to write multiple rows from an iterable, like a list of lists, in a single, efficient operation.

• This approach is more performant than calling writerow() in a loop, especially when you're adding a large amount of data.
• The method expects your new_data to be structured as a list where each nested list corresponds to a single row you want to add to the file.

Using pandas to append to CSV files

import pandas as pd

new_data = pd.DataFrame([['David', 'Miller', 29, 'Engineer']],
                       columns=['first_name', 'last_name', 'age', 'position'])
new_data.to_csv('staff.csv', mode='a', header=False, index=False)--OUTPUT--# No visible output, but data is appended to staff.csv

For more complex data tasks, the pandas library is a powerhouse. You start by creating a DataFrame—a powerful, table-like structure that holds your new rows. Then, you can use its built-in to_csv() method to write the data. The key is in the parameters you pass to the method:

• Setting mode='a' ensures the data is appended rather than overwriting the file.
• Use header=False to prevent adding a new header row with each append operation.
• index=False stops pandas from writing the DataFrame's row index as a new column.

Advanced CSV techniques

Moving beyond the standard methods, you can write more resilient and scalable code by adding error handling, defining custom CSV formats, and creating reusable components.

Conditional appending with error handling

import csv
import os

file_path = 'records.csv'
new_record = ['Michael', 'Taylor', 33, 'Consultant']

try:
   file_exists = os.path.isfile(file_path)
   with open(file_path, 'a', newline='') as file:
       writer = csv.writer(file)
       if not file_exists:
           writer.writerow(['first_name', 'last_name', 'age', 'position'])  # Write header if new file
       writer.writerow(new_record)
except IOError as e:
   print(f"Error appending to CSV: {e}")--OUTPUT--# No output if successful
# If error occurs: Error appending to CSV: [error details]

This approach builds resilience into your script. It wraps the file operation in a try...except block to gracefully handle potential IOError exceptions, like permission issues. It also adds a smart check before writing:

• Using os.path.isfile(), the code first determines if the CSV already exists.
• If the file is new, it writes the header row to ensure your data stays organized from the start.

This conditional logic prevents duplicate headers on subsequent runs and makes your code more robust.

Using context managers and custom CSV dialects

import csv

csv.register_dialect('custom', delimiter='|', quoting=csv.QUOTE_ALL)

with open('export.csv', 'a', newline='') as file:
   writer = csv.writer(file, dialect='custom')
   writer.writerow(['Sarah', 'Wilson', 36, 'Director'])--OUTPUT--# No visible output, but a new row is added to export.csv with pipe delimiter and quotes

When you're working with non-standard CSVs, Python's "dialects" offer a clean solution. You can define a reusable format with csv.register_dialect() instead of passing formatting options every time. This makes your code much cleaner and more maintainable.

• The delimiter='|' sets the separator to a pipe character.
• quoting=csv.QUOTE_ALL wraps every field in quotes, which can prevent parsing issues.

Simply pass your registered dialect's name to the csv.writer to apply the custom formatting.

Creating a reusable CSV appender class

import csv

class CSVAppender:
   def __init__(self, filename, fieldnames=None):
       self.filename = filename
       self.fieldnames = fieldnames
       
   def append(self, data, is_dict=False):
       with open(self.filename, 'a', newline='') as file:
           if is_dict:
               writer = csv.DictWriter(file, fieldnames=self.fieldnames)
               writer.writerow(data)
           else:
               writer = csv.writer(file)
               writer.writerow(data)

# Usage
appender = CSVAppender('contacts.csv')
appender.append(['Emily', 'Clark', 27, 'Researcher'])--OUTPUT--# No visible output, but a new row is added to contacts.csv

For projects where you frequently append data, creating a class like CSVAppender streamlines your code. This approach bundles the file handling logic into a reusable object, so you don't have to repeat the with open(...) block everywhere.

• The __init__ method initializes the object with the target filename and optional fieldnames.
• The versatile append method handles both lists and dictionaries. It uses the is_dict flag to intelligently switch between csv.writer for simple lists and csv.DictWriter for structured dictionary data.

Move faster with Replit

Replit is an AI-powered development platform that transforms natural language into working applications. Describe what you want to build, and Replit Agent creates it—complete with databases, APIs, and deployment.

The CSV appending techniques from this article can be turned into production-ready tools. Replit Agent can build them directly from your description.

• A real-time analytics logger that captures and appends user interaction events to a daily CSV report.
• A bulk data import tool that processes and appends records from multiple files into a single, consolidated dataset.
• A serverless backend for a contact form that saves each new submission as a structured row in a leads file.

Bring your concept to life. Try Replit Agent to get a working application built from your description.

Common errors and challenges

Even with the right tools, you might run into a few common roadblocks when appending data to CSV files in Python.

Handling file not found errors when appending to csv files

A FileNotFoundError is a classic issue that pops up when your script can't locate the CSV file you're trying to append to. While append mode ('a') often creates the file for you if it's missing, relying on this can sometimes hide path-related bugs in your code.

A more robust solution is to wrap your file-opening logic in a try...except FileNotFoundError block. This allows you to explicitly handle the case where the file doesn't exist—perhaps by creating it and writing the header row—before proceeding with your append operation.

Fixing encoding issues with international characters

If your data includes names, locations, or text with international characters like accents or emojis, you might encounter a UnicodeEncodeError. This error means Python's default text encoding can't process one of the characters you're trying to write.

The fix is straightforward: always specify the encoding when you open the file. By adding encoding='utf-8' to your open() call, you tell Python to use a universal standard that supports a vast range of characters, ensuring your data is saved correctly.

Resolving csv.Error when mixing writerow() and writerows()

A csv.Error can occur if the data structure you provide doesn't match what the writing method expects. It's a common mix-up between writerow() and writerows(), as they require different data shapes.

• The writerow() method is designed for a single row and expects a flat list of values, like ['name', 'email'].
• In contrast, the writerows() method is for multiple rows and expects a list of lists, such as [['name1', 'email1'], ['name2', 'email2']].

If you pass a list of lists to writerow() or a flat list to writerows(), you'll get incorrect output or an error. Always double-check that your data's structure aligns with the method you're using.

Handling file not found errors when appending to csv files

A FileNotFoundError doesn't just happen when the file is missing. It also occurs if you try to write to a file in a directory that doesn't exist. The append mode, 'a', can create a file but not its parent folder. The following code demonstrates this issue.

import csv

# This will fail if the directory doesn't exist
with open('reports/data.csv', 'a', newline='') as file:
   writer = csv.writer(file)
   writer.writerow(['John', 'Doe', 30, 'Developer'])

The open() function fails because the reports/ directory doesn't exist. Python can't create a file inside a missing folder, which triggers the error. The corrected code below shows how to handle this scenario gracefully.

import csv
import os

# Create directory if it doesn't exist
directory = 'reports'
if not os.path.exists(directory):
   os.makedirs(directory)
   
with open('reports/data.csv', 'a', newline='') as file:
   writer = csv.writer(file)
   writer.writerow(['John', 'Doe', 30, 'Developer'])

The fix is to create the directory before writing the file, a common step when organizing output into folders. The code handles this proactively:

• It first checks if the directory exists using os.path.exists().
• If not, it creates the folder with os.makedirs().

This simple check prevents errors because the open() function can't create parent directories on its own—only the file itself.

Fixing encoding issues with international characters

When your data contains non-English characters or symbols like the euro sign (€), Python's default encoding can fail. This happens because the standard writer doesn't know how to handle these special characters, leading to a UnicodeEncodeError. The code below demonstrates this problem.

import csv

data = ['José', 'García', 'Position with € symbol']
with open('international.csv', 'a', newline='') as file:
   writer = csv.writer(file)
   writer.writerow(data)

The writer.writerow() call triggers the error because the open() function doesn't specify an encoding capable of handling characters like é and €. The corrected code below shows how to fix this.

import csv

data = ['José', 'García', 'Position with € symbol']
with open('international.csv', 'a', newline='', encoding='utf-8') as file:
   writer = csv.writer(file)
   writer.writerow(data)

The fix is to explicitly set the encoding by adding encoding='utf-8' to your open() call. This tells Python to use a universal standard that supports a vast range of characters, from accented letters to symbols, which prevents the UnicodeEncodeError. It's a good habit to always specify the encoding—especially when you're working with data from international sources or user-generated content—to ensure everything is saved correctly without errors.

Resolving csv.Error when mixing writerow() and writerows()

A csv.Error often signals a mismatch between your data's structure and the method you're using. The writerow() and writerows() functions are a common source of this confusion because they expect differently shaped data, leading to unexpected output. The code below demonstrates how this mix-up can corrupt your CSV file.

import csv

with open('mixed.csv', 'a', newline='') as file:
   writer = csv.writer(file)
   writer.writerow('Single string')  # This will iterate over each character
   writer.writerows(['Row 1', 'Row 2'])  # This expects lists of lists

The writerow() function treats the string as a list of characters, writing each one to a new column. Likewise, writerows() iterates over the list but then also breaks each string down character by character. The corrected code shows how to provide the right data structure.

import csv

with open('mixed.csv', 'a', newline='') as file:
   writer = csv.writer(file)
   writer.writerow(['Single string'])  # Properly wrapped in a list
   writer.writerows([['Row 1'], ['Row 2']])  # List of lists as expected

The fix is to ensure your data structure matches the method you're using. This error often appears when your data's shape isn't consistent.

• The writerow() method expects a single list of values, like ['value1', 'value2'].
• The writerows() method requires a list of lists, where each inner list is a row, such as [['row1'], ['row2']].

Always double-check your data's structure before writing to prevent the CSV writer from misinterpreting it.

Real-world applications

Beyond fixing errors, these appending skills are essential for real-world tasks like logging system data or collecting information from APIs.

Appending API data to a CSV file with requests

You can automate data collection by using the requests library to fetch information from an API and then appending it directly to a CSV file.

import csv
import requests

response = requests.get('https://jsonplaceholder.typicode.com/users/1')
user = response.json()
with open('api_data.csv', 'a', newline='') as file:
   writer = csv.writer(file)
   writer.writerow([user['id'], user['name'], user['email']])

This script combines data fetching with file writing. First, it uses requests.get() to pull data from a public API. The .json() method then parses the response into a Python dictionary, making the data easy to access.

• The script opens api_data.csv in append mode.
• It extracts specific values like the user's ID, name, and email from the dictionary.
• Finally, writer.writerow() appends this information as a new row in your CSV file.

Monitoring system resource usage with CSV logging

CSV appending is also a great way to build a simple system monitor that logs performance data, such as CPU and memory usage, with a timestamp for each entry.

import csv
import psutil
from datetime import datetime

with open('system_metrics.csv', 'a', newline='') as file:
   writer = csv.writer(file)
   timestamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
   cpu_percent = psutil.cpu_percent()
   memory_percent = psutil.virtual_memory().percent
   writer.writerow([timestamp, cpu_percent, memory_percent])

This script leverages two powerful libraries to gather and record system data. The datetime module provides the current time, which strftime() formats into a standard timestamp string.

• The psutil library does the heavy lifting for system metrics.
• psutil.cpu_percent() fetches the current CPU load.
• psutil.virtual_memory().percent gets the active memory usage.

These collected data points—timestamp, CPU, and memory—are then bundled into a list and appended as a single new row using writer.writerow().

Get started with Replit

Turn your new skills into a real tool. Describe what you want to build to Replit Agent, like "a script that appends daily API weather data to a CSV" or "a tool to log personal expenses to a file."

The agent writes the code, tests for errors, and deploys your app from your description. Start building with Replit.