How to remove duplicate rows from a CSV file (manual method + free checklist)
A step-by-step way to find and remove duplicate rows in a CSV using Excel, Google Sheets, or a short Python script, plus what to watch out for with near-duplicates.
Before you work through this by hand: you can check your own CSV free and see which of these problems your file actually has — duplicate rows, near-duplicates, broken quoting, encoding damage. It runs in your browser tab, nothing is uploaded, and there's no signup or email.
Option 1: Excel or Google Sheets
- Open the CSV file in Excel or Google Sheets.
- Select the full data range including the header row.
- In Excel: Data → Remove Duplicates, then choose which columns must match exactly. In Google Sheets: Data → Data cleanup → Remove duplicates.
- Review the summary showing how many rows were removed before saving, since this step is not easily reversible once you overwrite the file.
Option 2: A short Python script (pandas)
- Install pandas:
pip install pandas - Run:
import pandas as pd; df = pd.read_csv('file.csv'); df.drop_duplicates().to_csv('file_deduped.csv', index=False) - Always write to a new file name so you keep the original as a backup.
The catch: near-duplicates
Both methods above only catch exact duplicate rows. In real exported lead/contact/order lists, the more common problem is near-duplicates — the same person or record with a typo, an extra space, or different capitalization (e.g. "Jane Doe" vs "jane doe " vs "Jane D. Doe"). Spreadsheet dedupe tools will not catch these, and a script that fuzzy-matches safely takes real engineering effort to get right without deleting rows that only look similar.
See also: our CSV cleanup checklist and guide to comparing two CSV files, which runs into the same near-duplicate issue.
See CSV Repair & Deduper (offline browser tool) — $19 one-time