Why you should normalize text case before matching or deduplicating CSV rows
How '[email protected]' and '[email protected]' silently fail to match as the same record, and the right way (and wrong way) to normalize case.
Before you work through this by hand: you can check your own CSV free and see which of these problems your file actually has — duplicate rows, near-duplicates, broken quoting, encoding damage. It runs in your browser tab, nothing is uploaded, and there's no signup or email.
Why case differences break exact matching
To a computer, "Jane" and "JANE" and "jane" are three different strings unless something explicitly tells it to ignore case. Data entered by different people, exported from different systems, or typed on a phone with autocapitalize all introduce inconsistent casing — and any exact-match comparison (a lookup, a join, a duplicate check) will treat those as unrelated values.
Normalizing in a spreadsheet
Excel/Sheets: =LOWER(A2) converts a cell to all-lowercase; =UPPER(A2) and =PROPER(A2) (title case) are the other two options depending on what you need it for.
Normalizing with pandas
df['email_normalized'] = df['email'].str.lower().str.strip() — lowercasing combined with trimming whitespace (see our whitespace guide) catches the two most common reasons two "identical" values fail to match.
The right way to use a normalized column
Don't overwrite the original display value with the normalized one — keep both. Use the lowercased/trimmed version only as the comparison key for matching/deduping, while keeping the original casing in the column people actually read (e.g. a name is more readable as "Jane Doe" than "jane doe" even though they should match as the same person).
Where this matters most
Email addresses are effectively always safe to lowercase for comparison purposes (the domain part is case-insensitive by standard, and in practice the local part almost always is too). Names and addresses are trickier — normalize them for matching, but never for display, and always keep the original value available in case the normalization was wrong for an edge case.
See CSV Repair & Deduper (offline browser tool) — $19 one-time