CSV Duplicate Remover
Pick the columns that define a duplicate and clean your list — nothing uploaded, nothing silently deleted.
Nothing you paste or drop is uploaded — the whole table is de-duplicated inside this browser, so it can be used on customer and mailing lists without a security review.
Your table
Or drop a CSV/TSV file —
Which columns make a row a duplicate?
Tick the column(s) that define a duplicate — for example just Email. Untick all to match on the whole row.
Matching on the whole row — a row counts as a duplicate only when every column is identical.
When rows match, keep…
Matching options
Options change matching only — your original rows are always kept for display and export.
Paste or drop a table above — or load the sample — to remove duplicate rows and see exactly how many were dropped.
Why "remove duplicates" needs a key, not just a click
A spreadsheet duplicate is almost never a byte-for-byte copy of another row. A CRM export has the same customer twice because one row came from a web form ("jane.doe@gmail.com ") and the other from a phone call typed by an agent ("Jane.Doe@Gmail.com"). A product catalog has the same SKU twice because one file used a full-width "１２３" from a Japanese POS system and the other used ASCII "123". Excel's built-in Remove Duplicates treats these as different rows and leaves both in place — so the mailing list still gets two emails, and the inventory count is still off by one. Deduplicating well means deciding, on purpose, which columns identify a "real" record and how much spelling variation still counts as the same value — then applying that rule consistently across every row, not eyeballing it.
How to run a dedupe pass
- Paste the table or drop a CSV/TSV file. The delimiter and header row are detected automatically — override them if the guess is wrong (a file with addresses that contain commas often needs Semicolon or Tab picked manually).
- Tick the column(s) under "Which columns make a row a duplicate?" — this becomes the matching key. One column (Email) gives the broadest match; several columns (First name + Last name + Company) give a composite key that only collapses rows agreeing on all of them.
- Choose which copy survives when rows collide: the first occurrence (usually the earliest import) or the last (usually the most recently updated record).
- Pick a country preset as a starting point, then fine-tune individual matching options — each one changes only how keys are *compared*, never what is stored or exported.
- Read the badge: "N rows → M kept, K duplicates removed." Check the "Removed duplicates" tab before trusting the result, especially the first time you dedupe a given file.
The matching pipeline, in order
Each ticked cell's raw text goes through a small pipeline before two rows are compared, and the order matters: whitespace trimming happens first, then Unicode normalization (NFC or NFKC), then case folding, then the optional accent/zero-stripping steps. Two values are "the same" only if every step produces an identical string. The original spelling from your file is never touched — only the temporary comparison key is transformed.
Three worked examples
Row A:
" Jane.Doe@Gmail.com " · Row B: "jane.doe@gmail.com"With "Trim & collapse spaces" and "Ignore upper/lower case" on, both keys reduce to
jane.doe@gmail.com → treated as a duplicate; Row B (the last one) is kept if "keep last"
is selected, so the cleaned file has one row and the removed-duplicates file has Row A.
Row A:
"İstanbul" · Row B: "istanbul"A plain JavaScript
.toLowerCase() turns capital dotted İ into i̇ (a lowercase
i plus a separate combining dot, U+0069 U+0307) — not into plain i — so Row A and Row B
would stay different even with case-insensitive matching on. Turning on the Türkiye preset ("Turkish-safe
case") maps İ→i and I→ı explicitly before comparing, so both rows collapse into one, exactly as a
human proofreader would expect.
Row A:
"00123" · Row B: "123"By default these stay two different rows — an HR export where "00123" and "123" are genuinely different badge numbers must not be silently merged. Only if you explicitly check "Ignore leading zeros" (useful for numeric IDs that were stored inconsistently, e.g. some exports zero-pad to 5 digits and others don't) do the two rows collapse into one.
Matching options at a glance
| Option | Example before | Example after (matches) | Typical use |
|---|---|---|---|
| Trim & collapse spaces | "John Smith " / "John Smith" | equal | Copy-pasted or form-entered names |
| Remove all spaces | "홍 길동" / "홍길동" | equal | Korean names with inconsistent spacing |
| Ignore upper/lower case | "ACME Inc" / "acme inc" | equal | Company or email fields |
| Unicode NFC | decomposed 한글 jamo / precomposed 한글 | equal | Files mixing macOS-decomposed and Windows-composed Korean/Latin text |
| NFKC (fullwidth↔halfwidth) | "ＡＢＣ－１２３" | "ABC-123" | Japanese/Chinese POS or ERP exports |
| Ignore accents | "Café" / "Cafe" | equal | European name and address fields |
| Turkish-safe case | "İstanbul" / "istanbul" | equal | Turkish locale names (see example 2) |
| Ignore leading zeros | "00123" / "123" | equal | Inconsistently zero-padded numeric IDs |
Common mistakes
- Ticking too few columns. Matching on Email alone will also merge two different people who share a shared household inbox. If that's not intended, add a second key column such as Last name.
- Ticking too many columns. A composite key across five columns rarely collides, even for records that are clearly the same customer, because one field (a phone number formatted differently) breaks the match. Start narrow, check the "Removed duplicates" tab, then widen.
- Turning on every normalization option "to be safe." Stacking accent-stripping, NFKC and leading-zero removal on an ID column can merge genuinely different records (postal codes, SKUs). Turn on only the options your specific column needs.
- Trusting the row count without checking the removed tab. The badge tells you how many rows were dropped, not which ones — always skim the "Removed duplicates" panel before you ship the cleaned file, particularly the first time you dedupe a new source.
What this tool does not do
It finds exact matches after normalization — it is not fuzzy matching. "Jon Smith" and "John Smith" (a genuine misspelling, not a formatting difference) will not be merged, nor will "555-0142" match "(555) 014-2" unless your columns already share a normalized format before you paste them in. For approximate name matching, phonetic matching, or record linkage across misspelled fields, a dedicated fuzzy-matching tool is a better fit than exact-key deduplication.
Sources & further reading
Frequently asked questions
Is my list sent to a server, and why can I use this without security approval?
No. This CSV duplicate remover runs 100% in your browser — the table you paste or the file you drop is parsed, normalized and de-duplicated with JavaScript on your own machine, and nothing is uploaded, logged or stored on any server. There is no account and no network request that carries your data. That is exactly why teams that handle customer lists, mailing lists or SKU tables can deduplicate CSV online here even when an upload-based SaaS would fail a security review: the data never leaves the tab. Close the tab and the rows are gone; only your matching options, separator and column choices are remembered in this browser's local storage — never the list itself.
How do I remove duplicate rows by column — for example just by email, ID or phone?
Paste or drop your table, then tick the column(s) that should define a duplicate under "Which columns make a row a duplicate?". Pick a single column such as Email to collapse every record that shares an address, or tick several columns to build a composite key (for example First name + Last name + Company). Only the ticked columns are compared; the rest of each row rides along and is exported untouched. Untick everything to match on the whole row, where a row is a duplicate only when every cell is identical. This column-level key is the main thing a plain "find duplicate rows csv" whole-line match cannot do, and it is what makes the tool practical for a real dedupe mailing list job.
How is this different from Excel's "Remove Duplicates"?
Excel's Remove Duplicates opens the file, deletes rows in place, and never tells you how many it dropped — and it matches raw text, so "ＡＢＣ" vs "ABC" or "café" vs "cafe" slip through. This csv duplicate remover is non-destructive: it produces a cleaned CSV, a separate file of just the removed duplicates, and a report badge showing "N rows → M kept, K duplicates removed". Nothing is deleted silently — you can review exactly which rows were dropped before you use the result. It also normalizes keys the way Excel cannot: whitespace, upper/lower case, fullwidth characters, accents, Turkish-safe case and optional leading zeros, so it catches the duplicates Excel misses when you remove duplicates from csv exports.
Why are "ＡＢＣ" and "ABC", "Café" and "Cafe", or "00123" and "123" treated as same or different?
By default the tool compares text as-is, so "00123" and "123" stay different — that protects employee numbers and postal codes from being silently merged. The country preset and the matching options let you loosen this on purpose. Fullwidth "ＡＢＣ" equals halfwidth "ABC" when NFKC is on (Japan/China preset); "Café" equals "Cafe" when "Ignore accents" is on (Europe preset); and "00123" equals "123" only when you turn on "Ignore leading zeros". Every option changes the matching key only — your original spelling is always preserved for display and export. This locale-aware matching is why the tool finds duplicate rows in a CSV that a plain byte-for-byte compare would leave as separate records.
How large a CSV can it handle, and what happens with very big files?
Tables up to about 50,000 rows are de-duplicated instantly on the main thread. Above that the parsing, normalization and de-duplication move to a background Web Worker with a progress bar and a Cancel button, so the page stays responsive — the tool is built to handle roughly 300,000 rows / 30 MB comfortably. If a file is larger than about 1,000,000 rows or 50 MB, it asks before processing only the first slice rather than freezing or silently truncating your data. Because everything is in-browser there is no upload wait, but the practical ceiling is your device's memory. For anything within those limits you can deduplicate a CSV online here with no install and no sign-up.