Remove Duplicate Lines
Remove duplicate lines from text online, keeping the first occurrence and the original order. Optionally ignore case, extra spaces and empty lines.
Something not working, or missing a feature? Send feedback
About the Remove Duplicate Lines
The Remove Duplicate Lines tool deletes repeated lines from a list and keeps the first occurrence of each, in the original order. It is useful for cleaning up email lists, log extracts, CSV columns, keyword lists and configuration entries without opening a spreadsheet or writing a script.
Everything runs in your browser, so your input is never uploaded to a server.
How to use the Remove Duplicate Lines
- Paste your text, one item per line.
- Choose whether "Ada" and "ada" count as the same line, and whether to ignore surrounding spaces.
- Optionally remove empty lines, then copy or download the cleaned list. The summary shows how many lines were removed.
Examples
Input
[email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
Result
[email protected] [email protected] [email protected]
[email protected] matches [email protected] because the comparison ignores case, and the last [email protected] matches the indented one because surrounding spaces are ignored. The first copy is kept exactly as written.
Input
ERROR timeout WARN slow ERROR timeout error timeout
Result
ERROR timeout WARN slow error timeout
With case sensitivity on, "error timeout" is kept as a different line from "ERROR timeout".
Common mistakes
- Expecting a sorted result. Removing duplicates keeps the original order. Use Sort Lines with "Remove duplicates" on if you want a sorted, unique list.
- Invisible differences. Lines that look the same can differ by a tab, a non-breaking space or a trailing carriage return. Keep "Ignore surrounding spaces" on, and check suspicious lines with the Unicode Escape tool.
Works well with the Remove Duplicate Lines
- Sort Lines: sort the cleaned list.
- Word & Character Counter: count the lines before and after.
- Text & Code Diff: see exactly which lines were removed.
Frequently asked questions
Which copy of a duplicate is kept?
The first one. Later copies are removed and the remaining lines stay in their original order, so nothing is re-sorted.
Does "Ignore surrounding spaces" change my lines?
No. Spaces at the start and end are ignored only when comparing; the line that is kept is written exactly as it was.
Is there a size limit?
No fixed limit: everything runs in your browser, so lists with hundreds of thousands of lines work on a typical computer.
Related tools
Case Converter
Convert text between camelCase, PascalCase, snake_case, kebab-case, CONSTANT_CASE, Title Case and more online. Rename variables and keys in one paste.
Sort Lines
Sort lines of text online alphabetically, naturally (item 2 before item 10), by number or by length, in ascending or descending order. Runs in your browser.
URL Parser
Parse a URL online into protocol, host, port, path, query parameters and fragment. Read long tracking links and debug query strings at a glance.
Cron Expression Parser
Explain any cron expression in plain English and see its next run times in your time zone. Validates crontab syntax, steps, ranges, names and @daily macros.
Word & Character Counter
Count words, characters, lines, sentences and bytes online, with estimated reading time. Check essays, posts and meta descriptions against length limits.
JSON Formatter
Format and beautify JSON with configurable indentation and optional key sorting. Errors are reported with exact line and column.