Remove duplicate lines from a list in one step
Email lists, keyword lists, log files and exported columns often contain the same line many times. Paste them here and the tool keeps one copy of each line, in the original order, in a moment.
More than a simple dedupe
Choose to ignore upper and lower case and extra spaces, so "Apple", "apple" and " apple " count as one line. Keep the first or the last copy, keep only the lines that appear once, or show only the lines that repeat. A table lists the most repeated lines with their counts, and the result can be sorted A to Z, by length or in natural number order. Switch the options off if you need an exact comparison, where a capital letter or a trailing space makes two lines different.
How to use Remove Duplicate Lines
- Paste your list into the box, one item per line.
- Pick what to do: remove duplicates, keep only unique lines, or show only duplicated lines.
- Choose how lines are compared: ignore case, trim spaces, and whether blank lines are kept.
- Optionally sort the result or add the number of repeats in front of each line.
- Solve the security check, press Remove duplicates, then copy or download the result.
About Remove Duplicate Lines
Each line is turned into a comparison key. With the default options the key is the line without spaces at both ends and in lower case, so "Apple", "apple" and " apple " match. Turn the options off to compare lines exactly as typed. Remove duplicates keeps one copy of every line, the first or the last as you choose, and keeps the original order. Keep only unique lines drops every line that appears more than once, including its first copy. Show only duplicates lists one copy of each line that repeats. The line shown in the result is always the original text of the copy that was kept. Sorting uses natural order, so item 2 comes before item 10.
Purpose of Remove Duplicate Lines Tool
Use it to clean an email or phone list before a mailing, to dedupe keywords and URLs, to find repeated entries in a log or a list of names, to compare two lists by joining them and keeping only the lines that appear once, or to count how often each value appears.
Limitations of Remove Duplicate Lines
- Only whole lines are compared. Two lines that differ by one character, a plural or a typo are different lines.
- Matching is by text. Accents count as different letters, so cafe and café are not the same line.
- Very large lists (over 500000 lines or 10 million characters) are refused to keep the page fast.
- Blank lines are kept as they are when the keep blank lines option is on. Use the Remove Empty Lines tool to delete them.
Privacy and data
Remove Duplicate Lines runs entirely in your browser. What you type or select is processed on your own device and is never sent to or stored on our server. See our Privacy Policy for details.
Frequently asked questions
How do I remove duplicates but keep the original order?
Leave Sort set to Keep original order. The tool keeps the first copy of each line where it first appeared and drops the later ones.
What is the difference between removing duplicates and keeping unique lines?
Removing duplicates leaves one copy of every line. Keeping only unique lines leaves just the lines that appeared exactly once and drops the repeated ones completely.
Is the duplicate line remover free and safe to use?
Yes. The duplicate line remover is free to use and runs in your browser, so your list is not uploaded, stored or logged.
How do I remove duplicate lines and ignore capital letters?
Turn on ignore case, and Apple and apple are treated as the same line. You can also ignore extra spaces.
Can I see only the duplicate lines or count the repeats?
Yes. Choose to show only duplicates, and the tool can count how many times each line repeats.
Rate Remove Duplicate Lines
No ratings yet. Be the first to rate this tool.
Report a bug or give feedback
Popular tools
Morse Code to Text
Decode Morse code made of dots and dashes into readable text, with flexible separators and error marking.
Text ToolsWebpage URL to Text
Turn a web page address, pasted page source or an HTML file into clean readable text without ads and menus.
Text ToolsASCII to Text Converter
Decode decimal, hex or octal ASCII codes into text, with auto-detection, a code table, control-character names and clear errors for bad codes.
Text ToolsBinary to Text Converter
Decode binary into text as UTF-8, 7-bit ASCII or 16-bit UTF-16, with or without spaces, and find exactly what is wrong when it will not decode.
Text ToolsBMI Calculator
Work out your BMI, category and healthy weight range in metric or imperial units.
Health ToolsBullet List Generator
Turn lines, commas or sentences into a bullet, numbered or checklist, with nested levels, as text, Markdown or HTML.
Text Tools