How to Remove Duplicate Lines (Excel, Notepad++ and Online)
Updated October 22, 2026
Duplicate lines creep into email lists, keyword lists, logs and exported data. Here are the quickest ways to remove them, depending on what you have open.
Excel
- Paste the list into one column, select it, then go to Data and click Remove Duplicates. Keep the box for your column ticked and press OK.
- In Excel 365 you can leave the original untouched and use =UNIQUE(A1:A100) in another cell. The result is a list without duplicates.
Google Sheets
- Select the column, then choose Data, Data cleanup, Remove duplicates.
- Or use =UNIQUE(A1:A100) to show the unique values in a new column.
Notepad++
Open the Edit menu, then Line Operations. Choose Remove Duplicate Lines to remove every repeat, or Remove Consecutive Duplicate Lines to remove only repeats that sit directly under each other. Sort the lines first if you use the consecutive option.
Command line
| Where | Command | Keeps original order? |
|---|---|---|
| Linux or macOS | sort -u list.txt | No, the result is sorted |
| Linux or macOS | awk '!seen[$0]++' list.txt | Yes |
| PowerShell | Get-Content list.txt | Select-Object -Unique | Yes |
Why duplicates are sometimes missed
- Different capital letters: Apple and apple are different lines unless you ignore case.
- Invisible differences: a space at the end of a line makes "apple" and "apple " different. Remove extra spaces first.
- Different line endings, which come from files made on different systems.
The fastest way: an online tool
Paste your list into the Remove Duplicate Lines tool and it removes the repeats right away, keeping the first time each line appears and the original order. You can also sort the result with the Sort Lines tool. Nothing is uploaded: it all happens in your browser.
Frequently asked questions
How do I remove duplicates but keep the first one?
Excel's Remove Duplicates, the awk command and the online tool all keep the first occurrence and remove the later ones. sort -u also keeps one of each but sorts the list.
Can I remove duplicate lines and ignore capital letters?
Convert the text to lowercase first with the Lowercase tool, then remove the duplicates. Excel's Remove Duplicates already ignores case.