Remove duplicates from an array in JavaScript
To remove duplicates from an array in JavaScript, a Set does it in one line and 31 times faster than the filter version. Neither works on objects.
The short answer, and the one to use for arrays of primitives:
const unique = [...new Set(array)];
[...new Set([1, 2, 2, 3, 4, 4, 5])]
// [1, 2, 3, 4, 5]
A Set is a collection that cannot hold the same value twice, so building one from your array drops the duplicates. Spreading it back out gives you an array again.
Order is preserved, and the item kept is the first occurrence:
[...new Set(["c", "a", "c", "b"])]
// ["c", "a", "b"]
Every result on this page came from Node v24.14.1.
The other common approach, and what it costs
You will see this one everywhere:
array.filter((value, index) => array.indexOf(value) === index)
It works by keeping an item only when the first position of that value is the position you are currently at, which is true exactly once per distinct value.
It is also much slower, because indexOf scans the array from the start for every element. Measured on an array of 5,000 items with 2,500 distinct values:
| Approach | Comparisons | Time |
|---|---|---|
[...new Set(arr)] |
~5,000 insertions | 0.2ms |
filter + indexOf |
~25,000,000 | 6.2ms |
Thirty-one times slower at five thousand items, and the gap widens with size, because the Set version grows linearly while the filter version grows with the square of the length. At 50,000 items the filter approach is doing roughly 2.5 billion comparisons.
For a ten-item array it makes no practical difference and either is fine. The reason to default to Set is that arrays have a habit of getting longer than you planned for. If you want the underlying idea, Big O notation explained covers why one of these curves is survivable and the other is not.
Two edge cases in Set equality
Set compares with SameValueZero, which differs from === in one place and matches it in another that surprises people:
[...new Set([NaN, NaN])].length
// 1
NaN === NaN is false, yet a Set treats two NaNs as the same value and keeps one. This is almost always what you want, and it is a genuine difference from the indexOf approach, which cannot find NaN at all.
[...new Set([0, -0])]
// [0]
Positive and negative zero collapse to one entry.
It does not work on objects
This is the limitation that matters most in real code:
const objs = [{ id: 1 }, { id: 1 }];
new Set(objs).size
// 2
Two objects that look identical are two distinct values, because objects compare by reference. A Set removes duplicate references, not duplicate contents, so an array of objects loaded from an API will never deduplicate this way.
The fix is to decide what makes two items the same, then key on that. A Map does it in one line:
const objs = [
{ id: 1, n: "a" },
{ id: 1, n: "b" },
{ id: 2, n: "c" },
];
[...new Map(objs.map((o) => [o.id, o])).values()]
// [{ id: 1, n: "b" }, { id: 2, n: "c" }]
Each object is turned into an [key, value] pair, and building a Map keeps one entry per key. Note which one survives: { id: 1, n: "b" }, the last occurrence, because later entries overwrite earlier ones.
If you want the first occurrence instead, reverse the input first, or filter with a Set of seen keys:
const seen = new Set();
const unique = objs.filter((o) => !seen.has(o.id) && seen.add(o.id));
That reads a little cleverly because seen.add returns the set, which is truthy. A plainer version with an explicit if inside a loop is easier for the next reader, and worth preferring in shared code.
Deduplicating on more than one field
When identity is a combination, build a composite key:
const key = (o) => `${o.type}|${o.name}`;
[...new Map(objs.map((o) => [key(o), o])).values()]
Pick a separator that cannot appear in the values. A pipe is usually safe; a hyphen is not, since it turns up inside names and produces false matches.
For deep equality on whole objects, JSON.stringify as the key works when the field order is consistent and the values are JSON-safe. Both of those are real conditions, so it is a shortcut rather than a general answer.
Counting duplicates instead of dropping them
Often the actual question is which values repeated, and how often. A Map answers that directly:
const counts = new Map();
for (const v of array) counts.set(v, (counts.get(v) ?? 0) + 1);
const duplicates = [...counts].filter(([, n]) => n > 1);
If you only need the duplicates, comparing lengths tells you whether any exist without building anything:
const hasDuplicates = new Set(array).size !== array.length;
That is the cheapest possible check and a good guard before doing more work.
Case-insensitive and normalised deduplication
"Ada" and "ada" are different values to a Set, so a plain dedupe keeps both. When they should count as one, normalise into the key and keep the original as the value:
const seen = new Map();
for (const name of names) {
const key = name.trim().toLowerCase();
if (!seen.has(key)) seen.set(key, name);
}
const unique = [...seen.values()];
The Map holds the normalised key and the original text, so you deduplicate case-insensitively while preserving whatever capitalisation arrived first. Dropping .trim() is the usual reason this appears to fail on user input, since a trailing space makes two otherwise identical strings distinct.
The same shape covers emails, tags, URLs with and without a trailing slash, and anything else where equality is a business rule rather than a language one.
Merging two arrays without duplicates
A Set takes any iterable, so combining and deduplicating is one expression:
const merged = [...new Set([...a, ...b])];
For the values common to both, or in one but not the other, Set has methods for it in current runtimes:
const sa = new Set(a);
const sb = new Set(b);
sa.intersection(sb); // in both
sa.difference(sb); // in a, not in b
sa.union(sb); // either
Before those landed, the equivalent was a.filter((x) => sb.has(x)), which is still perfectly good and works everywhere. Either way, the important part is putting one side into a Set first, so each lookup is a hash check rather than a scan.
Which to use
| Situation | Use |
|---|---|
| Array of primitives | [...new Set(arr)] |
| Objects, keyed by a field | [...new Map(arr.map(o => [o.id, o])).values()] |
| Objects, first occurrence wins | Set of seen keys plus filter |
| Just checking | new Set(arr).size !== arr.length |
| Need the counts | Map of value to count |
The one to stop reaching for is filter with indexOf. It is not wrong, it is just slower than the alternative in every case and no shorter to write.
Three common mistakes
Expecting Set to deduplicate objects. It compares references, so two identical-looking objects are two values. Key on a field with a Map instead.
Forgetting which duplicate survives. The Map approach keeps the last occurrence, and a Set keeps the first. When the duplicates differ in some other field, that choice changes your data. Decide it deliberately rather than discovering it.
Deduplicating after sorting, or sorting after deduplicating, without thinking about it. Set preserves insertion order, so deduplicating a sorted array keeps it sorted. Deduplicating first and sorting after gives the same values in a different order, which matters if you then take the first item.
The Set is often the better data structure
Worth stepping back: if you are deduplicating the same array repeatedly, the array is the wrong shape.
const tags = new Set();
tags.add("js");
tags.add("js");
tags.has("js"); // true
tags.size; // 1
A Set cannot contain duplicates, so there is nothing to clean up later. Membership checks are also much faster than array.includes, for the same reason the timings above differ: a hash lookup rather than a scan.
Convert to an array only at the point you need array behaviour, such as rendering in order or calling map. Holding the data as a Set and spreading at the edges is usually cleaner than holding an array and deduplicating repeatedly.
The one thing to watch is that a Set has no index and no map, so code that needs positions will need the array anyway.
Preventing duplicates instead
Worth asking whether the array should have had duplicates at all. If you are building it yourself, a Set from the start removes the cleanup step:
const seen = new Set();
for (const item of source) seen.add(item);
And if the values come from a server, deduplicating at the source is usually cheaper and more correct than doing it in the browser, since only the server knows whether two records really are the same thing.
Want to try these against real arrays? Start with the JavaScript track, or read sorting an array of objects for the other operation these lists usually need.
More from the blog
What if __name__ == '__main__' actually does in Python
In Python, if __name__ == '__main__' runs code only when you run the file directly, not when it is imported. What __name__ holds and when you need it.
Read moreHow to reverse a list in Python (and a string)
To reverse a list in Python, use reverse() to change it in place, [::-1] for a reversed copy, or reversed() to loop backwards. Which to use and why.
Read moreReady to write some code?
Put this into practice - start your first free lesson. No setup, no credit card.