JavaScript: How to Filter Duplicate Items in Array

The most straightforward method for that is the JavaScript Set method.

Remember that math subject about Venn diagram?

In it, the Set cannot have duplicates. Like the word "BOB". The set from the word will be {B, O}, and not {B, O, B}.

The similar concept in JavaScript Set.

This Set direct filtering works only for String and Number items from an Array.

---

Input Array

For example, we have these:

const myNumber = [0, 2, 3, 4, 5, 2, 0, 4] const myString = ["a", "b", "c", "dee", "dee", "doo", "bee"] const myMix = [2, "b", 2, "dee", "dee", 0, "b"]

---

Filtering Duplicates Using Set Method

Then the straightforward method to filter the duplicates can be done like so:

[...new Set(inputArray)]

The new Array from that will consist of unique item.

---

Helper Function

Let's wrap that method into a helper function:

function removeDuplicates(inputArray) { return [...new Set(inputArray)] }

Or, using arrow function syntax:

const removeDuplicates = (inputArray) => [...new Set(inputArray)]

---

Usage

Then we can use that helper function:

const myUniqueNumber = removeDuplicates(myNumber) // INPUT myNumber = [0, 2, 3, 4, 5, 2, 0, 4] // OUTPUT myUniqueNumber = [0, 2, 3, 4, 5] const myUniqueString = removeDuplicates(myString) // INPUT myString = ["a", "b", "c", "dee", "dee", "doo", "bee"] // OUTPUT myUniqueString = ["a", "b", "c", "dee", "doo", "bee"] const myUniqueMix = removeDuplicates(myMix) // INPUT myMix = [2, "b", 2, "dee", "dee", 0, "b"] // OUTPUT myUniqueMix = [2, "b", "dee", 0]

---

How About Input Object item type?

We can first try transform that with JSON.stringify to make the Object as String, then we can use those transformed items. Then we use JSON.parse to revert it back.

const removeDups = (inputArray) => { try { const strings = inputArray.map((item) => JSON.stringify(item)) const filtered = [...new Set(strings)] const output = filtered.map((item) => JSON.parse(item)) return output } catch (error) { console.error(error) return inputArray } }

Let's try that:

removeDups([{a: 1}, {a: 1}, {b: 2}, [0, 2, 3], [0, 2, 3], "a", "a"]) // OUTPUT:  [{"a": 1}, {"b": 2}, [0, 2, 3], "a"]

Please be careful if the object is large using the function above. We need to make sure our input will not be "some wonky giants".

Anyway, we can use map approach first to avoid creating full new Set to make it more efficient. But I won't show it here because that's too specific edge-case, and it depends on what case.

And keep in mind, JSON.stringify for this kind of input { theKey: undefined } will completely omit theKey.

JSON.stringify({ theKey: undefined }) // OUTPUT: '{}' JSON.parse(JSON.stringify(undefined)) // SyntaxError: "undefined" is not valid JSON

And of course, if there's Function in the array, it will also be "trimmed" by the JSON.stringify.

JSON.stringify(function a() {}) // OUTPUT: undefined JSON.stringify(a = () => {}) // OUTPUT: undefined

---

Reference

• Set documentation at MDN
• Spread operator (the triple dots ...) documentation at MDN
  The spread operator in a = [...otherArray] creates a new array by copying elements from otherArray, ensuring a shallow copy without modifying the original.