JavaScript: How to Filter Duplicate Items in an Array

Hi!

The most straightforward method for that is the JavaScript Set method.

Remember that maths subject about Venn diagram?

In it, the set cannot have duplicates.

For example, the word "BOB".

The set from the word will be {B, O}, and not {B, O, B}.

Similar concept in JavaScript Set.

This Set direct filtering works only for string and number items from an array.

---

Input Array

For example, we have these arrays:

const myNumber = [0, 2, 3, 4, 5, 2, 0, 4] const myString = ["a", "b", "c", "dee", "dee", "doo", "bee"] const myMix = [2, "b", 2, "dee", "dee", 0, "b"]

---

Filtering Duplicates Using Set Method

[...new Set(inputArray)]

The new Array from that will consist of unique items.

---

Helper Function

Let's put that method as a function:

function removeDuplicates(inputArray) { return [...new Set(inputArray)] }

Or, using arrow function syntax:

const removeDuplicates = (inputArray) => [...new Set(inputArray)]

---

Usage

const myUniqueNumber = removeDuplicates(myNumber) // INPUT myNumber = [0, 2, 3, 4, 5, 2, 0, 4] // OUTPUT myUniqueNumber = [0, 2, 3, 4, 5] const myUniqueString = removeDuplicates(myString) // INPUT myString = ["a", "b", "c", "dee", "dee", "doo", "bee"] // OUTPUT myUniqueString = ["a", "b", "c", "dee", "doo", "bee"] const myUniqueMix = removeDuplicates(myMix) // INPUT myMix = [2, "b", 2, "dee", "dee", 0, "b"] // OUTPUT myUniqueMix = [2, "b", "dee", 0]

---

Object (or Array) Item Type

We can use JSON.stringify to convert each item into a valid JSON string. Afterwards, use JSON.parse to revert each back.

const removeDups = (inputArray) => { try { const strings = inputArray.map((item) => JSON.stringify(item)) const filtered = [...new Set(strings)] const output = filtered.map((item) => JSON.parse(item)) return output } catch (error) { console.error(error) return inputArray } }

Let's try that:

removeDups([{a: 1}, {a: 1}, {b: 2}, [0, 2, 3], [0, 2, 3], "a", "a"]) // OUTPUT:  [{"a": 1}, {"b": 2}, [0, 2, 3], "a"]

Please be careful if the object is MASSIVE when using the method or function above.

And keep in mind, JSON.stringify for this kind of input —

{ theKey: undefined }

— will completely omit theKey.

JSON.stringify({ theKey: undefined }) // OUTPUT: '{}' JSON.parse(JSON.stringify(undefined)) // SyntaxError: "undefined" is not valid JSON

And of course, if there's Function in the array, it will also be trimmed by the JSON.stringify.

JSON.stringify(function a() {}) // OUTPUT: undefined JSON.stringify(a = () => {}) // OUTPUT: undefined

---

👋