Removing array duplicates in JavaScript

Reading Time: 3 minutes 🕑

Last updated: April 26, 2022.

In JavaScript, it can often be useful to remove values that appear more than once in an array. For example, this can occur when merging lists:

/* Problem 1: Duplicate primitive values */

const founders = ["Lisa", "Mark", "George"]; 
const regulars = ["Hayley", "Lisa", "Astrid", "Adam", "Mark"];

const mergedArray = [...founders, ...regulars];
console.log(mergedArray); // ["Lisa", "Mark", "George", "Hayley", "Lisa", "Astrid", "Adam", "Mark"]

We’ll also look at the more complex case of how to remove duplicates from an array of custom objects:

/* Problem 2: Duplicate object literals */

const objectsArray = [ {name: "Lisa", age: 34}, {name: "Astrid", age: 23}, {name: "Lisa", age: 34} ];

Finally, we’ll cover how to remove either type of duplicate in a mixed array.

/* Problem 3: Duplicate objects and primitives */

const mixedArray= [ {name: "Lisa", age: 34}, {name: "Astrid", age: 23}, {name: "Lisa", age: 34}, "Mark", "George", "Mark"];
Table of contents

Remove duplicate primitives

Simple solution with Set()

A syntactically succinct way of removing duplicate primitives from an array is to create a new object of type Set. Objects of this type contain unique values only, so it does all the heavy lifting for us.

All that you need to do is insert the values of the Set object into a new array. This can be done using the spread operator (...). The final result: an array of unique values.

/* Remove duplicates primitives with Set */

const myArray = ["Lisa", "Mark", "George", "Hayley", "Lisa", "Astrid", "Adam", "Mark"];

const mySet= new Set(myArray) // [ "Lisa", "Mark", "George", "Hayley", "Astrid", "Adam" ]

// Spread set into new array:
const myUniqueArray = [...mySet];
myUniqueArray // [ "Lisa", "Mark", "George", "Hayley", "Astrid", "Adam" ]

This is a nice and easy solution for duplicate primitives. But it doesn’t work for object duplicates that exist in an array:

/* Using Set does not remove object duplicates */

const objectArray = [{name:"Mark"}, {name:"Mark"}, {name:"Brian"}];

[...new Set(objectArray)]; // [ {name:"Mark"}, {name:"Mark"}, {name:"Brian"} ]

Adapting reduce()

Alternatively, reduce() can be adapted for the purpose by only including an item in the new array if it is not already included in the accumulator:

/* Remove duplicate array items that are primitive */

function removeDuplicates () {
    return array.reduce((previousValue, currentValue) => {
        if (previousValue.includes(currentValue)) {
            return previousValue; // currentValue not added to accumulator if not unique
        } else {
            return [currentValue, ...previousValue]; // currentValue added to the accumulator
        }
    },[] // starting value of the accumulator is an empty array
)}

console.log(removeDuplicates()); //  // [ "Lisa", "Mark", "George", "Hayley", "Astrid", "Adam" ]

Remove primitive and object duplicates with filter()

Remove primitive duplicates

For an array of primitives, the filter() method can be used to remove duplicates in the following way:

/* Using the filter() method to remove duplicate primitives */

const arrayOfPrimitives = ["Lisa", "Mark", "George", "Hayley", "Lisa", "Astrid", "Adam", "Mark"];

const uniqueArray = arrayOfPrimitives.filter((item, index, arr) => {
    return index === arr.findIndex((foundItem) => {
        return foundItem === item;
    })
})

console.log(uniqueArray); // [ "Lisa", "Mark", "George", "Hayley", "Astrid", "Adam" ]

The key is to understand that findIndex finds the index of the first array item it finds that matches item. If its index is the same as the index parameter (i.e. it is the first instance), the custom function passed into filter() returns true. Only this first instance will be passed into the new array filter() produces.

Remove object duplicates

The filter() method can be adapted to remove object duplicates.

To do so, check whether index matches the first occurrence of item.name. This will only be true for the first occurrence of the object. So only it will be produced in the new array:

/* Remove array duplicates that are custom objects */

const objectsArray = [ {name: "Lisa", age: 34}, {name: "Astrid", age: 23}, {name: "Lisa", age: 34} ];

const newArray = objectsArray.filter((item, index, arr) => {
    return index === arr.findIndex((foundItem) => {
        return foundItem.name === item.name;
        // 'name' should correspond to an identifying property of the objects
    })
})

newArray; // {name: "Lisa", age: 34}, {name: "Astrid", age: 23}

Remove primitive and object duplicates

The functionality of removing primitive and object duplicates can be merged. To do so, use an if…else statement to check if the array item is a primitive or object. The result of this determines whether the function check for an array or object duplicate:

/* Remove primitive and objects duplicates */

function removeDuplicates(array, prop) {
    return array.filter((item, index, arr) => {
        if (typeof item === 'object' && !Array.isArray(item) && item !== null)
        { // runs if item is an object
            return index === arr.findIndex((foundItem) => {
                return foundItem[prop] === item[prop];
            })
        } else { // runs for non-object item
            return index === arr.findIndex((foundItem) => {
                return foundItem === item;
            })
        }
    })
}

// Usage:
const mixedArray= [ {name: "Lisa", age: 34}, {name: "Astrid", age: 23}, {name: "Lisa", age: 34}, "Mark", "George", "Mark"];

removeDuplicates(mixedArray,'name'); 
// [ {name: "Lisa", age: 34}, {name: "Astrid", age: 23}, "Mark", "George"];

Summary

By creating a new Set of unique values or using the filter or reduce methods, it is possible to remove duplicate items from an array.

For removing duplicates that are custom objects or a mix of primitives and custom objects, the filter method for removing duplicate primitives can be adapted to search for duplicate property values.