preface

Array de-duplication is a common question in interviews and is often used in daily development. Here I have summarized 7 ways of array de-duplication in detail.

Example: Remove duplicate elements from the following array (for multiple data types)

const arr = [1.2.2.'abc'.'abc'.true.true.false.false.undefined.undefined.NaN.NaN]
Copy the code

1. Using the Set () + Array. The from ()

  • SetObject:The value of the collectionYou can do it in the order of insertionThe iterationIts elements. The elements in a Set will onlyonce, that is, in SetThe element is unique.
  • Array.from()How to do it: OneSimilar to an arrayoriterableCreate a new, shallow-copy array instance.
const result = Array.from(new Set(arr))
console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN ]
Copy the code

Note: The above de-duplication works for NaN and undefined as well, because both NaN and undefined can be stored in sets and NaN are treated as the same value (although in JS: NaN! = = NaN).

2. Use the splice method of the two-layer loop + array

The array elements are compared one by one through a two-level loop, and the duplicate elements are removed through the splice method. This method cannot be used to de-duplicate NaN because NaN! = = NaN.

function removeDuplicate(arr) {
  let len = arr.length
  for (let i = 0; i < len; i++) {
    for (let j = i + 1; j < len; j++) {
      if (arr[i] === arr[j]) {
        arr.splice(j, 1)
        len-- // Reduce the number of loops to improve performance
        j-- // make sure that the value of j does not change after adding}}}return arr
}

const result = removeDuplicate(arr)
console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN, NaN ]
Copy the code

3. Use the indexOf method of arrays

Create an empty array, iterate over the array that needs to be removed, save the array elements into the new array, check whether the array already contains the current element before saving, if not, save. This method also fails to de-weigh NaN.

  • indexOf()Method: Returns the index of the first occurrence of the specified value in the String calling it, fromfromIndexTo conduct a search. If the value is not found, -1 is returned.
function removeDuplicate(arr) {
  const newArr = []
  arr.forEach(item= > {
    if (newArr.indexOf(item) === -1) {
      newArr.push(item)
    }
  })
  return newArr Return a new array
}

const result = removeDuplicate(arr)
console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN, NaN ]
Copy the code

4. Use the includes method of the array

This method has the same logic as indexOf except that includes is used to determine whether duplicate elements are included.

  • includes()Method: Determines whether an array contains a specified value, depending on the case, and returns if it doestrueOtherwise returnfalse.
function removeDuplicate(arr) {
  const newArr = []
  arr.forEach(item= > {
    if(! newArr.includes(item)) { newArr.push(item) } })return newArr
}

const result = removeDuplicate(arr)
console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN ]
Copy the code

Note: The underlying implementation of Includes is involved in how NaN is detected in the array. The following is part of the code for an includes implementation, which calls the sameValueZero method to compare the inclusion of an element, or isNaN() for conversion if it isNaN.

For details, see developer.mozilla.org/zh-CN/docs/…

Simple test includes()

const testArr = [1.'a'.NaN]
console.log(testArr.includes(NaN)) // true
Copy the code

Filter ()+indexOf()

The filter method stores the elements that meet the criteria into a new array, and then uses the indexOf method to determine.

  • filter()Method: creates a new array containingTests implemented by the provided functionsAll elements of.
function removeDuplicate(arr) {
  return arr.filter((item, index) = > {
    return arr.indexOf(item) === index
  })
}

const result = removeDuplicate(arr)
console.log(result) // [ 1, 2, 'abc', true, false, undefined ]
Copy the code

Note: NaN is not included in the output because indexOf() cannot determine NaN, i.e. arr.indexof (item) === index returns false. The tests are as follows:

const testArr = [1.'a'.NaN]
console.log(testArr.indexOf(NaN)) // -1
Copy the code

6. Use the Map ()

A Map object is a data structure provided by JavaScript in the form of key-value pairs. Array elements are stored as keys of the Map, and then the combination of has() and set() methods is used to determine whether the keys are duplicated.

  • MapObject: Used to hold key-value pairs and remember the original insertion order of the keys. Any value (object or raw value) can be a key or a value.
function removeDuplicate(arr) {
  const map = new Map(a)const newArr = []

  arr.forEach(item= > {
    if(! map.has(item)) {// has() is used to determine whether map is an item attribute value
      map.set(item, true) // Use set() to set item to map and set its property value to true
      newArr.push(item)
    }
  })

  return newArr
}

const result = removeDuplicate(arr)
console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN ]
Copy the code

Note: NaN can also be de-duplicated using Map() because Map assumes NaN is equal to NaN and all other values are equal based on the result of the === operator.

7. Use objects

It is implemented in much the same way as Map(), taking advantage of the fact that object property names are not repeatable.

function removeDuplicate(arr) {
  const newArr = []
  const obj = {}

  arr.forEach(item= > {
    if(! obj[item]) { newArr.push(item) obj[item] =true}})return newArr
}

const result = removeDuplicate(arr)
console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN ]
Copy the code