Fading Coder

One Final Commit for the Last Sprint

Home > Tech > Content

Techniques for Removing Duplicate Objects from JavaScript Arrays

Tech 3

Consider an array of objects where duplicates need to be eliminated based on a specific property:

const dataSet = [
  { id: '01', name: 'Lele' },
  { id: '02', name: 'Bobo' },
  { id: '03', name: 'Taotao' },
  { id: '04', name: 'Haha' },
  { id: '01', name: 'Lele' }
];

Method 1: Using an Object Lookup

This approach employs an object as a lookup table to track encountered keys.

function removeDuplicatesByProperty(arr, property) {
  const uniqueItems = [];
  const seenKeys = {};
  
  for (let i = 0; i < arr.length; i++) {
    const currentKey = arr[i][property];
    if (!seenKeys[currentKey]) {
      uniqueItems.push(arr[i]);
      seenKeys[currentKey] = true;
    }
  }
  
  return uniqueItems;
}

const deduped = removeDuplicatesByProperty(dataSet, 'id');
console.log(deduped);
// Output: [{id: "01", name: "Lele"}, {id: "02", name: "Bobo"}, {id: "03", name: "Taotao"}, {id: "04", name: "Haha"}]

Method 2: Utilizing the Reduce Method

The reduce method accumulates unique items while checking against a tracking object.

function deduplicateWithReduce(array, keyField) {
  const keyTracker = {};
  
  return array.reduce((accumulator, currentElement) => {
    const identifier = currentElement[keyField];
    if (!keyTracker[identifier]) {
      keyTracker[identifier] = true;
      accumulator.push(currentElement);
    }
    return accumulator;
  }, []);
}

const result = deduplicateWithReduce(dataSet, 'id');
console.log(result);
// Output: [{id: "01", name: "Lele"}, {id: "02", name: "Bobo"}, {id: "03", name: "Taotao"}, {id: "04", name: "Haha"}]

Method 3: Comprehensive Object Comparison

For scenarios requiring deduplication based on entire object content rather than a single property, this method serializes objects for cmoparison.

function eliminateObjectDuplicates(objectArray) {
  const uniqueObjects = [];
  const serializedSet = {};
  
  for (let i = 0; i < objectArray.length; i++) {
    const currentObj = objectArray[i];
    const propertyNames = Object.keys(currentObj).sort();
    let serializedString = '';
    
    for (let j = 0; j < propertyNames.length; j++) {
      const propName = propertyNames[j];
      serializedString += JSON.stringify(propName) + JSON.stringify(currentObj[propName]);
    }
    
    if (!serializedSet.hasOwnProperty(serializedString)) {
      uniqueObjects.push(currentObj);
      serializedSet[serializedString] = true;
    }
  }
  
  return uniqueObjects;
}

const complexData = [
  { id: '01', value: 'Test', tag: 'A' },
  { tag: 'A', value: 'Test', id: '01' },
  { id: '02', value: 'Demo' }
];

const filtered = eliminateObjectDuplicates(complexData);
console.log(filtered);
// Output: [{id: "01", value: "Test", tag: "A"}, {id: "02", value: "Demo"}]

Related Articles

Understanding Strong and Weak References in Java

Strong References Strong reference are the most prevalent type of object referencing in Java. When an object has a strong reference pointing to it, the garbage collector will not reclaim its memory. F...

Comprehensive Guide to SSTI Explained with Payload Bypass Techniques

Introduction Server-Side Template Injection (SSTI) is a vulnerability in web applications where user input is improper handled within the template engine and executed on the server. This exploit can r...

Implement Image Upload Functionality for Django Integrated TinyMCE Editor

Django’s Admin panel is highly user-friendly, and pairing it with TinyMCE, an effective rich text editor, simplifies content management significantly. Combining the two is particular useful for bloggi...

Leave a Comment

Anonymous

◎Feel free to join the discussion and share your thoughts.