How to Merge Two Arrays in Javascript and De-Duplicate Items
I Have Two Javascript Arrays: Var Array1 = ["Vijendra","Singh"]; Var Array2 = ["Singh", "Shakya"]; I Want the Output to Be: Var Array3 =...
I have two JavaScript arrays:
var array1 = ["Vijendra","Singh"];
var array2 = ["Singh", "Shakya"];
I want the output to be:
var array3 = ["Vijendra","Singh","Shakya"];
The output array should have repeated words removed.
How do I merge two arrays in JavaScript so that I get only the unique items from each array in the same order they were inserted into the original arrays?
87 Answers
To just merge the arrays (without removing duplicates)
ES5 version use Array.concat:
var array1 = ["Vijendra", "Singh"];
var array2 = ["Singh", "Shakya"];
console.log(array1.concat(array2));
ES6 version use destructuring
const array1 = ["Vijendra","Singh"];
const array2 = ["Singh", "Shakya"];
const array3 = [...array1, ...array2];
Since there is no 'built in' way to remove duplicates (ECMA-262 actually has Array.forEach which would be great for this), we have to do it manually:
Array.prototype.unique = function() {
var a = this.concat();
for(var i=0; i<a.length; ++i) {
for(var j=i+1; j<a.length; ++j) {
if(a[i] === a[j])
a.splice(j--, 1);
}
}
return a;
};
Then, to use it:
var array1 = ["Vijendra","Singh"];
var array2 = ["Singh", "Shakya"];
// Merges both arrays and gets unique items
var array3 = array1.concat(array2).unique();
This will also preserve the order of the arrays (i.e, no sorting needed).
Since many people are annoyed about prototype augmentation of Array.prototype and for in loops, here is a less invasive way to use it:
function arrayUnique(array) {
var a = array.concat();
for(var i=0; i<a.length; ++i) {
for(var j=i+1; j<a.length; ++j) {
if(a[i] === a[j])
a.splice(j--, 1);
}
}
return a;
}
var array1 = ["Vijendra","Singh"];
var array2 = ["Singh", "Shakya"];
// Merges both arrays and gets unique items
var array3 = arrayUnique(array1.concat(array2));
For those who are fortunate enough to work with browsers where ES5 is available, you can use Object.defineProperty like this:
Object.defineProperty(Array.prototype, 'unique', {
enumerable: false,
configurable: false,
writable: false,
value: function() {
var a = this.concat();
for(var i=0; i<a.length; ++i) {
for(var j=i+1; j<a.length; ++j) {
if(a[i] === a[j])
a.splice(j--, 1);
}
}
return a;
}
});
With Underscore.js or Lo-Dash you can do:
console.log(_.union([1, 2, 3], [101, 2, 1, 10], [2, 1]));
<script src=""></script>
First concatenate the two arrays, next filter out only the unique items:
var a = [1, 2, 3], b = [101, 2, 1, 10]
var c = a.concat(b)
var d = c.filter((item, pos) => c.indexOf(item) === pos)
console.log(d) // d is [1, 2, 3, 101, 10]
Edit
As suggested a more performance wise solution would be to filter out the unique items in b before concatenating with a:
var a = [1, 2, 3], b = [101, 2, 1, 10]
var c = a.concat(b.filter((item) => a.indexOf(item) < 0))
console.log(c) // c is [1, 2, 3, 101, 10]
[...array1,...array2] // => don't remove duplication
OR
[...new Set([...array1 ,...array2])]; // => remove duplication
This is an ECMAScript 6 solution using spread operator and array generics.
Currently it only works with Firefox, and possibly Internet Explorer Technical Preview.
But if you use Babel, you can have it now.
const input = [
[1, 2, 3],
[101, 2, 1, 10],
[2, 1]
];
const mergeDedupe = (arr) => {
return [...new Set([].concat(...arr))];
}
console.log('output', mergeDedupe(input));
Using a Set (ECMAScript 2015), it will be as simple as that:
const array1 = ["Vijendra", "Singh"];
const array2 = ["Singh", "Shakya"];
console.log(Array.from(new Set(array1.concat(array2))));
You can do it simply with ECMAScript 6,
var array1 = ["Vijendra", "Singh"];
var array2 = ["Singh", "Shakya"];
var array3 = [...new Set([...array1 ,...array2])];
console.log(array3); // ["Vijendra", "Singh", "Shakya"];
- Use the spread operator for concatenating the array.
- Use Set for creating a distinct set of elements.
- Again use the spread operator to convert the Set into an array.
Here is a slightly different take on the loop. With some of the optimizations in the latest version of Chrome, it is the fastest method for resolving the union of the two arrays (Chrome 38.0.2111).
var array1 = ["Vijendra", "Singh"];
var array2 = ["Singh", "Shakya"];
var array3 = [];
var arr = array1.concat(array2),
len = arr.length;
while (len--) {
var itm = arr[len];
if (array3.indexOf(itm) === -1) {
array3.unshift(itm);
}
}
while loop: ~589k ops/s
filter: ~445k ops/s
lodash: 308k ops/s
for loops: 225k ops/s
A comment pointed out that one of my setup variables was causing my loop to pull ahead of the rest, because it didn't have to initialize an empty array to write to. I agree with that, so I've rewritten the test to even the playing field, and included an even faster option.
let whileLoopAlt = function (array1, array2) {
const array3 = array1.slice(0);
let len1 = array1.length;
let len2 = array2.length;
const assoc = {};
while (len1--) {
assoc[array1[len1]] = null;
}
while (len2--) {
let itm = array2[len2];
if (assoc[itm] === undefined) { // Eliminate the indexOf call
array3.push(itm);
assoc[itm] = null;
}
}
return array3;
};
In this alternate solution, I've combined one answer's associative array solution to eliminate the .indexOf() call in the loop which was slowing things down a lot with a second loop, and included some of the other optimizations that other users have suggested in their answers as well.
The top answer here with the double loop on every value (i-1) is still significantly slower. lodash is still doing strong, and I still would recommend it to anyone who doesn't mind adding a library to their project. For those who don't want to, my while loop is still a good answer and the filter answer has a very strong showing here, beating out all on my tests with the latest Canary Chrome (44.0.2360) as of this writing.
Check out Mike's answer and Dan Stocker's answer if you want to step it up a notch in speed. Those are by far the fastest of all results after going through almost all of the viable answers.
The ES6 offers a single-line solution for merging multiple arrays without duplicates by using destructuring and set.
const array1 = ['a','b','c'];
const array2 = ['c','c','d','e'];
const array3 = [...new Set([...array1,...array2])];
console.log(array3); // ["a", "b", "c", "d", "e"]
I simplified the best of this answer and turned it into a nice function:
function mergeUnique(arr1, arr2){
return arr1.concat(arr2.filter(function (item) {
return arr1.indexOf(item) === -1;
}));
}
Just steer clear of nested loops (O(n^2)), and .indexOf() (+O(n)).
function merge(a, b) {
var hash = {};
var i;
for (i = 0; i < a.length; i++) {
hash[a[i]] = true;
}
for (i = 0; i < b.length; i++) {
hash[b[i]] = true;
}
return Object.keys(hash);
}
var array1 = ["Vijendra", "Singh"];
var array2 = ["Singh", "Shakya"];
var array3 = merge(array1, array2);
console.log(array3);
Just throwing in my two cents.
function mergeStringArrays(a, b){
var hash = {};
var ret = [];
for(var i=0; i < a.length; i++){
var e = a[i];
if (!hash[e]){
hash[e] = true;
ret.push(e);
}
}
for(var i=0; i < b.length; i++){
var e = b[i];
if (!hash[e]){
hash[e] = true;
ret.push(e);
}
}
return ret;
}
This is a method I use a lot, it uses an object as a hashlookup table to do the duplicate checking. Assuming that the hash is O(1), then this runs in O(n) where n is a.length + b.length. I honestly have no idea how the browser does the hash, but it performs well on many thousands of data points.
Array.prototype.merge = function(/* variable number of arrays */){
for(var i = 0; i < arguments.length; i++){
var array = arguments[i];
for(var j = 0; j < array.length; j++){
if(this.indexOf(array[j]) === -1) {
this.push(array[j]);
}
}
}
return this;
};
A much better array merge function.
EDIT:
The first solution is the fastest only when there is few items. When there is over 400 items, the Set solution becomes the fastest. And when there is 100,000 items, it is a thousand times faster than the first solution.
Considering that performance is important only when there is a lot of items, and that the Set solution is by far the most readable, it should be the right solution in most cases
The perf results below were computed with a small number of items
Based on jsperf, the fastest way (edit: if there is less than 400 items) to merge two arrays in a new one is the following:
for (var i = 0; i < array2.length; i++)
if (array1.indexOf(array2[i]) === -1)
array1.push(array2[i]);
This one is 17% slower:
array2.forEach(v => array1.includes(v) ? null : array1.push(v));
This one is 45% slower (edit: when there is less than 100 items. It is a lot faster when there is a lot of items):
var a = [...new Set([...array1 ,...array2])];
And the accepted answers is 55% slower (and much longer to write) (edit: and it is several order of magnitude slower than any of the other methods when there is 100 000 items)
var a = array1.concat(array2);
for (var i = 0; i < a.length; ++i) {
for (var j = i + 1; j < a.length; ++j) {
if (a[i] === a[j])
a.splice(j--, 1);
}
}
I know this question is not about array of objects, but searchers do end up here.
so it's worth adding for future readers a proper ES6 way of merging and then removing duplicates
array of objects:
var arr1 = [ {a: 1}, {a: 2}, {a: 3} ];
var arr2 = [ {a: 1}, {a: 2}, {a: 4} ];
var arr3 = arr1.concat(arr2.filter( ({a}) => !arr1.find(f => f.a == a) ));
// [ {a: 1}, {a: 2}, {a: 3}, {a: 4} ]
Why don't you use an object? It looks like you're trying to model a set. This won't preserve the order, however.
var set1 = {"Vijendra":true, "Singh":true}
var set2 = {"Singh":true, "Shakya":true}
// Merge second object into first
function merge(set1, set2){
for (var key in set2){
if (set2.hasOwnProperty(key))
set1[key] = set2[key]
}
return set1
}
merge(set1, set2)
// Create set from array
function setify(array){
var result = {}
for (var item in array){
if (array.hasOwnProperty(item))
result[array[item]] = true
}
return result
}
Must Read
Performance
Today 2020.10.15 I perform tests on MacOs HighSierra 10.13.6 on Chrome v86, Safari v13.1.2 and Firefox v81 for chosen solutions.