How can I Copy/append all unique rows from SOURCE Spreadsheet to DESTINATION Spreadsheet.

• Spreadsheet #1 (SOURCE) - Contains ID's and Names. This sheet has duplicate rows. 500k rows (Access: View Only)
• Spreadsheet #2 (DESTINATION) - Contains only unique ID's with Names. (Access: Edit)

This script works, but it copies all (including duplicates).

function transferIDs() {
  var sss = SpreadsheetApp.openById('%'); //SOURCE
  var ss = sss.getSheetByName('Sheet1');
  var SRange = ss.getDataRange();
  var A1Range = SRange.getA1Notation();
  var SData = SRange.getValues();

  var dss = SpreadsheetApp.openById('#'); //DESTINATION
  var ds = dss.getSheetByName('Sheet1');
  ds.clear({contentsOnly: true});
  ds.getRange(A1Range).setValues(SData);
}

Spreadsheet #1 SOURCE (contains duplicate rows)

Spreadsheet #2 DESTINATION (Populated using GAS, no duplicates, Expected Outcome)

CodePudding user response：

Use filter with set:

Change SData to

const SData = SRange.getValues().filter(
  (set => row => set.has(row[0]) ? false : set.add(row[0]))(new Set)
);

And modify the destination range:

ds.getRange(1,1,SData.length,SData[0].length).setValues(SData);