Consolidate unique items in two files to a third file

dweiss · May 08, 2015#12015-05-08T15:56+00:00

Can I use UltraCompare to create a consolidated listing of not duplicate items from the lists in two existing files? Specifically, I have two large files, each containing a list of items of less than one line length each. List 1 has 580 lines and List 2 has 380 lines. What I want to do is compare the files, eliminate any duplicates, and produce a third file containing only the unique list items from the two files.

Can UC assist me in compiling the consolidated list of the unique items from each of the two lists as a third file on at least a somewhat automated basis?

Thanks, Danny

Mofi · May 09, 2015#22015-05-09T16:21+00:00

It is possible to

compare two text files with UltraCompare,
click in toolbar on the unequal symbol to see just the differences,
click into pane of file 1 and press Ctrl+A to select all displayed lines only (the unique lines in first file),
copy the selected lines with Ctrl+C to clipboard and paste them into a new file in a text editor,
click into pane of file 2 and press Ctrl+A to select all displayed lines only (the unique lines in second file),
copy the selected lines with Ctrl+C to clipboard and paste them into a new file in a text editor.

But for a completely automated file comparison with getting all unique lines of both files into a new file, it is better to use UltraEdit/UEStudio with a macro or script for this task. See Odd comparison results for short lines with very similar data containing links to several other topics with macros and scripts for this or similar tasks.

The first link listed there is to Compare two files line by line and get not matching lines into two new files.

As you want all unique lines just in a single new file, here is first script from referenced topic with the small modifications necessary for a single file result.

Open in UltraEdit the two files. Create a new ASCII file with DOS line terminators and copy & paste the following script code into this new file. Save the new file with a name like GetUniqueLinesFromTwoFiles.js. Run the script via Scripting - Run Active Script.

Code: Select all

if (UltraEdit.document.length > 1)  // Are at least 2 files opened?
{
   // Define environment for the script.
   UltraEdit.insertMode();
   if (typeof(UltraEdit.columnModeOff) == "function") UltraEdit.columnModeOff();
   else if (typeof(UltraEdit.activeDocument.columnModeOff) == "function") UltraEdit.activeDocument.columnModeOff();

   // Get all lines from first (most left) file.
   UltraEdit.document[0].selectAll();
   var asLinesFileA = UltraEdit.document[0].selection.split("\r\n");
   // Remove last element from the array if it is an empty string
   // because the file ended with a DOS line termination.
   if (asLinesFileA[asLinesFileA.length-1] == "") asLinesFileA.pop();

   // Get all lines from second file.
   UltraEdit.document[1].selectAll();
   var asLinesFileB = UltraEdit.document[1].selection.split("\r\n");
   if (asLinesFileB[asLinesFileB.length-1] == "") asLinesFileB.pop();

   // Remove the lines existing in both files.
   for (var nLineA = 0; nLineA < asLinesFileA.length; nLineA++)
   {
      var sLineA = asLinesFileA[nLineA];
      for (var nLineB = 0; nLineB < asLinesFileB.length; nLineB++)
      {
         if (asLinesFileB[nLineB] == sLineA)
         {
            asLinesFileB.splice(nLineB,1);
            asLinesFileA.splice(nLineA,1);
            nLineA--;
            break;
         }
      }
   }
   UltraEdit.selectClipboard(9);
   // Are unique lines from file A remaining, write them to a new file.
   var bNewFileCreated = false;
   if (asLinesFileA.length > 0)
   {
      UltraEdit.newFile();
      UltraEdit.clipboardContent = asLinesFileA.join("\r\n") + "\r\n";
      UltraEdit.activeDocument.paste();
      bNewFileCreated = true;
   }
   // Are unique lines from file B remaining, write them to same new file.
   if (asLinesFileB.length > 0)
   {
      if(!bNewFileCreated) UltraEdit.newFile();
      UltraEdit.clipboardContent = asLinesFileB.join("\r\n") + "\r\n";
      UltraEdit.activeDocument.paste();
   }
   if(bNewFileCreated) UltraEdit.activeDocument.top();
   UltraEdit.clearClipboard();
   UltraEdit.selectClipboard(0);
}

dweiss · May 10, 2015#32015-05-10T03:36+00:00

Mofi, thank you very much. That did the trick. I appreciate it very much.