Get list of unique files in directories

Get list of unique files in directories


    Jun 10, 2015#1

    Maybe a better way to say this is I want the opposite of "Find Duplicates". I have two folders that contain music, the structure of the folder and subfolders is different. But I know much of the music is the same. Since there are thousands of files, I just want to know what files (regardless of directory) are different on each starting folder.

    I have the same problem for pictures and documents. I don't care where the document is, just that it exists somewhere under both starting folders.

    Not sure how to do this.


    Grand MasterGrand Master

      Jun 11, 2015#2

      I think, a "find unique files" in all directories of 1 or more directory trees, i.e. a reverse Find Duplicates, can't be done with UltraCompare.

      I can offer an UltraEdit script which outputs a list of unique files with path for 1 or more directory trees. The script just compares the file names case-insensitive. It does not check for equal file size or equal file date or even equal file content. And Unicode file names are also not supported by this script.

      Copy and paste the following code into a new ASCII/ANSI file. Additionally copy and paste the code of the functions GetListOfFiles and GetNameOfFile also into the file. Please read the comments at top of function GetListOfFiles especially when not using English UltraEdit in case of a customization of variables sSummaryInfo and sResultsDocTitle is necessary for your UE configuration.

      Save the file for example with name GetUniqueFiles.js and run the script in UltraEdit with Scripting - Run Active Script.

      Note: You have to enter or copy and paste the real directory path of "My Music" and not the virtual folder name on prompt by script on execution.

      Code: Select all

      var g_nDebugMessage = 2;
      var bFilesFound = false;
      // Ask user of script for directory paths in loop until nothing entered
      // and get for each entered directory the list of file names with path.
         var sDirectory = UltraEdit.getString("Enter path of directory:",1);
         if (sDirectory.length > 0)
            if (GetListOfFiles(0,sDirectory,"*",true))
               bFilesFound = true;
      while (sDirectory.length > 0);
      if (bFilesFound)
         var sLineTerm = "\r\n";   // Default line terminator type is DOS.
         var nLineTermPos =\r\n|\n|\r/);
         if (nLineTermPos >= 0)    // Any line terminator found?
            // The list file is a Unix file if first character found is a line-feed.
            if (UltraEdit.activeDocument.selection[nLineTermPos] == '\n') sLineTerm = "\n";
            // The list file is a Mac file if first character found is a carriage
            // return and the next character is not a line-feed as in a DOS file.
            else if (UltraEdit.activeDocument.selection[nLineTermPos+1] != '\n') sLineTerm = "\r";
         var asFileNamesWithPath = UltraEdit.activeDocument.selection.split(sLineTerm);
         // The list is not needed anymore and therefore the results window is closed.
         asFileNamesWithPath.pop();  // Remove empty string at end of the list.
         // Get total number of files found in directory tree.
         var nTotalFileCount = asFileNamesWithPath.length;
         // Create a second array for the file names without path in lower case.
         var asFileNamesOnly = new Array(nTotalFileCount.length);
         var nFile = 0;
         var sFileName;
         while (nFile < nTotalFileCount)
            sFileName = GetNameOfFile(asFileNamesWithPath[nFile]);
            asFileNamesOnly[nFile] = sFileName.toLowerCase();
         // Compare each file name in array of file names with all files
         // names after current file name and remove all duplicates.
         nFile = 0;
         while (nFile < asFileNamesOnly.length)
            sFileName = asFileNamesOnly[nFile];
            var bDuplicateFound = false;
            var nNext = nFile + 1;
            while (nNext < asFileNamesOnly.length)
               if (asFileNamesOnly[nNext] != sFileName)
                  nNext++; // File names are different.
                  // Remove this duplicate file name from both arrays.
                  bDuplicateFound = true;
            // Remove the current file name also from both arrays
            // if there was at least 1 duplicate in the array.
            if (bDuplicateFound)
               nFile++  // File is unique in all directories.
         // Get number of unique files.
         var nUniqueFileCount = asFileNamesWithPath.length;
         if (nUniqueFileCount > 0)   // Are there unique files at all?
            // Output them to an ASCII/ANSI file with DOS line terminators.
            // Append an empty string to get last line also with a line termination.
            // Use clipboard to write the list into the new file as this was
            // in the past faster than using command write on a large list.
            if (typeof(UltraEdit.clipboardContent) == "string")
               UltraEdit.clipboardContent = asFileNamesWithPath.join("\r\n");
         // Inform user about number of unique files and total number of files.
         UltraEdit.messageBox("Found " + nUniqueFileCount + " unique file" + ((nUniqueFileCount!=1) ? "s" : "") +
                              " of total " + nTotalFileCount + " file" + ((nTotalFileCount!=1) ? "s." : "." ));
      Best regards from an UC/UE/UES for Windows user from Austria