Using UltraEdit to merge in iTunes Library data?

crsouser · Sep 15, 2011#12011-09-15T01:41+00:00

I recently did a clean install of my computer OS and I had everything backed up.
Though when I re-setup my computer and imported my iTunes library from backups ... everything, but the ratings and play count basically got imported.

I would like to import my rating back into new library by comparing with UltraCompare my two iTunes Library.xml files and do the comparison. But how to I get them to selectively sync the XML data.

Is there a macro or something that I could do to perform this function easily? I read about people wanting to do this all over the place, but haven't found any good solution that doesn't involve pre-creating playlists by rating.

Can anyone help?

Possible Keys: Name, Artist, Album, Size, Location

Want to bring over into new library and in most cases these fields do not exist in the new library: Play Count, Rating, Album Rating, Genre, Comments

So here are two examples of the exact same file entry in iTunes format:
The New Library:

Code: Select all

			<key>63794</key>
		<dict>
			<key>Track ID</key><integer>63794</integer>
			<key>Name</key><string>Addicted</string>
			<key>Artist</key><string>Saving Abel</string>
			<key>Album Artist</key><string>Saving Abel</string>
			<key>Album</key><string>Saving Abel</string>
			<key>Genre</key><string>Rock</string>
			<key>Kind</key><string>MPEG audio file</string>
			<key>Size</key><integer>7777675</integer>
			<key>Total Time</key><integer>222876</integer>
			<key>Disc Number</key><integer>1</integer>
			<key>Disc Count</key><integer>1</integer>
			<key>Track Number</key><integer>2</integer>
			<key>Track Count</key><integer>11</integer>
			<key>Year</key><integer>2008</integer>
			<key>Date Modified</key><date>2011-07-05T07:21:41Z</date>
			<key>Date Added</key><date>2011-09-05T14:20:41Z</date>
			<key>Bit Rate</key><integer>276</integer>
			<key>Sample Rate</key><integer>44100</integer>
			<key>Comments</key><string>Amazon.com Song ID: 204450999 __SKIPPED__</string>
			<key>Artwork Count</key><integer>1</integer>
			<key>Persistent ID</key><string>3F431ADD2FBB26B1</string>
			<key>Track Type</key><string>File</string>
			<key>Location</key><string>file://localhost/Volumes/CRS_RAID1/CRSouser_Music_Itunes/Music/Saving%20Abel/Saving%20Abel/02%20Addicted.mp3</string>
		</dict>

My Old Library Entry with Ratings and other information:

Code: Select all

<key>55376</key>
		<dict>
			<key>Track ID</key><integer>55376</integer>
			<key>Name</key><string>Addicted</string>
			<key>Artist</key><string>Saving Abel</string>
			<key>Album Artist</key><string>Saving Abel</string>
			<key>Album</key><string>Saving Abel</string>
			<key>Genre</key><string>Rock</string>
			<key>Kind</key><string>MPEG audio file</string>
			<key>Size</key><integer>7777675</integer>
			<key>Total Time</key><integer>222876</integer>
			<key>Disc Number</key><integer>1</integer>
			<key>Disc Count</key><integer>1</integer>
			<key>Track Number</key><integer>2</integer>
			<key>Track Count</key><integer>11</integer>
			<key>Year</key><integer>2008</integer>
			<key>Date Modified</key><date>2011-07-05T07:21:41Z</date>
			<key>Date Added</key><date>2008-11-29T20:41:20Z</date>
			<key>Bit Rate</key><integer>276</integer>
			<key>Sample Rate</key><integer>44100</integer>
			<key>Comments</key><string>Amazon.com Song ID: 204450999 __SKIPPED__</string>
			<key>Play Count</key><integer>29</integer>
			<key>Play Date</key><integer>3395146936</integer>
			<key>Play Date UTC</key><date>2011-08-03T00:22:16Z</date>
			<key>Skip Count</key><integer>1</integer>
			<key>Skip Date</key><date>2009-04-19T03:14:53Z</date>
			<key>Rating</key><integer>100</integer>
			<key>Album Rating</key><integer>100</integer>
			<key>Album Rating Computed</key><true/>
			<key>Artwork Count</key><integer>1</integer>
			<key>Persistent ID</key><string>F6DF3126A53F5B3E</string>
			<key>Track Type</key><string>File</string>
			<key>Location</key><string>file://localhost/Volumes/CRS_RAID1/CRSouser_Music_Itunes/Music/Saving%20Abel/Saving%20Abel/02%20Addicted.mp3</string>
			<key>File Folder Count</key><integer>5</integer>
			<key>Library Folder Count</key><integer>1</integer>
		</dict>

Mofi · Sep 15, 2011#22011-09-15T05:35+00:00

First, why not simply restoring the library from the backup?

Second, it looks like the key values have changed which makes it not easy to compare the two XML files in UltraCompare and simply merge (copy) all missing lines from file A (old) to file B (new).

Perhaps even the order of the blocks is different. In this case UltraCompare can't be used. It is necessary to code a macro or better a script executed with UltraEdit to copy the data from old to new XML file. Coding such a script is not really difficult for an advanced script writter, but some information is needed. Which tags should be used to identify that block X from old XML file is an equivalent of block Y from new XML file? The Location and/or Artist and/or Album, ...? In other words what is surely 100% (case sensitive string compare) identical in both XML files in all blocks and can be used to build a relation between a block on old and new XML file.

For coding the script it would be necessary to have 2 example XML files containing also the header and footer. Could you create such examples files by copying your 2 files and deleting most lines so that just 5 blocks remain in both files and only 3 of them are for the same songs, but have different key values? With such examples files it would be possible to code a script and report also which blocks from old XML file could not be found in new XML file and also which blocks in new XML file where not updated. The XML files must be packed with ZIP or RAR and the archive file can be uploaded as attachment. Uploading the XML files directly is not possible.

crsouser · Sep 15, 2011#32011-09-15T14:52+00:00

Mofi wrote:First, why not simply restoring the library from the backup?

I did actually restore the old library and with the way iTunes works it actually purges all MetaData when you try putting the old files in place, it forces you to do an import of the old library, but doesn't import the ratings or plays counts.

Mofi wrote:Second, it looks like the key values have changed which makes it not easy to compare the two XML files in UltraCompare and simply merge (copy) all missing lines from file A (old) to file B (new)."

Yep.. the key value is unique per iTunes installation or library.

Mofi wrote:Which tags should be used to identify that block X from old XML file is an equivalent of block Y from new XML file? The Location and/or Artist and/or Album, ...?

Looking at several samples the following keys seem to be consistently the same:
Name, Artist, Album, Size, Location

Having size and location, the keys would also handle duplicate files in a library.

Took me a few minutes to put together to create example libraries, without any playlists.

For reference and checking, as I understand it the iTunes rating system is
00 = Unrated
10 = Half Star
20 = 1 Star
30 = 1.5 Stars
40 = 2 Stars
50 = 2.5 Stars
60 = 3 Stars
70 = 3.5 Stars
80 = 4 Stars
90 = 4.5 Stars
100 = 5 Stars

5 Songs in Library without any playlists:
Addicted = 5 Stars (Value 100)
Television = 4 Stars(Value 80)
Touch It = 4 Stars (value 80)
Fable = 3 Stars (Value 60)
Superheros = 2 Stars (Value 40)

Thanks a ton for any help you can provide.

Christopher

Mofi · Sep 17, 2011#42011-09-17T15:42+00:00

It took me some hours to write the script, but here it is. Please read the comments at top of the file for the instruction how to use it and what it does. It worked on your example library files.

crsouser · Sep 17, 2011#52011-09-17T20:27+00:00

Thanks!

I tried it on the UE Mac version and something just isn't quite right as far as compatibility or the way I execute it as the menus are quite different. So I fired up UE 17 in a VMware session and your instructions were pretty clear cut and it seems to be running.

My iTunes library is about 33,000 entries so its taking a while, its on hour 4 right now and the status window hasn't updated and the UE state is an a "Non-Responding state" using about 60% of the CPU. I am going to let it sit for up to a day and see if it finishes.

Thank you, I'll keep you in the loop.

30 minutes later:

After doing some poking around I found that the UE Process wasn't using any of the CPU so I aborted it.

Tried running the script on UE Mac and I get the following error.

Running script: /Users/xxxx/Desktop/ItunesFix/iTunesMetaTagCopy.js
======================================================================
An error occurred on line : 113
Script failed.

Which is...just beyond the IF statement to make sure two files are open.

112 UltraEdit.insertMode();
113 UltraEdit.columnModeOff();
114 var bError = false;

Ideas?

Christopher

Mofi · Sep 18, 2011#62011-09-18T15:24+00:00

Do you have first tested the script on your 2 example files before you tried it on real library files?

I have written the script to show process information in the output window while it runs. So you should see during execution in the automatically opened output window:

Code: Select all

Reading library data to update from file:  "[path]\ITunes Library Sample No Ratings.xml"
Reading source library data from file:     "[path]\ITunes Library Sample with Ratings.xml"
Initializing track data updating process ...
Processing the track data update ...

50 track data from second file processed.
100 track data from second file processed.
150 track data from second file processed.

and so on up to

Code: Select all

All 33000 track data from second file processed.

I have used UltraEdit for Windows v17.20 to develop and test the script. I estimate that the script needs for 33000 entries just 30 seconds to 3 minutes. The script uses nearly no UltraEdit commands, just JavaScript core commands. I tested it also successfully with UE v14.10. On previous versions it does not work.

I don't have UEM and therefore don't know which script commands are supported by UltraEdit for Mac. If function UltraEdit.columnModeOff(); is an unknown function for UEM, comment this line with //. It is not an important command. It just turns off column editing mode in case column mode is enabled on script start. Or even better replace that command by:

Code: Select all

if (typeof(UltraEdit.columnModeOff) == "function") UltraEdit.columnModeOff();
else if (typeof(UltraEdit.activeDocument.columnModeOff) == "function") UltraEdit.activeDocument.columnModeOff();

crsouser · Sep 18, 2011#72011-09-18T22:47+00:00

The script as far as what it is displaying to me initially until I hit cancel never gets beyond.

Code: Select all

Reading library data to update from file:  "\\vmware-host\Shared Folders\My Desktop\ItunesFix\iTunes Music Library.xml"

Though if I hit Cancel within a few minutes, the output windows will report output of somewhere between 8000 and 12000 files processed. Thought hitting Cancel the destination file isn't reflecting as updated, so not sure I can run it for only a few minutes each time until I get through the entire library.

It works flawless it seems with the sample.

In UE Mac the script errors on line 113 even if I comment out lines 112 and 113. So that seems a bit suspicious to me as far as a GAP in UltraEdit feature sets between the different versions.

I'll keep playing some more and see if I can get lucky and see it complete.

Christopher

Mofi · Sep 19, 2011#82011-09-19T05:17+00:00

Well, it is the first time that I have written a script to display progress information during script execution in the output window for a script which makes everything in memory. So I don't know when the display of the output window is refreshed. Perhaps the output window display is refreshed only when something is written to a document window which happens on this script only at end. In this case the progress information printing to output window would be useless and just slows down the script.

As far as I know UEM v2.2.0.9 has yet not all the features UE v17.20.0.1016 has. So it is possible that the UltraEdit scripting commands set is also reduced at the moment. But the script uses mainly core functions and just a few basic UE scripting commands and therefore it should work also on UEM. But I can't verify my assumption because I don't have a Mac and UEM.

Sep 19, 2011#92011-09-19T17:25+00:00

Perhaps it is useful when the script is not executed on all 33.000 entries to find out if it is working on real library files or not. Therefore here is another version of the script where 3 not really needed lines are commented out and 2 lines are inserted to limit the number of track blocks to analyze to 100 for both files. So the script now reads in all 33.000 track blocks from both files, but processes only the first 100 track blocks from second file on the first 100 track blocks from first file.

This should produce much quicker a result which can be analyzed by you. Without having the real library files I can only hope that they look like the 2 examples files you provided. Otherwise I would not wonder if the script searches each of the 33.000 track blocks from second file all the time on all 33.000 track blocks of first file which would need hours. A small case sensitive difference for example in location string would already result in not finding any track information from second file in any of the track blocks in first file.