If I have you understand correct you want the data in your file split up into 2 other files. The first one should contain the lines with the first occurrence of a string left the equal sign and the second one should contain all other lines having the same string left the equal sign already found once.
For this task you need 2 macros. The macros are based on the macro I posted at
How do I remove duplicate lines? The macro property
Continue if search string not found must be checked for both macros. The macro property for the Cancel dialog should be unchecked for both macros.
The macros are designed to work on lines with DOS line endings because of using
^p in the search strings. If your file is a Unix file opened without conversion to DOS, you have to replace all
^p by
^n in the macro sources to get the macros correct working.
The first macro uses command
SelectLine which is faster, but requires that the source file is opened without any word wrap enabled.
It is important that your huge source file is opened with usage of a temporary file because the macro modifies this source file, but closes it without saving the changes (and reopens it). This works only if the modifications are not permanent which requires that the source file is opened with usage of a temporary file.
First click on
Macro - Edit Macro, button
New Macro and enter as macro name
FindDuplicates. The macro name for the first macro is important including the case of the letters. After setting the properties as written above, click OK and replace the existing lines with following macro code:
Loop
Clipboard 9
Find MatchCase "^c"
IfFound
SelectLine
Clipboard 8
CutAppend
Else
ExitLoop
EndIf
EndLoop
Next click again on button
New Macro and confirm that you want to save the modifications of the just created macro. For the second macro the name does not matter, use for example
SplitDupsFile. After setting the properties as written above, click OK and replace the existing lines with following macro code:
InsertMode
ColumnModeOff
HexOff
UnixReOff
Bottom
IfColNum 1
Else
"
"
EndIf
Top
Find RegExp "%^([~^p]^)"
Replace All "#MOFI_RULES#^1"
Clipboard 7
ClearClipboard
Clipboard 8
ClearClipboard
Clipboard 9
Loop
Find MatchCase RegExp "%#MOFI_RULES#*="
IfNotFound
ExitLoop
EndIf
Copy
Clipboard 7
CutAppend
Find RegExp "?++^p"
CutAppend
PlayMacro 1 "
FindDuplicates"
Top
EndLoop
CopyFilePath
CloseFile NoSave
Open "^c"
ClearClipboard
NewFile
Clipboard 7
Paste
ClearClipboard
Top
Find MatchCase RegExp "%#MOFI_RULES#"
Replace All ""
NewFile
Clipboard 8
Paste
ClearClipboard
Top
Find MatchCase RegExp "%#MOFI_RULES#"
Replace All ""
IfNotFound
"NO DUPLICATES :-)
"
EndIf
Clipboard 0
After closing the edit macro dialog with button
Close and confirming to save the modifications on the just created second macro, open your file if not already open and run once the just created second macro. It will take very long on your huge file, but as a result the source file is closed without saving the changes and reopened and you should get 2 new files. The first (left) one contains the first occurrence of a line with a unique string left the equal sign and the second (right) one the lines with the duplicates.
For example the source file contains:
ravi=rvI
ravi=roI
test1=test
test2=1
test2=3
test2=2
The first new file contains after macro execution:
ravi=rvI
test1=test
test2=1
The second new file contains after macro execution:
ravi=roI
test2=3
test2=2