Search and Replace with substitution and increment

lovebit · Apr 18, 2007#12007-04-18T18:13+00:00

I have a text file so:

F 93246 allapviep1n200011900a 1 APVI
I 0 Date 10/04/2000
I 1 Time 20:04:00
I 2 Station pcde24
I 3 User admin
I 4 IPath images/allapviep1n200011900a
I 5 Prio 5
D 828 allapviep1n200011900a_828 2
A I num_polizza=020468324
A I sig_cogn=XAS
A I cod_tipo_doc=199
A I data_doc=17/04/2007
A I protocollo_doc=IAAL0000000007400191
#Allnet I protdc=IAAL0000000007400191
#Allnet I coddoc=199
#Allnet I numpol=020400324
#Allnet I sigcog=XAS
#Allnet I statosal=B
#Allnet I malatt=N
#Allnet I ricove=N
#Allnet I vispec=N
#Allnet I fircon=S
#Allnet I firass=N
#Allnet I conprv=N
#Allnet I firprv=V
P 1 00001.tif
P 2 00002.tif
D 829 allapviep1n200011900a_829 4
A I num_polizza=020400324

I need to replace the number in red with a start value as 1 with and increment with value +1 and so on for all this lines.

F 93246 allapviep1n200011900a 1 APVI
I 0 Date 10/04/2000
I 1 Time 20:04:00
I 2 Station pcde24
I 3 User admin
I 4 IPath images/allapviep1n200011900a
I 5 Prio 5
D 1 allapviep1n200011900a_1 2
A I num_polizza=020468324
A I sig_cogn=XAS
A I cod_tipo_doc=199
A I data_doc=17/04/2007
A I protocollo_doc=IAAL0000000007400191
#Allnet I protdc=IAAL0000000007400191
#Allnet I coddoc=199
#Allnet I numpol=020400324
#Allnet I sigcog=XAS
#Allnet I statosal=B
#Allnet I malatt=N
#Allnet I ricove=N
#Allnet I vispec=N
#Allnet I fircon=S
#Allnet I firass=N
#Allnet I conprv=N
#Allnet I firprv=V
P 1 00001.tif
P 2 00002.tif
D 2 allapviep1n200011900a_2 4
A I num_polizza=020400324

Thanks for your help...

jorrasdk · Apr 18, 2007#22007-04-18T19:39+00:00

lovebit, I realize that you asked for a macro, but if you are using UE13 you could also use this simple script:

Code: Select all

UltraEdit.ueReOn(); /* Let's use UE's own regexp syntax */
UltraEdit.activeDocument.findReplace.regExp = true;
UltraEdit.activeDocument.findReplace.replaceAll = false;

//Find lines like:
//D 829 allapviep1n200011900a_829 2      
//We use 3 tagged expressions grouping like this (shown with [] pairs)
//[D ]829[ allapviep1n200011900a_]829[ 2      ]
var regexpFind = "%^([D][ ]^)[0-9]+^([ ]*_^)[0-9]+^([ ][0-9]*^)$";

// First go to top of document
UltraEdit.activeDocument.top();

var renumberValue = 1; /* Start value */
while ( (UltraEdit.activeDocument.findReplace.find(regexpFind)) && ! UltraEdit.activeDocument.isEof() ) {
  /* now replace using tagged values ^1 ^2 ^3 */
	UltraEdit.activeDocument.findReplace.replace(regexpFind,"^1"+renumberValue+"^2"+renumberValue+"^3");
	renumberValue++;	
}

// reposition to top after renumbering
UltraEdit.activeDocument.top();

maryv · Apr 18, 2007#32007-04-18T22:25+00:00

Very slick! Nice job!

lovebit · Apr 19, 2007#42007-04-19T10:25+00:00

Superlative, great thanks!!!

Zababa · Oct 04, 2007#52007-10-04T08:53+00:00

Yes, it works. That is a kind of thing I need but still not all of it. Therefore I'd like some more help on this script.

I have modified the core lines to find a simple string of "string".

Code: Select all

var regexpFind = "^(string^)";

UltraEdit.activeDocument.top();

var renumberValue = 1; /* Start value */

while ( (UltraEdit.activeDocument.findReplace.find(regexpFind)) && ! UltraEdit.activeDocument.isEof() ) {
UltraEdit.activeDocument.findReplace.replace(regexpFind,"^1"+renumberValue);
   renumberValue++;   
}

Of the multiple occurences of "string" in the document it makes
string1, string2, string3, ... string9, string10, string11, ...

But how must the script look like if I need to number with one, two or three leading zeroes? i. e. to number multiple occurences of "string" to look like:

string01, string02, ... string09, string10, string11 ...

or:

string001, string002, ... string009, string010, string011 ... string099, string100, string101, ...

or even the same with three zeroes to number 1000 or more occurences of "string"?

Actually the best solution would be if the script counts the occurences of "string" before replacing. If it finds the 153 occurences it would decide to number them with two leading zeroes, if it finds 6540 occurences it would number them with three leading zeroes, etc.

I would much appreciate your help on this.

Mofi · Oct 04, 2007#62007-10-04T15:50+00:00

Here is a quick solution for your numbering with leading zeros. It is not very smart, but works for numbers up to 99999. If you need more you can easily expand it to support more.

Code: Select all

UltraEdit.ueReOn(); /* Let's use UE's own regexp syntax */
UltraEdit.activeDocument.findReplace.regExp = true;
UltraEdit.activeDocument.findReplace.replaceAll = false;

var regexpFind = "^(string^)";

// First go to top of document
UltraEdit.activeDocument.top();

var renumberValue = 1; /* Start value */

while ( (UltraEdit.activeDocument.findReplace.find(regexpFind)) && ! UltraEdit.activeDocument.isEof() ) {
   UltraEdit.activeDocument.findReplace.replace(regexpFind,"^1"+renumberValue+"#END#");
   renumberValue++;
}
// reposition to top after renumbering
UltraEdit.activeDocument.top();

renumberValue--;
UltraEdit.activeDocument.findReplace.replaceAll = true;

if (renumberValue >= 100000) {
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9]+^)#END#","^1^2");
   UltraEdit.messageBox("The script does not fully support numbers higher than 99999 !!!","Error: Too Large Number");
} else if  (renumberValue >= 10000) {
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9]^)#END#","^10000^2");
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9][0-9]^)#END#","^1000^2");
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9][0-9][0-9]^)#END#","^100^2");
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9][0-9][0-9][0-9]^)#END#","^10^2");
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9][0-9][0-9][0-9][0-9]^)#END#","^1^2");
} else if  (renumberValue >= 1000) {
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9]^)#END#","^1000^2");
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9][0-9]^)#END#","^100^2");
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9][0-9][0-9]^)#END#","^10^2");
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9][0-9][0-9][0-9]^)#END#","^1^2");
} else if  (renumberValue >= 100) {
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9]^)#END#","^100^2");
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9][0-9]^)#END#","^10^2");
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9][0-9][0-9]^)#END#","^1^2");
} else if  (renumberValue >= 10) {
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9]^)#END#","^10^2");
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9][0-9]^)#END#","^1^2");
} else {
   UltraEdit.activeDocument.findReplace.replace(regexpFind+"^([0-9]^)#END#","^1^2");
}

By the way: You hopefully know the Insert Number feature of UltraEdit in column mode and don't use this script to insert a number on strings with leading zeros which are positioned in a column. Using insert number would be much faster than the script.

Zababa · Oct 04, 2007#72007-10-04T16:41+00:00

Thank you, Mofi.

I do not need this script to number strings placed in column. My situation is very much like lovebit described.

I am very glad that you could come with some solution as soon as this.

Thank you once again.

Jun 04, 2008#82008-06-04T12:27+00:00

Hello Mofi,

I need your help again with that script. I don’t know why, but it does not work anymore.
I now am using UE 14.00b.

I changed the script a bit: Instead of a concrete string to search (e.g. "ref NA080105-04_B.") I put in a generalized regex template: "ref [A-Z][A-Z][0-9][0-9][0-9][0-9][0-9][0-9]-[0-9][0-9]_[A-Z]." (i. e. two capital letters, six numbers, dash, two numbers, underscore, one capital letter, dot) which should match everything I will ever need to renumber.

I have a source file with a numbering gap (133 followed by 135):
...
\ref NA071122-02_C.132
\ref NA071122-02_C.133
\ref NA071122-02_C.135
\ref NA071122-02_C.136
...

I want to renumber it (..., 132, 133, 134, 135, ...).

The renumbering script seems never to leave the 'while' loop. It correctly replaces the old running numbers with new ones (plus #END#), but somehow continues, until Windows and Ultraedit report memory overflow. The script does not remove the temporary '#END#' and does not add leading zeroes. )-: I checked the source code again and again but cannot find any bugs.

Please have a look at it. I send you my current version of the renumbering script and the file with a the numbering gap which is to be corrected by the script. (The file is encoded in UTF-8).

Mofi · Jun 04, 2008#92008-06-04T16:11+00:00

Uncheck search setting Continue find at End of File in the configuration dialog at Advanced - Configuration - Search - Miscellaneous. I have already reported to IDM after release of UE v14.00b that I have detected that

UltraEdit.activeDocument.isFound()

or

if (UltraEdit.activeDocument.findReplace.find("any search string"))

returns always true when this configuration setting is checked even if the search string cannot be found anymore in the file. IDM support confirmed that wrong behavior. It exists in all UE v14.00 up to v14.00b, but not in previous v13. I'm just wondering that I was the first one which detected that bug which results in an endless loop. Maybe this happens only with the UltraEdit regular expression engine. Would be worth to verify this. The bug was fixed with UE v14.00b+1 and UES v6.50b+1 where value of setting Continue find at End of File is ignored during script execution as it should be and as it was prior UE v14.00.

Zababa · Jun 05, 2008#102008-06-05T11:49+00:00

Mofi wrote:Uncheck search setting Continue find at End of File ...

Thank you, that was it. Script works now as before in UE 13.x

mjohnson767 · Nov 07, 2011#112011-11-07T16:55+00:00

For single value search and replacement of expression searches the tool works great. I'd like to take the process much further by incorporating the same search to replace multiple values in the same file. Then somehow put these search parameters into a batch file to automate it.

Here's an example of the value I'm search for.

1st instance - look for the first GUID and replace ALL throughout the entire file...
||PKNAME|FIELDLIST_ID|||PKVALUE|1C603557AB938FD8F46970305083EF7F||VALUE_DT^null~VALUE_MC_NU^-1~VALUE_YN_CH^^FIELDLIST_ID^1C603557AB938FD8F46970305083EF7F~CR

Replace with
||PKNAME|FIELDLIST_ID|||PKVALUE|STRING_VALUE_001||VALUE_DT^null~VALUE_MC_NU^-1~VALUE_YN_CH^^FIELDLIST_ID^STRING_VALUE_001~CR

2nd instance: (would be the 2nd instance in the file for PKVALUE...
||PKNAME|FIELDLIST_ID|||PKVALUE|STRING_VALUE_002||VALUE_DT^null~VALUE_MC_NU^-1~VALUE_YN_CH^FIELDLIST_ID^STRING_VALUE_002~CR

There are sometimes over 180 instances of PKValues alone and I've had to manually click on the GUID and Replace all in the file with an incremented string value for later use.

So, the question is: How do I create an expression search "PKVALUE||[0-9A-Z][0-9A-Z] etc. that would create a MATCHING search for the expression and then replace all.
Then, right a batch file that would run all of the search and replacements in the file. Manually, this process takes hours to do.

Mofi · Nov 08, 2011#122011-11-08T06:45+00:00

Because of the incrementing number a Replace All command can't be used. A script solution is required for this task. Above you can see already some script solutions for that task. Here is a script solution written for your task working for your example copied into an ASCII file with DOS line terminators using UE v17.30.0.1002.

Code: Select all

if (UltraEdit.document.length > 0)
{
   UltraEdit.insertMode();
   if (typeof(UltraEdit.columnModeOff) == "function") UltraEdit.columnModeOff();
   else if (typeof(UltraEdit.activeDocument.columnModeOff) == "function") UltraEdit.activeDocument.columnModeOff();
   UltraEdit.ueReOn();
   UltraEdit.activeDocument.top();

   UltraEdit.activeDocument.findReplace.mode=0;
   UltraEdit.activeDocument.findReplace.matchCase=true;
   UltraEdit.activeDocument.findReplace.matchWord=false;
   UltraEdit.activeDocument.findReplace.regExp=true;
   UltraEdit.activeDocument.findReplace.searchDown=true;
   UltraEdit.activeDocument.findReplace.searchInColumn=false;
   UltraEdit.activeDocument.findReplace.preserveCase=false;
   UltraEdit.activeDocument.findReplace.replaceAll=false;
   UltraEdit.activeDocument.findReplace.replaceInAllOpen=false;
   
   var sSearchRegExp = "^(||PKNAME|FIELDLIST_ID|||PKVALUE|^)*^(||*FIELDLIST_ID^^^)*~CR";
   var sReplaceRegExp = "^1STRING_VALUE_000^2STRING_VALUE_000~CR";
   var sNumber = "000";
   var nNumber = 0;
   do
   {
      nNumber++;
      if (nNumber >= 100) sNumber = nNumber.toString();
      else if (nNumber >= 10) sNumber = "0" + nNumber.toString();
      else sNumber = "00" + nNumber.toString();
      sReplaceRegExp = "^1STRING_VALUE_" + sNumber + "^2STRING_VALUE_" + sNumber + "~CR";
   }
   while (UltraEdit.activeDocument.findReplace.replace(sSearchRegExp,sReplaceRegExp));
   UltraEdit.save();
}

You can run this script also from within a batch file or via a shortcut with the command line:

"Path of UltraEdit program files folder\uedit32.exe" /fni "full name of the file to modify" /s,e="full name of the script file"

mjohnson767 · Nov 09, 2011#132011-11-09T18:35+00:00

Thanks for the reply and I've corrected my mistakes. I'm using UEStudio version 10.30.0.1003.

I understand the design of the .js script and I believe this is the right approach. I also understand that it does the following:
sSearchRegExp - sets up the value I'm looking for.
sReplaceRegExp - sets up what's going to replace the sSearchRegExp value with sNumber being the incrementing value concatenated with the sReplaceRegExp replacement value.
(The code follows a line to line approach.)

The problem with this approach is:
Many of the expressions themselves are reused throughout the Silk script because they are inter-joined with other methods they're seeking to complete. Following a line by line approach and replacing the expressions is only part of the equation.

What I can't figure out is how to find the expression and place it into a variable and then run the findReplaceAll on the expression found command; then move to the next expression search or next line.

In the attached file (deleted after post of Mofi) look for the first occurence of 302948E2E7330C1A75A170304BBC13CF which is found by looking for the LB: ACQUISITIONFIELDLIST_ID|||PKVALUE|. You see that the value is used four times in file. The value needs to be replaced with "STRING_VALUE_001" for all four instances, or concatenated in the file with: " + STRING_VALUE_001 + ". Then find the next instance of ACQUISITIONFIELDLIST_ID|||PKVALUE| and replace the returned expression with " + STRING_VALUE_002 + ". IF the value has already been replaced then ignore it and move on to the next expression search until done.

Mofi · Nov 10, 2011#142011-11-10T18:42+00:00

Okay. The task is a little bit different as I thought according to your first post. But the modifications necessary on the script to achieve it were small and easy for me to do. Here is the new script and this time with comments. I tested it on your file and I think it produced the result you want.

Code: Select all

if (UltraEdit.document.length > 0)
{
   // Define editing mode for the script.
   UltraEdit.insertMode();
   if (typeof(UltraEdit.columnModeOff) == "function") UltraEdit.columnModeOff();
   else if (typeof(UltraEdit.activeDocument.columnModeOff) == "function") UltraEdit.activeDocument.columnModeOff();
   // Use the UltraEdit regular expression engine.
   UltraEdit.ueReOn();
   // Set caret to top of file.
   UltraEdit.activeDocument.top();

   // Define the parameters for find and replace.
   UltraEdit.activeDocument.findReplace.mode=0;
   UltraEdit.activeDocument.findReplace.matchCase=false;
   UltraEdit.activeDocument.findReplace.matchWord=false;
   UltraEdit.activeDocument.findReplace.regExp=true;
   UltraEdit.activeDocument.findReplace.searchDown=true;
   if (typeof(UltraEdit.activeDocument.findReplace.searchInColumn) == "boolean")
   {
      UltraEdit.activeDocument.findReplace.searchInColumn=false;
   }
   UltraEdit.activeDocument.findReplace.preserveCase=false;
   UltraEdit.activeDocument.findReplace.replaceAll=true;
   UltraEdit.activeDocument.findReplace.replaceInAllOpen=false;

   // This expression in UltraEdit syntax is for finding the hexadecimal
   // values of ACQUISITIONFIELDLIST_ID or ACQUISITIONORGLIST_ID. The
   // asterisk in the regular expression string makes it an OR expression.
   var sSearchRegExp = "|PKNAME|ACQUISITION*_ID|||PKVALUE|[0-9A-F]+";
   var sNumber = "000";
   var nNumber = 0;
   // Run the following loop as long as a hexadecimal value can be found.
   while(UltraEdit.activeDocument.findReplace.find(sSearchRegExp))
   {
      nNumber++;
      if (nNumber >= 100) sNumber = nNumber.toString();
      else if (nNumber >= 10) sNumber = "0" + nNumber.toString();
      else sNumber = "00" + nNumber.toString();
      var sReplace = "\" + STRING_VALUE_" + sNumber + " + \"";
      // The found string is selected. Replaced should be only all
      // occurrences of the last 32 characters. It would be very easily
      // possible to get just the last 32 characters from the selected
      // string. But it is perhaps better to get everything after last |
      // of selected string into a string variable in case the length
      // of the hexadecimal value will change ever in the future.
      var nLastPipePos = UltraEdit.activeDocument.selection.lastIndexOf('|');
      var sFound = UltraEdit.activeDocument.selection.substr(++nLastPipePos);
      // Search for the hexadecimal value and replace all occurrences
      // with current string value number. Caret position in the file
      // is not changed on replace all. So next find continues on first
      // line where the string was just replaced.
      UltraEdit.activeDocument.findReplace.replace(sFound, sReplace);
   }
   // Save the file when modified.
   if (nNumber) UltraEdit.save();
}

mjohnson767 · Nov 10, 2011#152011-11-10T21:32+00:00

Wow, that is really cool. I thought I was going to have to learn a programming language like Groovy to get this to work...

Totally awesome. Thanks.