How to filter large log or trace files for lines of interest?

How to filter large log or trace files for lines of interest?


    Oct 20, 2011#1

    I have very large trace files I need to search through. I was curious if via script or macro if I could search for predefined text and every time that text is found have UE add a bookmark on that line (with the name of the bookmark being a specific portion of the line found 'i.e. customer name')?

    I looked through much of the bookmark and macro posts and did not find anything this specific. I did note that some old posts said there is a limit of 50 bookmakrs? Is this still the case with the most revent version of UE? I could not find that limitation in my help docs.

    Also, I often make a lot of small macros with no problem, but something as simple as having the macro acknowledge an "ok" push button for a pop up dialog I cannot figure out. I often use another macro language outside of UE to handle these situations but given I am here I thought I would ask. :)

    Grand MasterGrand Master

      Oct 22, 2011#2

      As you can read at Is there a limit for bookmarks per file? the limit is currently 500 for UE v17.xx.

      It is possible to set bookmarks on found lines, but without giving the bookmark a name. You can see on help page Editing Macro command all available macro commands. What is not listed on this page can't be executed from within a macro. Commands requiring user interaction are in general not supported, with the exception of GetValue, GetString and SaveAs with an empty string as new file name. More details on the macro commands can be found in my macro reference packed in the ZIP file which can be downloaded from sticky Macro examples and reference for beginners and experts.

      I want to suggest other methods to evaluate large log files:
      1. Use advanced find option List Lines Containing String to get in a separate, resizable window a list of found lines which you can copy to clipboard and paste into a new file, or double click on the lines to set caret to this line in active file while window remains open.
      2. Use advanced find option Show Lines to display only the lines with information you are interested in and hide all others.
      3. Use Find in Files on Open Files to get the list of lines with the interested data either in output window or in a new edit window. On the results in the edit window you can right click on the file name with line number to see this line in the large log file or use the hotkey assigned to command FileOpenFileUnderCursor. If the found lines are printed to the output window, you can double click on the line in the output window to position the caret on the line in the large log file, or even better, use the keys Ctrl+Shift+Down Arrow and Ctrl+Shift+Up Arrow to navigate through the found lines as displayed in the output window from within the document window of the large log file.
      To find several strings of interest at once, use find option Regular Expressions: Perl with regular expression character | between the strings which means OR.

      Of course you can use also a macro or script to copy all interesting lines into a new file. There are several user contributed scripts on user-submitted scripts page and in the Scripts forum as well as macros in the Macros forum demonstrating how to extract lines from a file to a new file.

      Clipboard 9
      Find RegExp "^.*(?:
      string 1|string 2|string 3|and so on).*(?:\r\n|\n|\r)"
      Clipboard 0

      The other regular expression characters outside the OR combined search strings are just for matching the entire lines containing one of the strings in the OR expression.