Is it possible to edit a binary (hex) file with scripts?

Is it possible to edit a binary (hex) file with scripts?

2
NewbieNewbie
2

    Sep 21, 2015#1

    hi all,
    I need to change certain hex values in a file and I thought this might be possible with a nice script.
    My question is, is it possible?

    And if so, I would appreciate some hints which commands I should/could use to:

    - place (select) at a certain position (offset)
    - how to read this value at the position
    - how to write another value at this position

    TIA SBob

    [UltraEdit 15.20.1.1000 SE]

    6,688586
    Grand MasterGrand Master
    6,688586

      Sep 21, 2015#2

      Yes, it is possible to edit a binary file with an UltraEdit script in hex editing mode.

      In general replacing bytes in a binary file is done most efficiently with using a Perl regular expression Replace in Files executed on a specific file not being opened at all in UltraEdit because with Perl it is possible to define search and replace strings using hexadecimal notation. But this method could be used only if the byte sequence to replace exists definitely always only once in the file, except all occurrences of the byte sequence should be replaced in the file.

      Scripting command gotoPos() can be used to move caret to a specific position in file. And making a selection is best done with gotoPosSelect().

      The selected bytes can be compared as string even if it contains null bytes as it is possible in JavaScript to define a string with characters of any code value and JavaScript compares strings based on string length and not up to first null byte.

      Replacing a selected binary byte sequence is done best using a clipboard and command paste() as this makes it very easy to replace X bytes by Y bytes.

      Here is a demo code using these methods:

      Code: Select all

      // Define environment for this script.
      UltraEdit.insertMode();
      if (typeof(UltraEdit.columnModeOff) == "function") UltraEdit.columnModeOff();
      else if (typeof(UltraEdit.activeDocument.columnModeOff) == "function") UltraEdit.activeDocument.columnModeOff();
      
      // Create a new ASCII file and write a simple text into the file.
      UltraEdit.newFile();
      UltraEdit.activeDocument.unicodeToASCII();
      UltraEdit.activeDocument.write("This is a demo containing number \x05 at offset 21h.");
      
      // Enable hex editing mode for the file and set caret to byte offset 33 (0x21).
      UltraEdit.activeDocument.hexOn();
      UltraEdit.activeDocument.gotoPos(33);
      
      // Select one byte.
      UltraEdit.activeDocument.gotoPosSelect(34);
      
      // Has the selected byte the value 5?
      if (UltraEdit.activeDocument.selection == "\x05")
      {
         // It should be an ASCII 5 with hexadecimal value 0x35.
         // Copy the right byte value to user clipboard 9 and
         // paste it over the selected byte in hex editing mode.
         UltraEdit.selectClipboard(9);
         UltraEdit.clipboardContent = "5";
         UltraEdit.activeDocument.paste();
         UltraEdit.clearClipboard();
         UltraEdit.selectClipboard(0);
      
         // Also working for this case would be deleting the byte with wrong value
         // and insert another one with right value. Note: "35" is written into
         // file in hexadecimal area of hex editing window and therefore inserts
         // a single byte with decimal value 53 instead of 2 bytes with 0x33 0x35.
      //   UltraEdit.activeDocument.gotoPos(33);
      //   UltraEdit.activeDocument.hexDelete(1);
      //   UltraEdit.activeDocument.write("35");
      }
      UltraEdit.activeDocument.top();
      
      I think, your special version of UltraEdit already supports access to clipboard content. So you should be able to use above. Try the script.

      Replacing a byte sequence with no support for accessing clipboard content is also possible as shown above when commenting in IF condition the currently active code lines and uncomment instead the currently commented code lines.

      Here is one more scripting example which I sent to IDM support on January 2010 with the issues I could see on using this script.
      Mofi wrote:Working with a script in hex edit mode is completely different

      This issue was detected with UE v15.20.0.1022. Use the attached script command to see all the issues I'm reporting here. The script produces 3 new files showing you 3 different problems. The first 2 new files are for showing you the display after inserting hexadecimal values in hexadecimal representation area while the third file shows you what happens when inserting "NOP" in the ASCII representation area.

      While manually editing binary values in hex editing mode 0-9A-F overwrite always the current byte value independent if INS or OVR is active if the cursor is in the hexadecimal representation area. If the cursor is in the ASCII representation area the current character is overwritten when pressing a key and the appropriate byte value changes again independent of INS or OVR. The command "Hex Insert/Delete" must be used to delete or insert bytes and then the inserted bytes (spaces) can be modified.

      But from within a script the command write() always inserts 0-9A-F in the hexadecimal representation area and all characters are inserted without inserting really a byte to the file in the ASCII representation area. In other words the command write() is currently not really working in hex editing mode and what it does is completely different in comparison to a manual editing.

      Also the display is not correct after inserting for example a single byte with command UltraEdit.activeDocument.write("30"); The command top() can be used to force a display refresh to get correct display in hexadecimal and ASCII area after inserting a byte using write method.

      Another problem is that when manually editing in hex edit mode the key TAB can be used to switch from the hexadecimal representation area to the ASCII representation area. But this possibility to change the editing area in hex editing mode is not possible from within a script. To change the editing area only the commands top() and bottom() can be used because top() sets the cursor to first byte in hexadecimal area while command bottom() sets the cursor right the last character in the ASCII area and further key moving commands are now executed in the appropriate area. It would be good to introduce a new script command to be able to switch between hexadecimal and ASCII area like the TAB key does when manually editing in hex edit mode.

      Alternatively you could introduce new script commands for writing in hex editing mode to solve compatibility problems although I think there are only a handful of scripts working in hex editing mode. For example UltraEdit.activeDocument.hexWrite("4E4F50") or hexWrite("4E 4F 50") would be the command to always overwrite independent of INS or OVR in the hexadecimal area. Spaces in the string must be removed by UltraEdit internally and then the string must have an even length (2 ASCII characters 0-9A-F per byte) and must contain only the characters 0-9A-F otherwise the command does nothing.

      UltraEdit.activeDocument.charWrite("NOP") would always overwrite in the ASCII area independent of INS or OVR. For compatibility the command write() can be still used to insert bytes in hexadecimal area.

      Note: In UE v13.00 and v13.10 the command write() overwrites bytes in the hexadecimal area like when manually editing. With UE v13.20 the behavior has changed.

      Code: Select all

      // Create array with all ASCII and ANSI Latin-1 characters starting
      // with the space character. Well, it depends on your default code
      // page if the characters 0x80 to 0xFF are really Latin-1 characters.
      asAnsiCharTable = new Array(" !\"#$%&'()*+,-./", // 0x20 to 0x2F
                                  "0123456789:;<=>?",  // 0x30 to 0x3F
                                  "@ABCDEFGHIJKLMNO",  // 0x40 to 0x4F
                                  "PQRSTUVWXYZ[\\]^_", // 0x50 to 0x5F
                                  "`abcdefghijklmno",  // 0x60 to 0x6F
                                  "pqrstuvwxyz{|}~",  // 0x70 to 0x7F
                                  "€‚ƒ„…†‡ˆ‰Š‹ŒŽ",  // 0x80 to 0x8F
                                  "‘’“”•–—˜™š›œžŸ",  // 0x90 to 0x9F
                                  " ¡¢£¤¥¦§¨©ª«¬­®¯",  // 0xA0 to 0xAF
                                  "°±²³´µ¶·¸¹º»¼½¾¿",  // 0xB0 to 0xBF
                                  "ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏ",  // 0xC0 to 0xCF
                                  "ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞß",  // 0xD0 to 0xDF
                                  "àáâãäåæçèéêëìíîï",  // 0xE0 to 0xEF
                                  "ðñòóôõö÷øùúûüýþÿ"); // 0xF0 to 0xFF
      
      var sData = "                                "+asAnsiCharTable.join("");
      UltraEdit.ueReOn();
      UltraEdit.insertMode();
      if (typeof(UltraEdit.columnModeOff) == "function") UltraEdit.columnModeOff();
      else if (typeof(UltraEdit.activeDocument.columnModeOff) == "function") UltraEdit.activeDocument.columnModeOff();
      UltraEdit.newFile();
      UltraEdit.activeDocument.unicodeToASCII();
      UltraEdit.activeDocument.unixMacToDos();
      UltraEdit.activeDocument.write(sData);
      UltraEdit.activeDocument.hexOn();
      UltraEdit.activeDocument.top();
      // It depends on the version of UE if the following characters are
      // overwriting the 32 spaces at top of the file or are inserted.
      // UE < v13.20 overwrite like when manually editing in hex editing
      // mode while all other versions insert the bytes.
      UltraEdit.activeDocument.write("00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F");
      UltraEdit.activeDocument.write("10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F");
      
      UltraEdit.newFile();
      UltraEdit.activeDocument.unicodeToASCII();
      UltraEdit.activeDocument.unixMacToDos();
      UltraEdit.activeDocument.write(sData);
      UltraEdit.activeDocument.hexOn();
      UltraEdit.activeDocument.top();
      UltraEdit.activeDocument.write("000102030405060708090A0B0C0D0E0F");
      UltraEdit.activeDocument.write("101112131415161718191A1B1C1D1E1F");
      
      UltraEdit.newFile();
      UltraEdit.activeDocument.unicodeToASCII();
      UltraEdit.activeDocument.unixMacToDos();
      UltraEdit.activeDocument.write(sData);
      UltraEdit.activeDocument.hexOn();
      // The top() command is used here to get initially a good display.
      UltraEdit.activeDocument.top();
      UltraEdit.activeDocument.bottom();
      UltraEdit.activeDocument.key("LEFT ARROW");
      UltraEdit.activeDocument.key("LEFT ARROW");
      UltraEdit.activeDocument.write("NOP");
      
      Best regards from an UC/UE/UES for Windows user from Austria

      2
      NewbieNewbie
      2

        Sep 22, 2015#3

        Hi Mofi,
        a zillion thanks for this amazing comprehensive answer.
        It is way more, than I expected.
        I'm pretty sure with those example scripts I can solve my issue in no time.

        Have a very nice day and thanks for your great "job" you are doing here.

        Best, SBob