Reformatting paragraph with long words doubles 1 or 2 characters in UE v24.00 (fixed)

Reformatting paragraph with long words doubles 1 or 2 characters in UE v24.00 (fixed)

25
Basic UserBasic User
25

    Mar 21, 2017#1

    Hi All,
    I have SQL Statements with long words AND/OR long comma separated field/value list with very few blanks.

    Code: Select all

    insert into AZTEST_BETRIEBE(ID,JAHR,REIHENFOLGE,BETRIEBSNUMMER,BESCHREIBUNG,KONSEQUENZ)values(null,2016,1,2762196,'asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl99','qwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiop88');
    When I try to break such a line into multiple lines via Ctrl+T, the character at position 81 of the long word / field list gets doubled.
    At the 2nd, 3rd, etc. Ctrl+T, a blank is added as well. 8O
    Can anyone reproduce this?

    insert into
    AZTEST_BETRIEBE(ID,JAHR,REIHENFOLGE,BETRIEBSNUMMER,BESCHREIBUNG,KONSEQUENZ)values
    s(null,2016,1,2762196,'asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl
    asdfghjkl asdfghjkl asdfghjkl asdfghjkl
    asdfghjkl99','qwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyu
    uiopqwertyuiopqwertyuiopqwertyuiopqwertyuiop88');

    UltraEdit Professioneller Text-/HEX-Editor (x64)
    Version 24.00.0.53, German
    on Windows 7 Pro German 64-bit

    Update: This bug was fixed with public released UltraEdit v24.00.0.73.
    Regards from Vienna

    6,603548
    Grand MasterGrand Master
    6,603548

      Mar 22, 2017#2

      Hi Miklos!

      Yes, this wrong behavior could be reproduced by me with UE v24.00.0.56 exactly as you have described it. Please report this issue by email to IDM support.

      I verified the Reformat Paragraph behavior with Right margin (after # column) set to 80 in Paragraph Setup/Formatting with provided example also with UE v22.20.0.49 (last XP version) and with UE v23.20.0.43 (on Windows 7). The older versions of UltraEdit do the paragraph reformatting right. There is no character duplicated. So this is an UltraEdit v24.00 bug.
      Best regards from an UC/UE/UES for Windows user from Austria

      25
      Basic UserBasic User
      25

        Mar 23, 2017#3

        Hi Mofi,
        Thanks for your feedback. Issue has been sent to IDM support by email including your comments.

        Don't you think bugs like these should show up during the test phase if IDM would use test automation with a sufficient diversity of test files? 8O
        Aren't UE macro capabilities stable enough to allow more thorough tests and more reliable releases? :?:
        e.g. the 2nd Ctrl+T should leave a paragraph as it is :idea: , which is not so difficult to check with every new UE version for a decent bunch of files.
        Regards from Vienna

        6,603548
        Grand MasterGrand Master
        6,603548

          Re: Reformatting paragraph with long words doubles 1 or 2 characters in UE v24.00

          Mar 24, 2017#4

          There is no UltraEdit script (or macro) command to run Reformat Paragraph. So it is not possible to check this feature with an UltraEdit script. This command is a user mode only feature because it depends on the Paragraph Setup/Formatting settings.

          But in general it would be possible to compile with the source code files of UltraEdit a special test application which tests this feature automated before releasing a new version of UltraEdit. We do that in our company producing devices and applications for protection, synchronization, automation and excitation of power plants and substations for many source code files. But this requires an appropriate architecture of the source code files for being compilable in different applications (real application and special test applications) and of course lots of extra time for writing the test applications/procedures. Sometimes writing the code for testing a function takes 3 to 10 times longer than writing the code of the tested function.

          So finally the questions are:

          Is it cost effective for IDM Computer Solutions, Inc. to take the time and write code / test procedures for automated tests and run them regularly before release of a new version in highly competitive market of text editors with lots of freeware text editors?

          What happens if a function of text editor UltraEdit does not work as expected and is it therefore worth the time (=money) to write a test procedure for an automated test of this function?

          Do you have ever thought why military systems, aerospace systems, aircrafts, railway systems, devices and applications for power plants, ... are much more expensive than devices and software for consumer market?
          Best regards from an UC/UE/UES for Windows user from Austria

          25
          Basic UserBasic User
          25

            Mar 31, 2017#5

            Thanks for pointing out the lack of the corresponding macro command.

            We are heading to the multidimensional never-ending (off-)topic of Performance-functionality-quality-price-time-knowledge, which we could evtl. discuss bei einem Bier.

              Apr 05, 2017#6

              Hi Mofi,
              I have just tested Ctrl+T in UE v24.0.0.70 UVB (private user verification build).
              Letters aren't anymore duplicated, that's great!
              But... when I do a "Reformat Paragraph" on a program code that has lines with 81 characters, without blanks but with a few commas and (), which ends with a word,
              the last word gets broken in two, although the line could be split at the last comma. It differs essentially from what programmers would expect.

              Can you reproduce it?

              Before:

              Code: Select all

              insert into
              AZTEST_BETRIEBE(ID,JAHR,REIHENFOLGE,BETRIEBSNUMMER,BESCHREIBUNG,KONSEQUENZ)values
              (null,2016,1,2762196,'asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl
              asdfghjkl asdfghjkl asdfghjkl asdfghjkl
              asdfghjkl99','qwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyu
              iopqwertyuiopqwertyuiopqwertyuiopqwertyuiop88');
              After Ctrl+T:

              Code: Select all

              insert into
              AZTEST_BETRIEBE(ID,JAHR,REIHENFOLGE,BETRIEBSNUMMER,BESCHREIBUNG,KONSEQUENZ)value
              s (null,2016,1,2762196,'asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl
              ...
              Regards from Vienna

              6,603548
              Grand MasterGrand Master
              6,603548

                Apr 05, 2017#7

                What programmers expect depends on programming language. The Reformat Paragraph command is a general text editing feature and not a reformatting feature according to rules for a specific programming or scripting language. Artistic Style Formatter is a reformatter tool for specific languages integrated into UE/UES.

                Reformat Paragraph takes only whitespace characters into account whereby I don't know if it uses only normal space, horizontal tab, carriage return and line-feed or also other rarely used whitespace characters according to Unicode definition on which inserting a line break is acceptable.

                Reformatting a paragraph only using whitespaces is already problematic in many languages like C/C++. Let us look on following C/C++ code line:

                Code: Select all

                const char MyString[] = "This is an example of a long string which can't be easily reformatted.";
                Command Reformat Paragraph with 80 for right margin reformats this line to:

                Code: Select all

                const char MyString[] = "This is an example of a long string which can't be
                easily reformatted.";
                Those two lines in a source file would not compile anymore.

                A correct reformatting would require not deleting space character between be and easily and inserting a backslash at end of first line.

                Example 1 for correct reformatting:

                Code: Select all

                const char MyString[] = "This is an example of a long string which can't be\
                 easily reformatted.";
                Example 2 for correct reformatting:

                Code: Select all

                const char MyString[] = "This is an example of a long string which can't \
                be easily reformatted.";
                Next let us look on another example, this time for HTML. The HTML file contains the line:

                Code: Select all

                <p>This paragraph links to <a href="http://www.ultraedit.com">UltraEdit website</a>.</p>
                It is reformatted to:

                Code: Select all

                <p>This paragraph links to <a href="http://www.ultraedit.com">UltraEdit
                website</a>.</p>
                That is no problem because the line termination within value of element A is interpreted as space character according to HTML standard and the link is displayed as UltraEdit website as it should. So everything is fine with those two lines.

                But what would happen if any non word character would be used by command Reformat Paragraph in any case. The result would be:

                Code: Select all

                <p>This paragraph links to <a href="http://www.ultraedit.com">UltraEdit website
                </a>.</p>
                This HTML code is still valid. But it results in a link being displayed as UltraEdit website with an underlined space at end. That's of course not good and definitely not wanted.

                So does it make sense to enhance general text editing feature Reformat Paragraph to wrap on whitespaces, but if a line does not contain any whitespace, wrap on any other non word character?

                YES, in your case for your language, but most likely NO in the majority of other use cases.

                In my opinion the Reformat Paragraph command should wrap only on whitespace characters with the exception of non breaking space, zero width joiner and other whitespace characters on which line breaks are not allowed according to Unicode definition.

                I would suggest for your task a script which reformats the selected text in memory according to the syntax rules of the used language to get the best result whenever you reformat paragraphs. The script could take the file extension of active file into account to really work language dependent in case of multiple languages should be supported by the script. In case of active file is a not yet saved file or has no file extension, the script could ask the user which language should be used for reformatting. (Most difficult to code would be a smart language based reformatting script working for HTML with other languages embedded. But there are already tools available and supported built-in by UE for reformatting HTML files.)
                Best regards from an UC/UE/UES for Windows user from Austria

                25
                Basic UserBasic User
                25

                  Apr 07, 2017#8

                  Hi,
                  Thank for the detailed explanation - even if there wasn't anything new for me in it except for the Artistic Style Formatter.
                  Apparently, SQL and most other languages differ at many points, so do their developers' habits.
                  e.g. how multiline strings are treated, whether you can execute code dynamically, etc.

                  My actual questions are:
                  Why does Ctrl+T still break lines at position 80?
                  Why aren't lines broken in two when typing lines longer than 130 characters?

                  Line break (Umbruchverfahren) is set for "Spaltennumer" = 130,
                  chosen option is "Umbruch nach Spalte Nr. ", which according to help, should do something like this:
                  German UE help wrote:Wählen Sie diese Option aus, um den Umbruch an der angegebenen Spalte durchzuführen, ohne einen festen Zeilenwechsel einzufügen. Hierbei erfolgt der Umbruch an Wortgrenzen. Es kann also vorkommen, dass der Umbruch bereits an einer kleineren Spaltennummer erfolgt.
                  (My PC still has UE 24.0.0.53, could you please confirm this with UE 24.0.0.70 UVB or later?)
                  Regards from Vienna

                  6,603548
                  Grand MasterGrand Master
                  6,603548

                    Apr 08, 2017#9

                    With private user verification build 24.00.0.70 the reformatting paragraph behavior for the example paragraph is restored to behavior as in UE v22.20.0.49 on Windows XP and v23.20.0.43 on Windows 7.

                    With using UE in toolbar/menu mode with traditional menus I opened Format - Paragraph formatting - Paragraph setup/formatting and configured:
                    • Hanging indent: not checked
                    • Margins group:
                      • Use paragraph left margin  selected.
                      • The option Specify margin (# columns)  was not used.
                      • Hanging indent (# columns)  was disabled.
                      • Right margin (after # columns)  was configured with 130.
                    • Alignment group:Left selected.
                    The selected paragraphs were:

                    Code: Select all

                    insert into AZTEST_BETRIEBE(ID,JAHR,REIHENFOLGE,BETRIEBSNUMMER,BESCHREIBUNG,KONSEQUENZ)values(null,2016,1,2762196,'asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl99','qwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiop88');
                    
                    insert into
                    AZTEST_BETRIEBE(ID,JAHR,REIHENFOLGE,BETRIEBSNUMMER,BESCHREIBUNG,KONSEQUENZ)values
                    (null,2016,1,2762196,'asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl
                    asdfghjkl asdfghjkl asdfghjkl asdfghjkl
                    asdfghjkl99','qwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyu
                    iopqwertyuiopqwertyuiopqwertyuiopqwertyuiop88');
                    
                    The result after execution of View - Reformat paragraph with Ctrl+T using UE UVB 24.00.0.70 was:

                    Code: Select all

                    insert into AZTEST_BETRIEBE(ID,JAHR,REIHENFOLGE,BETRIEBSNUMMER,BESCHREIBUNG,KONSEQUENZ)values (null,2016,1,2762196,'asdfghjkl
                    asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl
                    asdfghjkl99','qwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyu iopqwertyuiopqwertyuiopqwertyuiopqwertyuiop88');
                    
                    insert into AZTEST_BETRIEBE(ID,JAHR,REIHENFOLGE,BETRIEBSNUMMER,BESCHREIBUNG,KONSEQUENZ)values (null,2016,1,2762196,'asdfghjkl
                    asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl asdfghjkl
                    asdfghjkl99','qwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyuiopqwertyu iopqwertyuiopqwertyuiopqwertyuiopqwertyuiop88');
                    
                    So the first line of each paragraph has 125 characters, the second line 79 and the third line exactly 130. That is the absolutely correct behavior.

                    Please note that Format - Paragraph formatting - Paragraph setup/formatting opens the Reformatting Options (dialog window title in English UE). Those options are not used on having enabled the word-wrap feature for active file. Those options are only used on executing command Reformat paragraph.

                    The settings for wrapping long lines with word-wrap feature enabled have to be configured at Advanced - Settings/Configuration - Editor - Word wrap / tab settings. I suppose that you have configured there 80 for option Wrap column #. For more details about word-wrap see forum topic File extension based word wrap, tab and indent settings and perhaps also other topics found using advanced forum search with searching for word wrap file extension only in forum Configuration/INI Settings.
                    Best regards from an UC/UE/UES for Windows user from Austria