Non DOS file display issue?

Non DOS file display issue?

1
NewbieNewbie
1

    15:02 - Aug 09#1

    Version 2024.0.0.35

    I relied on UE to visually validate files as being in good order and visually they looked good, however, when processed in Python I discovered every other line was empty. I opened one of the files in Notepad++ and it properly displays the empty lines. 

    UltraEdit
    ue.png (43.6KiB)

    Notepad++
    np.png (4.15KiB)

    Please help me to understand this behavior and how to setup UE to mirror Notepad++

    I have tried all of the Display and File handling options and the best I can tell is the file has a mix of non-dos and dos line terminators.

    I did not see this behavior in early version of UE.

    6,675585
    Grand MasterGrand Master
    6,675585

      18:02 - Aug 09#2

      There is no possibility with UltraEdit for Windows v2024.0.0.35 (and also v22.20.0.49 as verified by me too) to get CR CR LF displayed in text edit mode with enabled showing of line endings
      • with ± as placeholder for a carriage return as used for the Mac line ending type (just carriage return) and get this carriage return also interpreted as newline character, and
      • with as placeholder for carriage return plus line feed as used for the DOS/Windows line ending type on one more empty line.
      It is only possible to get the erroneous carriage return not automatically removed by UltraEdit and show it as standard character by having setting Automatically convert to DOS format not selected as otherwise the erroneous carriage return is automatically removed by UltraEdit (on opening for display and in file on storage media on next save). Some fonts have no glyph defined for this whitespace character. There is displayed a rectangle at the end of every line for the erroneous carriage return by these fonts and on showing of line endings currently enabled. There is used by default by UltraEdit the font Consolas which displays the carriage return like a normal space because of being a whitespace character which is not so good to see that erroneous newline character.

      It looks like Notepad++ replaces for the display of the newline characters a carriage return by the two characters C and R and a line feed by the two characters L and F. UltraEdit v2024.0.0.35 has no such special newline character display mode with replacing one newline character by two ASCII letters.

      There is the symbol for carriage return with Unicode code point value U+240D and the symbol for line feed with Unicode code point value U+240A. But these two symbols are very small in most fixed-width fonts and not often used for that reason as far as I know.

      I switch with Ctrl+H to hex edit mode and use Revert to saved for reloading the file in binary mode because of having selected by default Automatically convert to DOS format and see if there is a mixture of line ending types in the file if I suppose there is something wrong with a text file.

      There can be searched also for ^r^p (no regular expression or UltraEdit regular expression) or for \r\r\n (Unix/Perl regular expression) on suspecting the existence of such an invalid line ending type caused by a not good coded program or script or wrong downloaded file from an FTP server.

      There is in the traditional menu Search the command Character properties which is named Character info in ribbon mode on ribbon tab Edit in second group Review and in contemporary menu Edit to get displayed information about the character at current position of the text cursor in active file. Try that out with text cursor left to "space" at the line end and there can be see that this character has decimal value 13 (hexadecimal 0xd) which is not 32 (0x20) as a normal space has.

      There can be sent only a feature request to UltraEdit support on wanting a display of newline characters like in Notepad++ in a future version of UltraEdit and UEStudio.

      Hint: Open the Manage Themes dialog window, select second tab Editor and configure a font color like red or blue for last but one item Spaces, tabs, line ends. Then the placeholder characters for normal space, horizontal tab, just CR, just LF and CR+LF can be seen better. A "space" not displayed with a red/blue middle dot is not a normal space. It could be a no-break space or a carriage return or another whitespace. The command for showing the character properties/information should be used on such a "space".
      Best regards from an UC/UE/UES for Windows user from Austria

      1
      NewbieNewbie
      1

        11:53 - Oct 09#3

         Thank you for posting that it could be just the thing to give inspiration to someone who needs it! Keep up the great work!

        1
        NewbieNewbie
        1

          13:01 - Oct 23#4

          Thank You Mofi for your answer. I've the same issue with UltraEdit. If I choose the radio button "Automatically convert to DOS format" in the configuration, a text file in Linux format is not converted automatically to DOS format. I've to change line feed character to \r\n, so it is very inconvenient way to change the format and the explanation in the configuration menu is misleading. Yes, I can write a macro for it, but I mean a function like this should have all text editors especially a program like UltraEdit.

          6,675585
          Grand MasterGrand Master
          6,675585

            19:17 - Oct 23#5

            @mirco69 I do not understand your problem with text files with line ending in Unix/Linux format.

            Do you just open a Unix file and close it without making any change on file contents by yourself and saving the file?

            UltraEdit does not silently convert the file from Unix to DOS/Windows just because of opening and closing it without any user modification and save.

            The temporary file created on open from the original file with just \n (Unix) or with just \r (Mac < OS X) as line ending is converted automatically to a text file with \r\n on selected radio option Automatically convert to DOS format in configuration. The configuration setting is ignored by UltraEdit if the file is opened without usage of a temporary file as UltraEdit 2024.1.0.36 does not modify the file on storage media just because of opening it in UltraEdit. That would not be possible at all if the read-only attribute is set or the user has just the permissions to read the file but not write it as typical for large log files or CSV files.

            Text selected in the file opened with usage of a temporary file and with automatic conversion from Unix/Mac to DOS/Windows is copied to clipboard with \r\n and UltraEdit macros and scripts can be coded to recognize just \r\n as line ending and write lines into the file with always \r\n which makes it easier writing such macros and scripts. The macro/script should be used in this case only on small/medium files which are opened with usage of a temporary file. If the macro/script should work also on large/huge files opened without usage of a temporary file for more efficiency, the macro/script must recognize first the line termination type and use it on making modifications on the file directly on original file on storage media.

            The check box option Save file as input format (Unix/Mac/DOS) is important on save of a file. I have checked this option as it results in saving the file with original line ending format as it was detected by UltraEdit on opening the file. I edit often Unix files in UltraEdit on Windows which should remain Unix files after my edits in UltraEdit. The conversion to \r\n is done in this case only temporarily by UltraEdit on the created temporary file. The original file overwritten on save with the contents of the temporary file has the line endings in original format. (Note: Some commands like the command sort result in a permanent change to DOS/Windows line ending of the temporary file despite this option.)
            The temporary file with DOS/Windows line terminators is saved with \r\n on having not checked this check box option. The temporary file simply replaces the original file without a conversion of the newline characters back to Unix or Mac as the originally opened file has had until the file save.

            Important is that the automatic conversion from just \n or just \r is only done temporarily on file open on copying the original file to temporary file for the temporary file. The file on storage media is not changed immediately and also not permanently as long as the user does not make at least one change on file contents and saves the file. The user must explicitly do a conversion from Unix/Mac to DOS with executing this command. As the temporary file is already with DOS/Windows line ending, the temporary file is not modified in this case, but UltraEdit marks the file now as modified by the user and ignores the original line ending format as determined on opening the file. The user explicitly converted the file now to DOS/Windows and UltraEdit honors that on next file save.

            The text editor UltraEdit cannot be used without a macro or script for converting Unix/Mac files to DOS/Windows files by just opening and next closing the Unix/Mac file without doing any modification on file contents and do a file save independent on the configuration settings. A file not modified by the user is not silently modified by UltraEdit on storage media.

            There is the configuration setting Status bar shows original line terminator format (on disk). This check box option should be checked to see in the status bar always displayed how the file is currently stored on disk (storage media) respectively would be saved by UltraEdit on next save after a file contents change by the user. I have checked this option too.

            Summary of my configuration:
            • Automatically convert to DOS format is selected.
            • Save file as input format (Unix/Mac/DOS) is checked.
            • Status bar shows original line terminator format (on disk) is checked.
            • Large file threshold (KB) is configured by me with 4096 (4 MiB).
            • Open file without temp file but prompt for each file is selected.
            I can see therefore Unix in the status bar on opening a text file with just \n as line termination on file having less than 4 MiB or if a open a file with 4 MiB or more and decide on prompt to open the file nevertheless with usage of a temporary file. The created temporary file displayed in the document window in UltraEdit is with DOS/Windows line endings. Even if I make changes and insert lines with \r\n, the status bar still shows Unix informing me that on next file save UltraEdit converts all \r\n in the temporary file back to just \n on overwriting the original file. The temporary file is kept with \r\n. Only the original file is written with just \n on save. The exception is running a sort on some lines or the entire file. In this case the status bar changes from Unix to DOS indicating that UltraEdit will save the file now as DOS/Windows file although in most cases not wanted by me. I must convert the sorted file back to Unix before next save. I reported this issue years ago. But there is no change up to now (UE v2024.1.0.36). There are a few other commands which change also the line ending type permanently to DOS/Windows although I prefer always saving a file in original format as long as not running manually the Unix/Mac to DOS conversion command (or using save as with selecting a different line ending format).
            Best regards from an UC/UE/UES for Windows user from Austria