Issues using tabs with mix of Latin and Japanese characters

Issues using tabs with mix of Latin and Japanese characters

3
NewbieNewbie
3

    Oct 12, 2020#1

    I'm using a text file to store some Japanese grammar and vocabulary for personal use but the tabs don't work quite as expected in UltraEdit:
    jap uedit.png (5.94KiB)
    I suspect that this is because the Japanese characters have a different (and probably proportional) spacing.
    Still, this way of using tabs works perfectly fine in Notepad++:
    jap np++.png (6.14KiB)
    Actually, even Windows Notepad handles the tabs correctly (albeit fixed to 8 characters which destroys my 4-space formatting here).
    So is there some setting in UltraEdit to get the tabs to work like in other text editors?

    As a side note, I wondered if there is any control over the font used for the Japanese Kana and Kanji. I guess windows uses some default font when the selected font doesn't support Kana and Kanji but is there a way in UltraEdit to influence this substitution?

    6,685587
    Grand MasterGrand Master
    6,685587

      Oct 12, 2020#2

      Is it possible for you to post the text with Japanese also as text formatted as code block (last but one item on toolbar above edit field)?

      Or even better, could you attach a text file with the Japanese characters?

      That would make it possible for me to look into this issue and test configurations before writing a real reply.

      It is not possible to configure which font is used by the used Windows library on font currently configured for documents view in normal text editing mode is not supporting the Japanese characters as far as I know.

      Which font is configured in Notepad++ and which one in UltraEdit?
      Best regards from an UC/UE/UES for Windows user from Austria

      3
      NewbieNewbie
      3

        Oct 12, 2020#3

        I tried that but it looks like the forum software replaces tabs by 8 spaces which kinda invalidates this approach. So I attached a text snippet:
        jap example.txt (220 Bytes)   0

        The fonts is exactly the same in both editors, namely Liberation Mono, size 11. I also tried other fonts (Consolas, Courier New), but the problem persists in UltraEdit. I actually doubt it's related to the (main) font. Indeed, I would think it's more related to the default font that Windows (?) selects for the UTF-8 characters not supported by the main font. Despite what I said before, I am now sure that the Japanese font is also monospaced, but it's a bit wider than the main font. So each character needs one or two pixels more in width which seems to shift the next tab a bit to the right.
        I'd think the following screenshot shows what happens here:
        jap example2.png (4.06KiB)

        6,685587
        Grand MasterGrand Master
        6,685587

          Oct 12, 2020#4

          Yes, browsers display horizontal tabs in an HTML file within a preformatted text block as a series of spaces according to HTML specification and most often copy them also as spaces to clipboard. For that reason uploading the file was definitely the best. Thank you for the text file which helped a lot on looking into this issue.

          I must admit that Notepad++ makes really fantastic job on display of this text. I have not installed Notepad++, but according to my tests with UltraEdit and Windows Notepad and your Notepad++ screenshot it is really doing a very good job on the text display with mixed font.

          Windows Notepad is not so good, but the text file looks quite good with the default font Lucida Console. There are just very small alignment differences with just one or two pixels even on increasing font size to 16 pt.

          UltraEdit v27.10.0.108 has the most problems with the display of this text. That is a fact. The display is better on using proportional fonts like Arial, Calibri (default font in MS Office), Lucida Sans Unicode (my favorite for this text), Segoe UI (Windows default font for graphical user interface since Windows Vista). But the alignment is never as good as with Windows Notepad and never perfect as with Notepad++.

          It looks like on using a non-proportional (fixed-width) font, UltraEdit does not determine the width of each character on a line individually and sum them all up. It looks like UltraEdit just assumes that all characters are displayed with same width on using a fixed-width font and if that is not true because of some characters are rendered by the used Windows library using a different font with proportional character width, the character drawing is not as good anymore as it can be expected by a user.

          The middle dot for the space character and the arrow for a horizontal tab are definitely drawn graphically by Notepad++. UltraEdit uses the character with code point value 0xB7 (hexadecimal, middle dot) for a normal space and 0xBB (hexadecimal, right-pointing double angle quotation mark) for a horizontal tab character. That makes drawing the text with showing spaces/tabs fast, but has the disadvantage that the used font and active code page on non-Unicode file determines the display of the replacement symbols. This can be seen on some fonts like Arial with the example text file on which an enabled or disabled showing of spaces/tabs can result in a horizontal tab needs to be removed or added to get a better text alignment. The Notepad++ solution with drawing a dot and an arrow is perhaps slower, but for displaying this text displayed with mixed font definitely the better solution.

          I played not only with various fonts and with enabling/disabling showing spaces/tabs, but also with the font quality setting at Advanced - Settings or Configuration - Editor display - Advanced. I have never understood what the font quality setting is for at all. But I could see with the example file that the current setting affects the text display more or less depending on used font. However, I could not find a combination of font and font quality setting for getting a perfect alignment as Notepad++ does according to attached screenshot.

          I can only recommend to use in UltraEdit a font on which the effect is not so bad as with font Liberation Mono or use for this task Notepad++ or Windows Notepad or Microsoft Word which make a much better job on display of this text file.

          It would be definitely good to report this display issue also to IDM support by email. It could be that the developers of UltraEdit can improve the code for text display for such text in a future version once again. There have been such improvements already in the past on much older versions of UltraEdit using different libraries for the graphic document interface.

          I know from an email reply by IDM support many years ago that the East Asian characters/symbols, especially the Japanese ones, are the most difficult to handle. It was necessary to use the fonts created by Microsoft with a name starting with @ like @Arial Unicode MS or @MS Mincho on Windows XP many years ago. I work never with text files containing Asian characters and so my knowledge is quite poor on such issues.
          Best regards from an UC/UE/UES for Windows user from Austria

          3
          NewbieNewbie
          3

            Oct 13, 2020#5

            Issue was confirmed and logged into the issue tracking system by the friendly support people.