Can't paste some languages and symbols from Word into UltraEdit

cyberdiva · Nov 25, 2013#12013-11-25T20:21+00:00

I am a longtime user of UltraEdit version 14.00+ 1, which I am currently using on Windows 7 64-bit. Recently, I tried to copy some words in Russian, Hebrew, and Chinese into UltraEdit. My clipboard manager had no trouble showing those words, but when I pasted them into UltraEdit, all I saw were question marks rather than the foreign characters. I had a similar problem when I tried to paste symbols such as ♫ or ≠. On my UltraEdit main toolbar, I have "Convert to Unicode" and "Convert from Unicode," but neither of those had any effect on the question marks. What do I need to do to get UltraEdit to reproduce these other languages and symbols?

I should add that before I posted this message, I checked the Help file and did a quick search of the forum for "unicode support," but I didn't find an answer to my question. I'd be most grateful for some help.

Mofi · Nov 26, 2013#22013-11-26T06:06+00:00

The default setting is that new files are ASCII/ANSI files using the code page as configured in the Windows regional and language settings. An ANSI file stores characters with 1 byte per character and therefore supports only 255 characters.

So before you can paste Unicode characters which need 2 bytes per character into a text file, you have to convert an ANSI file to Unicode. You need File - Conversions - ASCII to Unicode or ASCII to UTF-8 depending on the encoding which should be used on saving the file.

Then you can paste the Unicode text into the Unicode file as now there is no conversion anymore on paste to the ANSI code page used before for the file. It can nevertheless happen that some characters are not displayed right. In this case you have to open View - Set Font and choose a font which supports all Unicode characters, or at least supports those Unicode characters present now in the file. Courier New supports many Unicode characters, but not all. Proportional font Arial Unicode MS supports all Unicode characters, but is unfortunately not a fixed width font.

cyberdiva · Nov 26, 2013#32013-11-26T19:24+00:00

Thanks, Mofi, for your prompt and informative response. Alas, I could not get any combination to work. I tried converting the file from ASCII to Unicode and from ASCII to UTF-8, and I tried these possibilities with Courier New and with something called Lingoes Unicode, which was the only other font listed in my copy of UltraEdit that seemed to suggest it would be a good possibility. But nothing made any difference. Have I overlooked something?

Mofi · Nov 27, 2013#42013-11-27T06:58+00:00

As you can see on the attached image, I have had no problem to convert a new ASCII file first to Unicode, see U-DOS in the status bar, and paste next the 6 characters copied before from the browser window into the new file in UltraEdit using Courier New as display font.

I have no further idea on what could be not correct configured on your system or what might go wrong on paste.

Please note that Courier New does not support all characters from the Unicode table. It can be seen very easily by looking on the file size of the font files in C:\Windows\Fonts which font supports all Unicode characters. The fonts supporting all Unicode characters have usually a file size larger than 20 MB. Most fonts are smaller than 1 MB which is an indication for just supporting some code pages or a subset of the Unicode character table.

cyberdiva · Nov 28, 2013#52013-11-28T01:22+00:00

Many thanks, Mofi, for your most recent message. After reading it, I tried again and discovered that if I tried to paste just the musical note and the "unequal to" symbols, I was able to do so after changing from ASCII to Unicode and using Courier New, as you did. But if I try to paste something that includes words in Russian and Hebrew as well as the musical note and the unequal to symbols, everything is shown as question marks, even the musical note and the unequal to sign. Since all my earlier attempts included the Russian and Hebrew words as well as the symbols, and since none of them appeared correctly, I assumed that I had totally failed. Here's a screenshot, reduced in size, that shows my success in pasting just the two symbols that you, too, had succeeded in pasting, and then underneath the symbols my attempt to paste a statement that included words in Russian and Hebrew, followed by the two symbols. As you can see, they're all now merely question marks.

By the way, I went to C:\Windows\Fonts to look at the file sizes of the different fonts, but although my listing was set to show details, to my surprise there was no indication of file size. I looked first in my usual file manager, Directory Opus, and then in Windows Explorer. Neither one showed the file sizes of the fonts.

Mofi · Nov 28, 2013#62013-11-28T06:30+00:00

In which applications do you have copied the text to the clipboard?

This is important to know because in word processing applications like MS Word it is possible that not the complete text uses the same encoding. In MS Word it is possible that some characters or words are stored and displayed with a different font and/or using a different ANSI code page. Such a text is copied to the clipboard with those hidden formatting tags in RTF or HTML format. On paste into a pure text file in UltraEdit, the pure text content must be pasted. But the clipboard containing the pure text version can contain the characters only in one encoding. The result is what you see. Those characters stored in MS Word with a different code page and/or font as the normal text are replaced by question marks. That is done by the clipboard itself and not by UltraEdit.

Please note that there are multiple clipboards, not just one as most users think, see Standard Clipboard Formats. With using in UltraEdit Edit - Paste Special to paste the clipboard content as RTF or HTML code, you can perhaps see the hidden formatting tags used to hold the words in another language and the symbols in a different code page/font than normal text arround.

What happens if you select only the Russian word and copy and paste it into UltraEdit?

I used Total Commander to view the size of the font files. But you can also use Windows Explorer. Open %windir%\Fonts in Windows Explorer, above the icons of the fonts there is on right side an arrow symbol. Click on this arrow symbol and click on Details. Now you get the font files listed with more details, but still without seeing the file size of each font file. Right click on the row with the column headers and activate Size. Now you have one more column which displays the size of each font file.

cyberdiva · Nov 30, 2013#72013-11-30T15:33+00:00

Many thanks, Mofi, for your very helpful reply. I think you've identified the problem. I had copied the text from a .doc file. When I went to a website and copied similar material, UltraEdit was able to show all the symbols. I tried first using the font Lingoes Unicode, and when that worked, I tried using Courier New. Both worked fine, as did Arial. So I guess the way Word codes the symbols is what caused the problem for UltraEdit. I should add that when I copied just the Russian word, it was all question marks when copied from the .doc file but fine when copied from the web. Again, MANY THANKS!!

Added note: Thanks also for your instructions about how to see the size of the font files. I was able to use your instructions not just for Windows Explorer but also for Directory Opus.