I upgraded to UE v20.00 and finding it to be far less useful than version 19.xx when handling large files. Any attempts to cut and paste large chunks of data result in Out of Memory errors making more like a reader than an editor. Is anyone else experiencing this issue with UE v20.00?
UE v20.00 unable to handle 1.71 GB file in clipboard - Out of Memory
UE v20.00 unable to handle 1.71 GB file in clipboard - Out of Memory
- 1
I loaded a 4.3 GB file, selected about 1/3 of it, and attempted a Ctrl-C copy command. I got a message dialog stating, "Cannot allocate memory."
UE is a 32-bit program. Perhaps I need to try this in a smaller portion, and not select more than 2 GB, since a 32-bit program cannot access more than 2 GB in general. EDIT: Okay, tried a 750 MB copy, and it couldn't handle that either. Lame. I honestly don't know how previous versions handle this, as I've never tried it before.
I do wish though, that for these reasons, that IDM made a 64-bit version of UE and UES. This is the reason I don't edit large files, because I don't feel UE handles them as well as it should, and I have a 64-bit text editor that will. The other editor just doesn't have the feature set that I would want for an editor, but it was free, so I can use it in those rare cases. I'm more likely to use hex editing for large files, though.
If they do ever make a 64-bit version, I'd also like to see it be able to use a temp file only if a file will not fit in 1/3 the available memory (so as to leave room to copy the entire file on more than one clipboard.)
UE is a 32-bit program. Perhaps I need to try this in a smaller portion, and not select more than 2 GB, since a 32-bit program cannot access more than 2 GB in general. EDIT: Okay, tried a 750 MB copy, and it couldn't handle that either. Lame. I honestly don't know how previous versions handle this, as I've never tried it before.
I do wish though, that for these reasons, that IDM made a 64-bit version of UE and UES. This is the reason I don't edit large files, because I don't feel UE handles them as well as it should, and I have a 64-bit text editor that will. The other editor just doesn't have the feature set that I would want for an editor, but it was free, so I can use it in those rare cases. I'm more likely to use hex editing for large files, though.
If they do ever make a 64-bit version, I'd also like to see it be able to use a temp file only if a file will not fit in 1/3 the available memory (so as to leave room to copy the entire file on more than one clipboard.)
I'm quite sure that it will be a task for 12 to 18 months or even more for the IDM developers to rewrite all code lines for 64-bit. It is not really difficult to code an application nowadays from beginning for compilation as 32-bit and 64-bit application. But making a nearly 20 years old software really working as 64-bit application is a nightmare task for every programmer. Every pointer must be changed in type. That often requires also changing of structures containing pointers or integers holding memory addresses. The change of structures requires often more changes on code, and, and, and ...
I have once converted a very small code never designed for 64-bit from 32-bit to 64-bit. As I started, I thought that I could finish it in 3 days. Than I needed more than 2 weeks. Which programmer has in the past (5 and more years ago) ever take care of type size_t in C/C++ and used this type instead of simply int or unsigned int. Well, the C library used always size_t, but I must confess that I have never done that for my C/C++ projects. I just started in the last 3 years for new projects to take care of bit width of variables and structure elements, and make sure that the code can be compiled also for other processors or controllers from 8-bit to 64-bit.
The author of my favorite file manager Total Commander worked more than 2 years on this great tool to make it working as 64-bit application in several steps.
Another problem is all the additional open source code included in UltraEdit, or loaded from a DLL, or executed from within UltraEdit. All this third-party source code needs to be x64 compatible, too. Total Commander is again a good example as there are lots of plugins for TC. With the release of Total Commander 8.00 with x64 support internally and also for the plugins, all the plugin authors were quickly forced by the user community to unpack their plugin source files not touched in the last years, get a 64-bit compiler, recode, compile and test their plugins for x64.
I'm quite sure that IDM must grasp the nettle somewhere in the next 3 years and start making UltraEdit compilable (can be quite easily achieved) and working (that's the really time consuming task) as 64-bit application although approximately 99% of all UltraEdit users never really need a 64-bit editor as using it only for editing small text files. For my tasks I don't need a 64-bit UltraEdit / UEStudio.
Okay, after sharing my opinion on the need of a 64-bit UltraEdit, back to task on how to copy large, very large chunks of text from one file into another.
UltraEdit has two functions for this task already included since I'm using UltraEdit which is since UE v8.00:
The blog article WOW64: Memory mapping of 32bit apps running on a 64bit Windows was very interesting to read.
I have once converted a very small code never designed for 64-bit from 32-bit to 64-bit. As I started, I thought that I could finish it in 3 days. Than I needed more than 2 weeks. Which programmer has in the past (5 and more years ago) ever take care of type size_t in C/C++ and used this type instead of simply int or unsigned int. Well, the C library used always size_t, but I must confess that I have never done that for my C/C++ projects. I just started in the last 3 years for new projects to take care of bit width of variables and structure elements, and make sure that the code can be compiled also for other processors or controllers from 8-bit to 64-bit.
The author of my favorite file manager Total Commander worked more than 2 years on this great tool to make it working as 64-bit application in several steps.
Another problem is all the additional open source code included in UltraEdit, or loaded from a DLL, or executed from within UltraEdit. All this third-party source code needs to be x64 compatible, too. Total Commander is again a good example as there are lots of plugins for TC. With the release of Total Commander 8.00 with x64 support internally and also for the plugins, all the plugin authors were quickly forced by the user community to unpack their plugin source files not touched in the last years, get a 64-bit compiler, recode, compile and test their plugins for x64.
I'm quite sure that IDM must grasp the nettle somewhere in the next 3 years and start making UltraEdit compilable (can be quite easily achieved) and working (that's the really time consuming task) as 64-bit application although approximately 99% of all UltraEdit users never really need a 64-bit editor as using it only for editing small text files. For my tasks I don't need a 64-bit UltraEdit / UEStudio.
Okay, after sharing my opinion on the need of a 64-bit UltraEdit, back to task on how to copy large, very large chunks of text from one file into another.
UltraEdit has two functions for this task already included since I'm using UltraEdit which is since UE v8.00:
- File - Save Selection As which can be used instead of Ctrl+C, Ctrl+N, Ctrl+V and F12.
- Insert - File to insert content of another file at current caret position into active file.
The blog article WOW64: Memory mapping of 32bit apps running on a 64bit Windows was very interesting to read.
I can tell you right now, I have a personal file which I keep for couple of years now its been through several versions of Ultraedit and its size is about over 30 mb.
It is saved in UTF-8 as I like to add weird character symbols into it.
It is now looking corrupted whenever I tried searching some words on it. It keep showing weirdly a chunk of kanji symbols. But if you continue to pull the scroll bar it sometimes get through this weird effect. But there were 2 times it showed a popup saying out of memory which is ridiculous since I checked RAM usage for ultraedit shows around 50+ MB.
I have since tried to solve it by:
1. Trim the file into half - showing same symptoms.
2. Restart the PC - same symptoms.
3. Uninstall v20 reinstall v19 issue is solved.
I guess this means there is either a memory leak or something big changed with this version.
I am running Windows 8 Pro with Media Center, 10GB RAM if that helps.
It is saved in UTF-8 as I like to add weird character symbols into it.
It is now looking corrupted whenever I tried searching some words on it. It keep showing weirdly a chunk of kanji symbols. But if you continue to pull the scroll bar it sometimes get through this weird effect. But there were 2 times it showed a popup saying out of memory which is ridiculous since I checked RAM usage for ultraedit shows around 50+ MB.
I have since tried to solve it by:
1. Trim the file into half - showing same symptoms.
2. Restart the PC - same symptoms.
3. Uninstall v20 reinstall v19 issue is solved.
I guess this means there is either a memory leak or something big changed with this version.
I am running Windows 8 Pro with Media Center, 10GB RAM if that helps.
Satan, please report this issue to IDM support by email.
The file handling for large files is completely changed in UE v20.00 in comparison to all previous versions and nobody can really expect that this works from the beginning for all types of large text files. There is surely fine tuning necessary for special types of large text files by the developers of IDM.
UTF-8 encoding is one of the worst encoding for an editor. As in memory supported by libraries there are only two types of string storage formats - NULL terminated strings with 1 byte per character or Unicode strings with wide characters (16 or 32-bit per character depending on library and compiler), an editor (and also all other applications) has to know from the beginning if a file is an ASCII/ANSI file or a Unicode encoded file. And the Unicode encoding must be also known from the beginning.
As all characters with a code value < 128 (decimal) are stored with UTF-8 encoding exactly as in an ASCII/ANSI file, UltraEdit has several methods to detect quickly if a file is an ASCII/ANSI file or encoded in UTF-8. UltraEdit recognizes:
If there is no real reason to store large Unicode text files with UTF-8 encoding like uploading the file to an HTML/FTP server for download by others, it is better to use UTF-16 encoding for the large Unicode text file as this speeds up loading and saving the file dramatically as no conversion routine must be executed for every character.
BTW: There is a setting which can be added manually to uedit32.ini to tell UltraEdit to interpret any file not definitely encoded in UTF-16 LE or UTF-16 BE as UTF-8 encoded file. This setting is for people who uses only Unicode text editing and store the text files always with UTF-8 encoding for minimizing size of file on storage media.
The file handling for large files is completely changed in UE v20.00 in comparison to all previous versions and nobody can really expect that this works from the beginning for all types of large text files. There is surely fine tuning necessary for special types of large text files by the developers of IDM.
UTF-8 encoding is one of the worst encoding for an editor. As in memory supported by libraries there are only two types of string storage formats - NULL terminated strings with 1 byte per character or Unicode strings with wide characters (16 or 32-bit per character depending on library and compiler), an editor (and also all other applications) has to know from the beginning if a file is an ASCII/ANSI file or a Unicode encoded file. And the Unicode encoding must be also known from the beginning.
As all characters with a code value < 128 (decimal) are stored with UTF-8 encoding exactly as in an ASCII/ANSI file, UltraEdit has several methods to detect quickly if a file is an ASCII/ANSI file or encoded in UTF-8. UltraEdit recognizes:
- a UTF-8 byte order mark (BOM) at beginning of a file,
- a UTF-8 character set declaration (HTML, XHTML) respectively a UTF-8 encoding declaration (XML) at top of the file according to HTML, XHTML and XML standards,
- UTF-8 byte sequences in the first 64 KB of the file on using UltraEdit for Windows < v24.10 or UEStudio < v17.10.
If there is no real reason to store large Unicode text files with UTF-8 encoding like uploading the file to an HTML/FTP server for download by others, it is better to use UTF-16 encoding for the large Unicode text file as this speeds up loading and saving the file dramatically as no conversion routine must be executed for every character.
BTW: There is a setting which can be added manually to uedit32.ini to tell UltraEdit to interpret any file not definitely encoded in UTF-16 LE or UTF-16 BE as UTF-8 encoded file. This setting is for people who uses only Unicode text editing and store the text files always with UTF-8 encoding for minimizing size of file on storage media.
Mofi,
I would tentatively say my issue with memory leak is resolved with release of v20.00.0.1045.
May I ask if what you meant previously is that it would serve me better to just use UTF-16 instead of UTF-8. What is it I must do in uedit32.ini to make changes? I thought I would change this in "Saved As" part? Do I use UTF 16 or UTF 16 NO BOM.
I would tentatively say my issue with memory leak is resolved with release of v20.00.0.1045.
May I ask if what you meant previously is that it would serve me better to just use UTF-16 instead of UTF-8. What is it I must do in uedit32.ini to make changes? I thought I would change this in "Saved As" part? Do I use UTF 16 or UTF 16 NO BOM.
You don't need to change something in uedit32.ini. Use File - Open with option Open As (Windows XP) or Encoding (Windows Vista and later Windows versions) set to UTF-8 to open your larger UTF-8 encoded file correct. Than use Save As and select UTF-16. It is better to save Unicode files with BOM as done with UTF-16 as than the editor has to read in just the first two bytes of a text file to know the used encoding.Satan wrote:What is it I must do in uedit32.ini to make changes? I thought I would change this in "Saved As" part? Do I use UTF 16 or UTF 16 NO BOM.
If you want to use in general UTF-16 LE for new text files, you can select Create new files as UTF-16 at Advanced - Configuration - Editor - New File Creation.
But please take into account that many interpreters and compilers do not support UTF-16 encoded files. For example built-in JavaScript interpreter in UltraEdit requires single byte encoded files and therefore only ASCII/ANSI files (respectively UTF-8 encoded files without BOM) can be used for UltraEdit *.js script files.
