What kind of checks does syntax highlighting do?

rseiler · Dec 15, 2023#12023-12-15T20:58+00:00

I uncovered a seeming deficiency in the way that UltraEdit does that, at least for the .json I was using.

The .json wasn't working as it should, so I looked at it in UE (latest version in Windows). It seemed perfectly fine based on what I knew should be in the file, color coding included.

When I looked at it in Notepad++ however, the text looked identical but there was one fundamental difference: no color-coding. I quicky looked at another .json in Notepad++ and found that it was color-coded, so it wasn't simply a case of that feature not being enabled. What this told me: something was fundamentally wrong with my problem json, and Notepad++ knew about it while UE hadn't a clue.

The problem turned out to be invisible until I enabled showing end-of-line characters: the json had only LFs at the end of each line rather than a CR and LF at the end of each (yes, yet the files did look identical).

Which brings me to my question: was this a fluke or the tip of the iceberg when it comes to this feature? How much checking for basic validity does UE do before it decides to syntax-highlight a file? And I don't mean just for json, for anything that it supports.

Mofi · Dec 16, 2023#22023-12-16T12:25+00:00

UltraEdit does not check the validity of any file which is syntax highlighted according to the definition in the applied wordfile. It just syntax highlights the file. A JSON file can have just a line-feed as line endings as that is the default for all text files on Unix/Linux and Mac since Mac OS X or carriage return + line-feed as typical for all text files on DOS/Windows. UltraEdit detects automatically the line ending type and displays it in the status bar at bottom of the main application window.

If the program reading in the JSON file supports only DOS/Windows line ending type, it is not good coded for processing a JSON file. A JSON file with just LF as line termination is as valid as a JSON file with CR+LF as line termination. A JSON file without any line ending is also valid. Breaking up the data into multiple lines is just for making it easier for human beings reading the data. Read the chapter Standards on Wikipedia page about JSON and read RFC 8259 if you are interested in the standard for JSON.

UltraEdit has the command Compress JSON which removes all newline characters (all carriage returns and line-feeds) and all indenting spaces and other whitespaces just for data alignment which any JSON reading program should ignore too. The UltraEdit command Reformat JSON reformats a JSON file to a human readable format with lines and indents.

rseiler · Dec 16, 2023#32023-12-16T20:03+00:00

@Mofi That's great information, thanks. It didn't occur to me to lookup whether either is permitted for the format, so I'll check into why that was a problem.

On the "line ending type and displays it in the status bar at bottom of the main application window," while I do see a lot there, including DOS vs Unix designations for the respective files, I don't see any mention of LF/CR. Is it an option I'm missing? I did try View/Line Endings, but that's about display of line endings in the file.

Mofi · Dec 16, 2023#42023-12-16T21:31+00:00

DOS in the status bar means the active file has (mainly) carriage return + line-feed as line ending. This is typical for MS-DOS and Windows text files.

Unix in the status bar means the active file has (mainly) just line-feed as line ending. This is typical for Unix/Linux text files and since Max OS X also for text files on Apple Mac computers.

MAC in the status bar means the active file has (mainly) just carriage return as line ending. This is typical for Mac text files up to Mac OS-9.

Please read the Wikipedia article about Newline with a table in chapter Representation.