Function list doesn't show functions/procedures with multiple lines of arguments for Delphi files

JeanPhi007 · Dec 14, 2020#12020-12-14T14:41+00:00

This declaration will be shown in the Function List pane for a Delphi file:

procedure ProcessSingleMsg(Edi_ID : string; NormalizeEdiIn, NormalizeEdiOut: Boolean; OutFormat : TOutputFormat);

But not this one:

Code: Select all

function ProcessMsg2(EdiIn, Edi_ID: string;
                     var StrLog, EdiOut : string;
                     OutFormat: TOutputFormat;
                     NormalizeEdiIn, NormalizeEdiOut : Boolean): Integer;

It appears that functions or procedures with arguments declared on multiple lines are not shown.

Mofi · Dec 14, 2020#22020-12-14T16:24+00:00

Open in UEStudio Advanced - Settings or Configuration - IDE - IntelliTips - Function tips.

Is the setting Use function tips data (if available) for function list checked or unchecked?

An enhancement request should be sent by email to IDM support on this setting is checked as in this case the symbol parser of UEStudio needs to be enhanced to support Delphi functions and procedures spanning over multiple lines. It is perhaps also possible to enhance ues_ctags.exe (that's the used symbol parser of UEStudio) without changing the source code of it by a developer of UEStudio as it is possible to use a configuration file containing arguments like regular expressions to use for finding symbols for a specific language. But I would need more information as written below to help enhancing symbol parser of UEStudio without modification of its code if that is possible at all.

Otherwise the regular expression strings defined in the wordfile for syntax highlighting Delphi source code files must be enhanced which can be done by every user. In the opened configuration window browse to Editor display - Syntax highlighting, select Delphi for Installed wordfile if not already pre-selected, click on button Open, and close the configuration window.

Copy and paste the block from first line with /TGBegin to last line with /TGEnd into your next post so that we know which regular expressions to enhance. The currently latest user-submitted Delphi wordfile contains:

Code: Select all

/TGBegin "Constructor"
/TGFindStr = "^constructor\s+([\w\.]+)\s*(?:\([^\)]*\))?[^;]*;[\s\S]*?begin"
/TGEnd
/TGBegin "Destructor"
/TGFindStr = "^destructor\s+([\w\.]+)\s*(?:\([^\)]*\))?[^;]*;[\s\S]*?begin"
/TGEnd
/TGBegin "Functions / Procedures"
/TGFindStr = "^(?:procedure|function)\s+([\w\.]+)\s*(?:\([^\)]*\))?[^;]*;[\s\S]*?begin"
/TGEnd

The regular expression search string for Functions / Procedures in this wordfile from list of user-submitted wordfiles for UltraEdit/UEStudio results in just displaying the function and procedure names without their parameters. Let us know if you want to see the function and procedure parameters in a flat list right to the function name or in a hierarchical (tree-style) list below each function / procedure name.

An example code block with Delphi source code with various functions and procedures and a link to a website explaining the Delphi syntax would be also helpful for us to help you to define the Perl regular expressions which result in listing the functions and procedures in Function List view as you would like them. The better the example source code file is the better will be the expressions for listing the functions and procedures in the Function List view. For one of several examples on how the regular expressions for function list can be enhanced with help of other users like me see the topic How to get JavaScript function list with anonymous and arrow functions also listed?

JeanPhi007 · Dec 14, 2020#32020-12-14T17:15+00:00

Hi Mofi,

thanks for your quick reply.
first, my delphi wordfile doesn't contain the block you send me. But even if I copy it in my wordfile, the function ProcessMsg2 doesn't appear.
then, it would be nice if the parameters are displayed in a hierarchical list but a flat list will be good enough

I send my code as attachment. except for the indentations, it should be correctly formatted as I passed it through the Cn Pack code formatter.

Mofi · Dec 15, 2020#42020-12-15T13:32+00:00

Thank you for the example Delphi file. That was a big help to find suitable regular expressions without any Delphi syntax knowledge.

Delphi code is really not easy to parse with regular expression finds because of the keyword var can exist multiple times inside a function definition and defines also the beginning of the function variable definition block, and ; can exist multiple times inside a function definition, but marks also end of the function definition, and it is possible that a function has no function variable definition block and has no round brackets to define an empty list of parameters. The function GetTimeStamp in the example file was a real nightmare on finding suitable regular expressions.

The block to find with Perl regular expression engine strings to be listed hierarchical in function list is:

Code: Select all

/TGBegin "Constructors"
/TGFindStr = "^constructor\s+([\w.]+)\s*(?:\([^\)]*\))?[^;]*;[\s\S]*?begin"
/TGBegin "Parameters"
/TGFindStr = "(\w+(?:[\t ]*,[\t ]+\w+)*)[\t ]*(:)[\t ]*(\w+)"
/TGFindBStart = "\("
/TGFindBEnd = "\)"
/TGEnd
/TGBegin "Initializations"
/TGFindStr = "(\w+)[\t ]*(:=)[\t ]*(\w+);"
/TGFindBStart = "begin"
/TGFindBEnd = "end"
/TGEnd
/TGEnd
/TGBegin "Destructors"
/TGFindStr = "^destructor\s+([\w.]+)\s*(?:\([^\)]*\))?[^;]*;[\s\S]*?begin"
/TGEnd
/TGBegin "Functions / Procedures"
/TGFindStr = "^(?:procedure|function)\s+([\w.]+)\s*(?:\([^\)]*\))?(?:[\t ]*(:)[\t ]*(\w+))?[^;]*;[\s\S]*?(?:begin|stdcall)"
/TGBegin "Parameters"
/TGFindStr = "((?:const[\t ]+)?\w+(?:[\t ]*,[\t ]*(?:const[\t ]+)?\w+)*)[\t ]*(:)[\t ]*(\w+)(?:[\t ]*(=)[\t ]*([^;]+)(?:;|$))?"
/TGFindBStart = "^(?:procedure|function)\s+[\w.]+\s*[(:;]"
/TGFindBEnd = "^begin|\)"
/TGEnd
/TGBegin "Variables"
/TGFindStr = "(\w+(?:[\t ]*,[\t ]+\w+)*)[\t ]*(:)[\t ]*(\w+);"
/TGFindBStart = "^(?:procedure|function)\s+[\w.]+\s*(?:\([^\)]*\))?(?:[\s\S](?!\b(?:begin|var|stdcall)))+"
/TGFindBEnd = "begin|stdcall"
/TGEnd
/TGEnd

The used syntax highlighting wordfile must contain also the line:

Code: Select all

/Regexp Type = Perl

Some additional information about these regular expression:

I don't know if using \w+ is really 100% correct for Delphi. It matches one or more word character according to Unicode. In C/C++ that would not be valid as the first character of a function or variable name must be a letter or an underscore and cannot be a digit. The digits 0 to 9 are also word characters. Further I don't know if non-ASCII word characters like äöüÄÖÜß and many others are valid in Delphi symbol names matched also by \w+.
I don't know how a function declaration like Edi2Xml and the next six function declarations differ from function definitions. I used in the regular expressions above the keyword stdcall to handle the function declarations different to function definitions.
The original expressions interpreted everything from end of line with function Edi2Xml to begin of constructor TError.Create as function definition which of course is not right.
It is possible to use multiple marking groups with (...) inside a regular expression and UltraEdit/UEStudio concatenates all strings found by the expressions inside the marking groups with a single space between them. I used that feature to get listed in function list the functions / procedures with their return type in a standardized format independent on how many spaces/tabs are in the source code file. The same feature is used to get the parameters with their types displayed standardized in the function list.
The parameters are displayed with their initialization if there is one at all for a parameter.
The keyword const is displayed in the function list.
The keyword var is intentionally not listed in the function list as I do not really know the meaning of this keyword in the list of function parameters. It is of course possible to get the parameters with the keyword var listed in function list if that is wanted by you.
I added also a block to find the variables definitions of a function or procedure. That was very tricky as such a block is optional and there can be really anything between the line starting with function or procedure and the line starting with begin on taking comments into account. It was not possible to simple use ^var as expression to find beginning of a function variables definition block as that results in a fall through to such a block in one of the functions / procedures below the current function / procedure if the current function / procedure does not have such a block at all.
It was the first time that I used a regular expression for finding the start of a block which matches multiple lines. I have never seen that on any wordfile before. This is necessary for Delphi, but causes an issue: The function variables are found and listed correct in the function list, but on double clicking on a variable definition in the function list the caret is positioned on wrong line. I could see easily the cause. UltraEdit and UEStudio do not take into account that the block starting string spans over multiple lines and each line ending inside the found block starting string must be counted too. I will report this issue by email to IDM support to get a related enhancement implemented in a future version of UE/UES.
I have no idea how to detect correct the end of a function definition which contains ) inside an initialization string or a comment. That means something like below is not correct parsed:
Code: Select all
```
function Brackets(RoundBrackets:string = '()'; CloseBracket : char=')'):string;
begin
end
```

Please let me know anything which is not correct working with these regular expressions and I will try to fix it. Please let me also know if you want something displayed different in the function list and I will try to change the regular expressions accordingly. Furthermore, I would like also comments and suggestions on how to improve the expressions for excluding symbols which are not valid according to Delphi syntax.

At the end I would like to submit the improved wordfile for Delphi to the IDM wordfiles repository for general usage by Delphi programmers using UltraEdit or UEStudio.

JeanPhi007 · Dec 15, 2020#52020-12-15T16:16+00:00

oh, by the way could tell me exactly where to put the wordfiles. For example I have seven (7!) css.uew files
C:\Program Files\IDM Computer Solutions\UEStudio\wordfiles\css.uew
C:\Program Files\IDM Computer Solutions\UEStudio\wordfiles\legacy\css.uew
C:\Program Files\IDM Computer Solutions\UltraCompare\wordfiles\css.uew
C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\volatile\wordfiles Saved [11.24.20]\css.uew
C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\volatile\wordfiles Saved [11.24.20]\legacy\css.uew
C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\wordfiles\css.uew
C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\wordfiles\legacy\css.uew

but I don't know which one is really used by UEdit.

for Delphi I've one in AppData\Roaming\IDMComp\UEStudio\wordfiles\ and another, different, one C:\Program Files\IDM Computer Solutions\UEStudio\wordfiles\.
Have I keep both, or, if not which need to be corrected?

Dec 15, 2020#62020-12-15T16:24+00:00

forget my previous question, my UEStudio is configured to put uew files in AppData\Roaming\IDMComp\UEStudio\wordfiles

Dec 15, 2020#72020-12-15T16:56+00:00

yessss, it works

I send you my delphi .uew file but sorry, I'm far to be a specialist of regular expressions, so I won't be very helpful to improve yours.
Anyway, I thank you very much for your great and quick help.

Mofi · Dec 15, 2020#82020-12-15T17:25+00:00

The wordfiles directory used by UEStudio is the directory displayed in the syntax highlighting configuration dialog opened with Advanced - Settings or Configuration - Editor display - Syntax highlighting. The default wordfiles directory is %APPDATA%\IDMComp\UEStudio\wordfiles which expands to C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\wordfiles on your computer.

The legacy wordfiles in C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\wordfiles\legacy are more or less the same as the wordfiles in the parent directory with the exception of using the legacy UltraEdit regular expression engine to find strings for the function list. I recommend to delete the directory C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\wordfiles\legacy as not used by UEStudio. A user has to select this directory in the syntax highlighting configuration dialog if the user wants to explicitly use the wordfiles in this directory which are in general no longer updated.

C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\volatile\wordfiles Saved [11.24.20] is a backup of C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\wordfiles created by UEStudio on 2020-11-24. You can delete completely the directory wordfiles Saved [11.24.20] if the backup is not needed by you.

The installer of UEStudio installs the standard wordfiles into the directory %ProgramFiles%\IDM Computer Solutions\UEStudio\wordfiles and the standard legacy wordfiles into the subdirectory legacy. These wordfiles are always overwritten on running the installer of UEStudio. The wordfiles in this directory are write-protected by Windows default for a standard user account with enabled user account control as all files in %ProgramFiles%.

UEStudio copies the entire directory %ProgramFiles%\IDM Computer Solutions\UEStudio\wordfiles to %APPDATA%\IDMComp\UEStudio if there is no directory wordfiles in the application data directory of UEStudio for the current user. This is done usually only once on first run after an installation of UEStudio. The wordfiles in %APPDATA%\IDMComp\UEStudio\wordfiles are fully customizable by every user of UEStudio. There can be added wordfiles like delphi.uew. Not needed wordfiles can be deleted to reduce the startup time of UEStudio and customizations can be applied to the wordfiles really loaded by UEStudio. The wordfiles in the user account related application data directory of UEStudio are never updated automatically by UEStudio, even if the installer installed newer versions of the standard wordfiles into %ProgramFiles%\IDM Computer Solutions\UEStudio\wordfiles as it is impossible to find out by UEStudio which differences between the wordfiles in these two directories are caused by customizations of the user or by IDM Computer Solutions, Inc. (after a verified user submit).

UltraCompare handles the wordfiles like UEStudio with one difference: UltraCompare uses by default the wordfiles in application data directory of UEStudio or UltraEdit if one of these two applications is also installed and %APPDATA%\IDMComp\UEStudio\wordfiles or %APPDATA%\IDMComp\UltraEdit\wordfiles exists because of these set of wordfiles is most likely what the user prefers to use.

The UltraCompare configuration dialog Home - Settings or Options - Configuration - Display - Text - Syntax highlighting shows from which directory the syntax highlighting wordfiles are loaded by UltraCompare and the user has the freedom to set whatever directory is wanted by the user like in the Syntax highlighting configuration dialog of UEStudio and UltraEdit.

My recommendations:

Store customized wordfiles like delphi.uew in C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\wordfiles.
Delete in the directory C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\wordfiles those *.uew files which you do not use to reduce the file system accesses during startup of UEStudio. Note: The program files folder of UEStudio contains the standard wordfiles in newest installed version too. So don't worry about deleting a file in this directory which you perhaps need in some months or years, but definitely not yet.
Delete the directory C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\wordfiles\legacy to get a bit more free storage space and reduce the confusion on which wordfiles are used by UEStudio.
Delete the directory C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\volatile\wordfiles Saved [11.24.20] for the same reasons as above on backup no longer needed.
Configure in UltraCompare to use also C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\wordfiles as wordfiles directory like UEStudio so that you have to do a customization on a wordfile only once in case of this directory is not already set as wordfiles directory to use by UltraCompare too.
Delete all files inside the directory C:\Users\leloupjp\AppData\Roaming\IDMComp\UltraCompare\wordfiles, but not the wordfiles directory itself, to get a bit more free storage space and reduce the confusion on which wordfiles are used by UltraCompare.

So you should have finally:

C:\Program Files\IDM Computer Solutions\UEStudio\wordfiles managed by installer of UEStudio.
C:\Program Files\IDM Computer Solutions\UEStudio\wordfiles\legacy managed also by installer of UEStudio.
C:\Program Files\IDM Computer Solutions\UltraCompare\wordfiles managed by installer of UltraCompare.
C:\Users\leloupjp\AppData\Roaming\IDMComp\UEStudio\wordfiles managed by you and used by UEStudio and UltraCompare.
C:\Users\leloupjp\AppData\Roaming\IDMComp\UltraCompare\wordfiles which is an empty folder as UltraCompare uses the folder above.

Dec 15, 2020#92020-12-15T17:34+00:00

I do not expect that you modify the regular expressions yourself. I have the expert knowledge in Perl regular expression syntax and in syntax highlighting of UltraEdit and UEStudio, but I don't have knowledge in Delphi syntax. So if you with your knowledge in Delphi syntax can describe what should be improved for the function list or point me to a website which describes the Delphi syntax in full details, I will try to do it by modifying the regular expressions.

Dec 19, 2020#102020-12-19T13:47+00:00

I looked on your wordfile delphi.uew and improved it further after reading Fundamental Syntactic Elements (Delphi) and taking into account also contents of currently latest user-submitted Delphi wordfile.

There are two wordfiles in the attached ZIP file:

delphi.uew is case-sensitive as your uploaded wordfile although Delphi is not case-sensitive.
delphi_nocase.uew is not case-sensitive like Delphi and the currently latest user-submitted Delphi wordfile.

The two wordfiles are identical with the exception of the order of the words containing letters in upper case. It is up to every Delphi programmer which one to use for own Delphi source code files.

The improvements are as follows:

All color and font style settings are removed as not longer stored or read from the wordfile since UltraEdit v20.00 and UEStudio v14.00.
The Perl regular expressions are improved further to ignore functions, procedures, parameters and variables starting invalid with a digit. A function definition like:
Code: Select all
```
function Brackets(RoundBrackets:string = '()'; CloseBracket : char=')'):string;
```
is processed now also correct by the regular expressions, but they still cannot process correct all functions or procedures with a closing parenthesis (round bracket) in a string or comment inside the parameters list of a function or procedure.
The dollar sign $ is added to the list of word delimiters. That does not have a negative effect on syntax highlighting hexadecimal values as the first character of a word or substring definition in wordfile can be also a word delimiter.
The indent/unindent and open/close fold strings are sorted alphabetically and duplicate close fold strings are removed as fold strings must not be defined in pairs like the brace strings. (These improvements have no effect on syntax highlighting.)
Function name VarARrayHighBound is corrected to VarArrayHighBound because I am quite sure the upper case R was a typing mistake.
The dollar sign $ is added to color group for special symbols to highlight this character if not being the beginning of a hexadecimal value.
Color group 5 is renamed from "Hex Colors" to "Hex Values".
The substring definition for $0 is extended to highlight all strings starting with a dollar sign and a hexadecimal digit as hexadecimal value.
The color group 6 with the directives is extended by some words listed on referenced page about Delphi syntax and not existing anywhere in the wordfile. There are added also the directives read and write although there are also the built-in functions Read and Write in the wordfile.
All empty color groups are removed from the wordfile.
Color group 10 "Types" is renumbered to become color group 7.
Color group 16 "Array Indices" is renumbered to become color group 8.
Color group 19 "Brace Color" is renumbered to become color group 9 and is renamed to "Parentheses". The color group could be also renamed to "Round Brackets".

The colors configured in Manage Themes dialog for syntax language Delphi for the color groups Types, Array Indices and Parentheses (or Round Brackets) must be reconfigured on using one of the two improved wordfiles.

The second ZIP file contains also delphi.uew and delphi_nocase.uew, but with some words moved to other color groups which I think are more suitable according to the Delphi syntax description.