Ctags / Find Symbol doesn't seem to work using UE 16.20+ (solved)

dkellner · Oct 13, 2010#12010-10-13T23:30+00:00

I have a problem since the last 2 updates - seemingly UE can't read the ctags format anymore. First I only got a "Could not find TAG" when finding a symbol; but then I tried another which occurs at several points around the project, and in this case, UltraEdit gives a list to show your options - "Which one did you mean?" - and this is where I understood what is happening. The list was like this:

Code: Select all

(1111): keyword  d:\myfile1.txt
(1112): keyword  d:\myfile1.txt
(1113): keyword  d:\myfile1.txt

Now clicking one of these results gave me a "d:\keyword is not found" message. I think UE started to confuse the keyword with the filename - weird but this is the effect I see! So it's no surprise that it couldn't find the file which is not really a file but the keyword itself I was looking for!

Does anyone have the same problem?
Is it a setting I screwed up, or is it a global symptom for those updated to 16.20.xxx?

(My ctags file is just like before, so NO, it's not about the tag file format.)

Mofi · Oct 14, 2010#22010-10-14T06:06+00:00

I opened one of my older 'C' projects using Ctags with UE v16.20.0.1011 English and executed Find Symbol command several times on different symbols and it always worked fine. Some of them were also multiple defined (for two different usage), so I get the dialog and it worked, too. I also executed Project - Create CTAG file to recreate the Ctags tag file and Find Symbol still worked fine. The newly created tag file was 100% identical with the previous one created with ctags.exe installed with UE v11.20 which I used to create the project with several *.c and *.h files.

Which parameters do you use for Ctags? Can you post some lines of the tag file?

dkellner · Oct 14, 2010#32010-10-14T09:41+00:00

I'm using PHP (it shouldn't be a difference, I just mention it).

ctag options: -n -L "%fi" -f "%fo"

Also tried without the quotes, also tried without -n - neither seems to help.

First few lines of tagfile:

Code: Select all

!_TAG_FILE_FORMAT	1	/original ctags format/
!_TAG_FILE_SORTED	1	/0=unsorted, 1=sorted, 2=foldcase/
!_TAG_PROGRAM_AUTHOR	Darren Hiebert	/[email protected]/
!_TAG_PROGRAM_NAME	Exuberant Ctags	//
!_TAG_PROGRAM_URL	http://ctags.sourceforge.net	/official site/
!_TAG_PROGRAM_VERSION	5.5.4	//
AdLogPath	d:\http\partvonal\inc.basic.php	/^	$AdLogPath       = "$FsRoot\/bs\/stats";$/
AdLogPath	d:\http\partvonal\inc.basic.php	/^	$AdLogPath       = "\/home\/elofizetok\/schubertg\/partvonal.hu\/bs\/stats";$/
AdLogPath	d:\http\partvonal\inc.basic.php	/^	$AdLogPath       = "\/http\/partvonal\/bs\/stats";$/
AddImage	d:\http\partvonal\admin\rte.files\insert_image.html	/^	function AddImage() {$/
AdsRoot	d:\http\partvonal\inc.basic.php	/^	$AdsRoot         = "$FsRoot\/img\/banners";$/
AdsRoot	d:\http\partvonal\inc.basic.php	/^	$AdsRoot         = "\/home\/elofizetok\/schubertg\/partvonal.hu\/img\/banners";$/
AdsRoot	d:\http\partvonal\inc.basic.php	/^	$AdsRoot         = "\/http\/partvonal\/img\/banners";$/
AdsRoot_web	d:\http\partvonal\inc.basic.php	/^	$AdsRoot_web     = "http:\/\/$Domain\/img\/banners";$/
AdsRoot_web	d:\http\partvonal\inc.basic.php	/^	$AdsRoot_web     = "http:\/\/me\/partvonal\/img\/banners";$/
AdsRoot_web	d:\http\partvonal\inc.basic.php	/^	$AdsRoot_web     = "http:\/\/www.partvonal.hu\/img\/banners";$/
BannerDataFile	d:\http\partvonal\admanager\inc.admanager.php	/^$BannerDataFile    = "$DOCS\/inc.bannerdata.php";$/
BannerDataFile	d:\http\partvonal\inc.basic.php	/^	$BannerDataFile  = "$FsRoot\/inc.bannerdata.php";$/
BannerDataFile	d:\http\partvonal\inc.basic.php	/^	$BannerDataFile  = "\/home\/elofizetok\/schubertg\/partvonal.hu\/inc.bannerdata.php";$/
BannerDataFile	d:\http\partvonal\inc.basic.php	/^	$BannerDataFile  = "\/http\/partvonal\/inc.bannerdata.php";$/
BannerDataSample	d:\http\partvonal\admanager\inc.admanager.php	/^$BannerDataSample  = "$DOCS\/inc.bannerdata.sample";$/
Charset	d:\http\partvonal\sx.core\inc-sitexpress.php	/^	$Charset = $sxDefaultCharset; if(!$Charset) $Charset="utf-8";$/
CharsetDecode	d:\http\partvonal\sx.core\inc-sitexpress.php	/^	$CharsetDecode = flag10(string_begins(strtolower($Charset),"iso"));$/
Cols	d:\http\partvonal\rolunk.inc.php	/^		$Cols = 5;$/
CompactHeading	d:\http\partvonal\index.php	/^	$CompactHeading = $x;$/

And wait! In some cases it works - when it's only one symbol to find and it's inside the current file which is also part of the project. So basically, when it comes to switching to another file, there the problem is.

I did a full Windows restart. No good. (I know it makes no sense anyway but don't say I was not trying.)

Mofi · Oct 14, 2010#42010-10-14T13:53+00:00

As I tried Find Symbol with my project the symbol definitions were in other files than the one I executed the command. I really don't have any problem with Find Symbol.

My tag files have _TAG_FILE_FORMAT 2. It is interesting that your tag file uses _TAG_FILE_FORMAT 1 because format 2 is the default.

Also interesting is that you use -n which is equal −−excmd=number, but none of the lines in your tag file has a line number stored. I'm not using -n, but my tag file contains mainly line numbers and only a minority of symbols have only a search pattern and those symbols are also found.

Where do you have UltraEdit installed?

I don't use the standard paths. I have UE installed in C:\Programs\UltraEd and therefore the path to ctags.exe is C:\Programs\UltraEd\GNU\ctags.exe and so there is surely no problem because of a space in path (and even all directory and file names are not longer than 8 characters, but that is not so important).

Furthermore my projects use groups (project files must be added to the project manually) and not folders (files are added to project automatically). That should not be important here because the tag file contains the full name of the files containing the symbol definitions, but maybe this has an effect on the Find Symbol behavior.

dkellner · Oct 15, 2010#52010-10-15T00:33+00:00

Here's what happens.
The corresponding lines in my tagfile are:

Code: Select all

sql_query	D:\http\lovebox\sql.interface.php	30;"	f
sql_query	D:\http\lovebox\sql.interface.php	53;"	f

And the strange thing is: when I delete one of the lines, the other correctly finds the way, jumping there immediately. If only 1 match is found, everything is fine.

I've changed the tags.lst file path (so that it's 8.3 conform, as you suggested), also changed to --format=2, which happened correctly, and yes, now my tag file is full of line numbers instead of whole lines. (See below.)

UltraEdit is in "C:\Program Files\UltraEdit v16" but now it only affects the ctags.exe call - which surely happens since the tags.lst file has correctly been generated at the new location. So there is no problem with the generation process.

Really, I can't think of anything. It's weird.
The error message shows that UE tries to append a file named "sql_query" (the symbol) in "d:\danger\ue" (which is the new place for tags file). Why would it do so?... OR, maybe it's a problem with the error message itself?

Code: Select all

!_TAG_FILE_FORMAT	2	/extended format; --format=1 will not append ;" to lines/
!_TAG_FILE_SORTED	1	/0=unsorted, 1=sorted, 2=foldcase/
!_TAG_PROGRAM_AUTHOR	Darren Hiebert	/[email protected]/
!_TAG_PROGRAM_NAME	Exuberant Ctags	//
!_TAG_PROGRAM_URL	http://ctags.sourceforge.net	/official site/
!_TAG_PROGRAM_VERSION	5.5.4	//
ARF_KEEPFIRST	D:\http\lovebox\inc.lovebox.php	64;"	d
ARF_KEEPLAST	D:\http\lovebox\inc.lovebox.php	65;"	d
AbsWebRoot	D:\http\lovebox\inc.lovebox.php	112;"	v
AbsWebRoot	D:\http\lovebox\inc.lovebox.php	120;"	v
AdmgrRoot	D:\http\lovebox\inc.lovebox.php	145;"	v
AdmgrRootWeb	D:\http\lovebox\inc.lovebox.php	146;"	v
AdsRoot	D:\http\lovebox\inc.lovebox.php	143;"	v
AdsRootWeb	D:\http\lovebox\inc.lovebox.php	144;"	v
CMP_photos	D:\http\lovebox\inc-userpage.php	66;"	f
CR	D:\http\lovebox\inc.lovebox.php	58;"	d
CRLF	D:\http\lovebox\inc.lovebox.php	60;"	d
Charset	D:\http\siteXpress\sx.core\inc-sitexpress.php	1184;"	v
CharsetDecode	D:\http\siteXpress\sx.core\inc-sitexpress.php	1185;"	v
Cond	D:\http\lovebox\inc.api.php	396;"	v
Converter	D:\http\siteXpress\sx.core\sx.lists.php	535;"	v
Converter	D:\http\siteXpress\sx.core\sx.lists.php	536;"	v
CopyrightLine	D:\http\siteXpress\sx.core\inc-sitexpress.php	125;"	f
DataRoot	D:\http\lovebox\inc.lovebox.php	133;"	v
Day	D:\http\lovebox\inc.lovebox.php	41;"	d

Mofi · Oct 15, 2010#62010-10-15T05:52+00:00

It is really weird what's going on on your computer. The tag list file is 100% correct. The dialog looks correct. The error message should be either a simple Could not find TAG if symbol not present in the tag file or Full name of file was not found. if the file could not be found. But the error message you get is totally strange.

I tried a lot to see also the error message you captured. I even converted the tag file and my source files to UTF-8 with UNIX line endings, but the Find Symbol command still worked.

I suggest to run the installer of UltraEdit again via Control Panel and choose the option to repair current installation. Perhaps one of the program files is partly damaged.

Another suggestion is to download, unpack to a local folder and run Filemon from Sysinternals (Microsoft), set the Include filter to d:\danger;d:\http (not case sensitive), start capturing the file and directory accesses and execute in UltraEdit the Find Symbol command. Maybe we can find out what's going on with the log file created by b]Filemon[/b].

dkellner · Oct 16, 2010#72010-10-16T01:32+00:00

Well... I followed your thinking and used ProcMon (new version of FileMon) to track what happens.
See the simple and unhappy results.

Man, what the hell is going on?

Really. I tried to edit the tags file so that filename comes first and then the symbol ("sql_connect" this time). Works as expected: symbol not found. Fine, because it's not there. Okay. Undid it, then tried to create the file "sql_connect" and THEN find the symbol sql_connect. List pops up - let's see the first one, and lo: sql_connect is really opened. I mean the file. So it literally does what it says: whenever in doubt, whenever symbol can be found at several places, the list comes up, and then the first column is considered a filename.

I'm ready to replace my uedit32.exe to something with a debug line - if it helps to corner the problem. But okay, I understand this is a rare symptom and no one else encountered it (?) so the decision is up to you. I can live without this feature, it's just... well, a painful loss.

Mofi · Oct 16, 2010#82010-10-16T09:13+00:00

It is really unbelievable what happens on your computer. I really don't understand why UltraEdit reads the symbol as file name and merges it with the path name.

I created a small project to simulate on my computer your project. I created a file D:\http\lovebox\sql.interface.php, added to 2 sql_query functions into this file, let Ctags create the tag file stored on a different drive which results in the same 2 lines as you posted, closed project file sql.interface.php, opened a new file, entered sql_query and executed command Find Symbol. The dialog opens and all 3 buttons worked fine - sql.interface.php was opened on first execution and the cursor was set on correct lines on all 3 executions of the command by pressing the 3 buttons in the dialog. Of course current version UE v16.20.0.1011 has a small bug because it places the cursor not at start of the line (column 1, internal character index position 0), it sets the cursor to column 2. I reported this bug immediately after release of build 1011 of UE v16.20 to IDM and hope this small bug is fixed in the next release. I tested that even with first released build 1009 in case you are not using latest v16.20 and the Find Symbol command also works fine with that version which has the same small bug with cursor at column 2 instead of column 1.

Do you select the symbol sql_query before executing command Find Symbol?
I just detected that when the active file is a Unicode file (UTF-8 or UTF-16) and I select sql_query and execute command Find Symbol (by pressing key F11) I get the error message Could not find TAG which is definitely a bug I need to report. That issue does not exist when active file is an ASCII/ANSI file. Just setting the cursor anywhere (beginning, middle or end) on symbol sql_query and pressing key F11 works for ASCII/ANSI files as well as for Unicode files of any encoding. So selecting the symbol name is not good.

I suggest that you uninstall UltraEdit and reboot your computer. Your configuration settings are not deleted during uninstall. Then install v16.20.0.1011 of UltraEdit again into the same program directory as before. Start UltraEdit and test if Find Symbol on sql_query is now working.

If still not working after reinstallation of UltraEdit, close UltraEdit, copy %APPDATA%\IDMComp\UltraEdit into the address bar of Windows Explorer and hit key RETURN to open this directory. Rename now uedit32.ini to uedit32_bak.ini and start UltraEdit. That simulates a "first start for new user respectively first start after installation" and therefore UltraEdit creates now uedit32.ini completely new with the standard settings and shows you the dialogs normally displayed only on first start. Take care on the options in these dialogs before pressing key OK, especially the file associations dialog. Your other settings like the wordfiles with your color settings, your menus, toolbars, your keyboard settings and so on are not modified. Open the project and execute command Find Symbol on sql_query. Is it now working or is the symbol name still read as file name?

If still not working with default settings, close UltraEdit, delete newly created uedit32.ini and rename uedit32_bak.ini back to uedit32.ini.

In this case I suggest you pack your whole project, or a smaller one where the wrong behavior also occurs, together with the ctags tag file, and the *.prj and *.pui file of the project, and whole directory %appdata%\IDMComp\UltraEdit into a ZIP or RAR archive and send this file with a description what happens to IDM support by email. Include also a link to this forum topic. Hopefully IDM can reproduce the wrong behavior with all your files and the developers can look into it to fix it.

dkellner · Oct 16, 2010#92010-10-16T16:51+00:00

Okay, first of all, thanks for the efforts and for approaching the problem in a serious way.

Now these uninstall-reinstall things are something I'm afraid of: I need to be productive and I believe that if my computer acts unpredictable now, it can be even more like that when I do such heavy operations. I won't take this risk.

BUT.

Your last few lines gave me a new idea. I'll do some further testing, find the MINIMUM circumstances that produce the error and let you & IDM know what it is. Even if it won't solve the problem, it will get me some closer to understand what's actually happening inside. There must be a point where the phenomenon starts to occur; I'll start a project from ground zero and try to find this point.

(Col 2 problem has also been noticed but I thought it has nothing to do with this one, and it's not disturbing at all.)

And again, whatever happens: you really tried and I appreciate that a lot.

Oct 16, 2010#102010-10-16T17:20+00:00

BREAKING NEWS. Your tip about renaming the uedit32.ini seems to work. Now I renamed it back - the problem returned - and I'll find out what's causing this. Very strange but now I have a better grip.

Oct 16, 2010#112010-10-16T21:38+00:00

SOLUTION

Well it was a single character thing. In uedit32.ini there is a line for "Found string", that is, result format for strings when doing a "Find in Files" scan. Now, I've inserted a "\t" character here instead of the space, hoping that in the result list there will be a tab between the filename and the string found, so that I can copy it to Excel into 2 separate cells. This couldn't be done - BUT THE TAB REMAINED THERE.

And this is what caused the problem.

I don't know why - it's not logical. It's like a bug to me. But okay: no one ever does this, I'm a tweaked mind to try to insert a tab into a result list

Well, it took me a while to find this one but as soon as my comparator pointed out that there is a "\t" difference in one of the lines, I just knew it's the one. And yes it is.

Now my Find Symbol function works again.
Thanks for the cooperation - and, well, we've learned something today

PS: If the "Find in Files" result list could provide a tab after the ":" sign, that would be a good thing to me. But even if it won't: it shouldn't be related to ctags results.

Mofi · Oct 18, 2010#122010-10-18T06:04+00:00

Glad to read that the Ctags problem is fixed now for you. I did not know that Advanced - Configuration - Search - Set Find Output Format - Found Line is affecting the Find Symbol feature. It is not possible to use tabs for found lines in the output window as you can read at Find in Files output replaces tabs with spaces. This configuration setting is stored in uedit32.ini with:

Code: Select all

[Find Out Format]
Found=1
Found String=$P($L): $S

Using multiple spaces should not be a problem. Or do you use Find in Files with Results to Edit Window where tabs are possible?

For your information: I just sent following two reports to IDM by email.

UE - Find Symbol not working for selected symbol name in Unicode file

I detected that in UE v16.20.0.1011 the command Find Symbol is not working if the active file is a Unicode file (UTF-16 or UTF-8), the symbol name is selected and then the command "Search - Find Symbol" is executed. UltraEdit shows in this case just the message "Could not find TAG".

To reproduce this problem unpack the content of the ZIP file into directory C:\Temp and open the project file CtagsUnicode.prj. File php_example.php is loaded which is a UTF-8 file with BOM. The cursor is already placed at line 9, column 20 - in the middle of symbol "microtimeFloat". If you execute now "Search - Find Symbol", the caret is moved to line 4.

As I already reported there is a mistake in UE v16.20 when positioning the caret to a line according to a line number in a Ctags tag file, or in the output window, or in the "List lines containing string dialog". The caret is set to column 2 of the line instead of column 1. This happens here, too.

However, the Find Symbol command worked. Go back to line 9 and double click on symbol "microtimeFloat" or execute "Edit - Select Word" to select this symbol. Execute now the command "Search - Find Symbol" and you get the error message "Could not find TAG". It looks like UE is not converting the selected string from Unicode to ASCII/ANSI as required to find the string in the ASCII/ANSI tag file created with Ctags.

Convert the file with "File - Conversions - UTF-8 to ASCII" to ASCII, select again symbol "microtimeFloat" in line 9 and execute the command "Find Symbol" again. Now the command works.

Update: This issue was fixed with UltraEdit for Windows v24.00 and UEStudio v17.00.

UE - Find Symbol not working with search pattern in Unicode file

I detected that in UE v16.20.0.1011 the command Find Symbol is not working if the target file is a Unicode file (UTF-16 or UTF-8) and Ctags stored the symbol name with a search pattern in the tag file. UltraEdit sets the cursor in this case to top of the the file instead of start of correct line.

To reproduce this problem unpack the content of the ZIP file into directory C:\Temp and open the project file CtagsUnicode.prj. File php_example.php is loaded which is a UTF-8 file with BOM. The cursor is already placed at line 9, column 20 - in the middle of symbol "microtimeFloat". If you execute now "Search - Find Symbol", the caret is moved to line 4.

That is working because the "Ctag Options" used to create the tag file were

-n -L %fi -f %fo

The parameter -n is not default, but is required to save the symbol name in the tag file with line number instead of a search pattern. So the tag file C:\Temp\CtagsUnicode\CtagsUnicode.tag contains as last line:

Code: Select all

microtimeFloat  C:\Temp\CtagsUnicode\php_example.php    4;"     f

Open "Project - Ctag Options" and change the options to default which is

-L %fi -f %fo

Execute now the command "Project - Create CTAG file". If you look into file C:\Temp\CtagsUnicode\CtagsUnicode.tag, you will see that the last line changes to:

Code: Select all

microtimeFloat  C:\Temp\CtagsUnicode\php_example.php    /^  function microtimeFloat ()$/;"      f

So instead of a line number a search pattern is now stored in the tag file to find the symbol. But with caret at line 9 column 20, executing the command "Find Symbol" does not result in setting the caret to start of line 4 as it should. Instead the caret is set to top of the file. The reason is that the target file (= active file) is a Unicode file.

If you convert the file now with "File - Conversion - UTF-8 to ASCII" into an ASCII file, set the cursor again to line 9, column 20 and execute again command "Find Symbol", you will see that now there is no problem with the search pattern and the caret is set to line 4.

Conclusion: Find Symbol is not working correct for Unicode files when a search pattern is stored in the tag file.

Update: This issue was fixed with UltraEdit for Windows v24.00 and UEStudio v17.00.

dkellner · Oct 19, 2010#132010-10-19T00:18+00:00

No, I was trying to use tabs in the output window. Didn't realize that it can't be done. Well, it was fun tracing down this thing anyway.

Have a nice time!

D.K.

Mofi · Feb 12, 2017#142017-02-12T13:51+00:00

The two issues with Find Symbol not working for Unicode encoded files as described above are fixed with UltraEdit for Windows v24.00 and UEStudio v17.00.