File Change Detection issue - modification on log file not recognized

shware · Jun 03, 2010#12010-06-03T22:41+00:00

Hi,

Using UEStudio version 10.00.0.1009 running on Windows 7.

Working on a Rails project and am having trouble getting UEStudio to refresh the development.log file. I have to keep reopening it.

The editor does detect changes to other files. File Change Detection is enabled, set to check on focus change, polling is set to 0. If I create a file in notepad, save it, open it in UES, then change again in notepad - UES does detect the file change.

I only recently upgraded to Windows 7 from XP. Didn't have this problem on XP...

Any thoughts?

Thanks,

Shawn

Mofi · Jun 04, 2010#22010-06-04T11:59+00:00

I have read twice that it is possible under Windows 7 that editing a file with Notepad results in editing a different file than editing the supposed same file in UltraEdit/UEStudio. Because the file change detection works for other files in UEStudio, I doubt there is a problem inside UEStudio that it does not work for the log file where you see that behavior.

For verification do following:

Open the log file in UEStudio.
Use Edit - Copy File Path/Name to copy the full name of the opened file into the Windows clipboard.
Open Windows Explorer and paste into the address bar with Ctrl+V the copied full name of the file. Delete the name of the file to get in the address bar only the path of the file and press key RETURN to open the directory of the file. Look on file size and file date and write both somewhere down.
Start now Notepad via the start menu. Press Ctrl+O to open the file open dialog and press Ctrl+V in the file name field to paste the full name of the file opened in UEStudio and open the file in Notepad.
Edit the file slightly, save it and close Notepad.
Back in Windows Explorer compare file size and file date with what you have written down. Are both changed?
Switch to UEStudio. Does UES detect the file change and reloads the file content with confirmation by you or automatically depending on the configuration settings Prompt for update when files change and Automatically update changed files?

If file change detection using this procedure works for the file, you probably edit with Notepad using your previous method to open the file a different file than that opened in UEStudio. If you can see a changed file size and file date, but UEStudio does not detect the file change, you need to download, install and use Process Monitor from SysInternals (Microsoft) to find out why UEStudio fails to detect a file change on this file although file change detection works for other files.

shware · Jun 04, 2010#32010-06-04T16:15+00:00

Thanks - your suggestions helped narrow things down. There are two things going on, one of which appears to be an UEStudio bug (see #2).

1) Windows definitely isn't updating the timestamp on the rails' development.log. In Windows Explorer, I have to click on the file name to have the timestamp refresh. Very strange. Editing the file in Notepad or UEStudio, the timestamp updates immediately. Not sure if this is a Windows or Ruby issue. I think, it might be a Windows thing on files opened for exclusive writing (which I believe is the mode rails is using on this file).

2) Here's where I've tripped an UES bug. With this delay issue, I tried to enable polling to see if it would help. Since it didn't I disabled polling in the config area (setting poll increment to "0") but DID NOT disable "File Change Polling" on the log file before doing this. This stopped the File Change Detection from working on this specific file. I think this is a bug in UEStudio.

Steps to reproduce issue #2:

Open Advanced / Configuration / File Handling / File Change Detection and set the polling increment to 2.
Open the file to be polled and click View / File Change Polling.
Using notepad, make changes to file opened in step #2 and polling works as expected.
In UES set the polling increment back to 0.
Using notepad, make more changes to open file and you no longer have File Change Detection.

IMHO, if the polling interval is set to 0, then any files marked for polling should be treated normally in terms of File Change Detection.

Thanks!

Shawn

bulgrien · Jun 04, 2010#42010-06-04T16:55+00:00

shware wrote:1) Windows definitely isn't updating the timestamp on the rails' development.log. In Windows Explorer, I have to click on the file name to have the timestamp refresh. Very strange. Editing the file in notepad or UEStudio, the timestamp updates immediately. Not sure if this is a windows or ruby issue. Methinks it might be a Windows thing on files opened for exclusive writing

This sounds like "write-behind caching". Windows is caching the development log output stream to speed up your system by eliminating frequent disk access. The output stream is not flushed to disk until you actually try to access the log via Windows Explorer.

If you wish to disable write-behind caching, click the Windows start icon, right-click on Computer and click Manage. In the Computer Management console, expand the Storage node and click on Disk Management. At the bottom of the screen you will see a graphical representation of the logical partitions on each physical disk. Right-click on the Physical Disk (Disk 0, Disk 1, etc) and choose Properties. Under the Policies tab there should be a Write-caching policy with check boxes to tweak the caching behavior.

Jun 04, 2010#52010-06-04T17:02+00:00

shware wrote:IMHO, if the polling interval is set to 0, then any files marked for polling should be treated normally in terms of File Change Detection.

Regarding item #2, this is a user-to-user forum. Perceived defects and/or enhancement requests should be emailed directly to IDM.

shware · Jun 04, 2010#62010-06-04T19:25+00:00

Thanks bulgrien,

Disabled write-behind caching, but issue still exists. I've even written a little C# routine to test things out. On Windows 7, as long as app keeps the file open, the file timestamps won't update until another program (explorer, text editor, etc.) attempts to read the data.

Same C# util works as expected on Windows XP. Even though the file is open, each time it's written to the timestamp is updated and a change notification is triggered.

Thanks also for the reminder as to where to post bugs

Shawn

bulgrien · Jun 04, 2010#72010-06-04T23:07+00:00

Did you reboot your system after making the change? It shouldn't be necessary, but if there is a registry setting associated with the change on the dialog, it is possible that the configuration won't kick in until after a restart.

If the problem still persists after a restart, perhaps it is the Rails environment itself. If the log file is being written out to a text stream but the text stream is not being flushed after every write, then it may be the application that is performing the caching rather than the OS. I had a similar issue on a C++ program I recently wrote... my log file was not being updated in realtime because I wasn't flushing the output text stream between writes.

Mofi · Jun 05, 2010#82010-06-05T13:09+00:00

Issue 2) is indeed not well done by the IDM developers and you should report it by email. Changing the polling interval in configuration dialog should result in disabling automatically all enabled polling on files.

Further I think bulgrien is right with his suspicion that your program appends to the log file only small amount of data (less than 16 KB) from time to time and is not closing the file. This results in writing the data into the file stream buffer (RAM) and not to disk and so no other program can see a change until the file stream is flushed either when buffer is full, or file is closed, or the flush function is used after every write.

shware · Jun 08, 2010#92010-06-08T16:24+00:00

Hi,

Yes, a reboot was done after the initial test didn't work.

I haven't drilled down into the rails log writer, but my simple C# routine does do a flush and the timestamps still don't update.

For those interested, here's the small util:

Code: Select all

using System;
using System.IO;
using System.Text;
using System.Collections.Generic;

class Program
{

    static void Main(string[] args)
    {
    	  string line = "";
        using (StreamWriter outfile = new StreamWriter("c:\\testing\\test_log\\junk_log.log")) {
          do { 
             Console.Write("Enter a Line:");
             line = Console.ReadLine();
             if (line != null)  {
                outfile.Write(line + "\n");
                outfile.Flush();
              }
          } while (line != "quit");         	
        }
    }
}

Again, this IS NOT a UEdit / UEStudio issue but I'm grateful for the responses. It's surely a buffer kinda thing but the behavior is different from Windows XP. I've tried to google the issue but so far nothing found.

For the file polling issue, a bug report was sent although I've received no response. Even after a restart the setting in the INI file was still overriding the File Change Detection. IOW, even tho file polling was disabled and program restarted there's a section in the INI file named [Poll File] which contains the files being polled. Until that section was deleted, simple File Change Detection wasn't working.

Thanks again,

Shawn

vemund · Jul 11, 2010#102010-07-11T19:15+00:00

I have seen the same problem using tail (for win32).. and also registered that using baretail on the same file it update the modification automatically. I have also tested with some small C# utility apps - and have the same problems/bug? as you. Im running Win7 Pro 64x.. I'm still searching the web for an answer to this issue. It works on WinXP but not Win7...

Some more details I just have noticed. Using SystemInternals and Process Monitor. I see that baretail open and close the file 4-5 times per second.. So maybe it force a refresh this way. And all writes from your test app is logged as a WriteFile in Procmon.

SystemInternals http://technet.microsoft.com/en-us/sysi ... fault.aspx
baretail (free) http://www.baremetalsoft.com/baretail/

Willem · Jul 28, 2010#112010-07-28T20:43+00:00

I have the same issue where the polling of a log file doesn't work as expected. But there is one thing that makes it more specific to UltraEdit behavior. If the file is the current file viewed then the polling doesn't work, but if the focus is changed to another file tab and then back to the log file tab it reloads the changes.
Also switching to another application and back to UltraEdit will do the trick.
I already tried a lot of settings but still no success, maybe this will trigger someone else mind.

Willem

Mofi · Jul 29, 2010#122010-07-29T05:31+00:00

Willem, according to your description you are not using the polling feature, you have just the file change detection feature enabled. The file change detection feature just checks the properties of a file for change when you switch from one file to another file. The properties of the file getting the focus are read and compared with the file properties already kept in memory of UE/UES for this file. With an enabled configuration setting Check files for changes only on application focus change the file properties of all opened files are checked for changes when UE/UES gets back the focus from another application. When working often with dozens of files open and often switching applications it is better to turn of this configuration setting.

The polling feature is a check for file property changes triggered by a timer and not an event as used by the file change detection feature. For the polling feature first the timer for polling must be turned on by setting the value of configuration setting Poll for file changes at forced interval in seconds at Advanced - Configuration - File Handling - File Change Detection to a value greater than 0. Next you have to tell UE/UES for which file(s) you want a periodic check on file property change. You surely want that for all files open, just for one or two log files. Therefore you have to make the file active for which you want polling and click on View - File Change Polling. Redo this click for all the files which should be checked periodically for file change. UE/UES remembers on exit for which files the polling feature is turned on and does the periodic file change check again when the file is opened again.

lpx · Aug 05, 2010#132010-08-05T18:13+00:00

Hi.

On Windows Vista and Windows 7, the last access time is not updated by default (to help performance). Try checking how it's set, re-enable it if necessary, reboot, then try again (from an administrator command prompt):

Code: Select all

C:\Windows\system32>fsutil behavior query disablelastaccess
DisableLastAccess = 1

C:\Windows\system32>fsutil behavior set disablelastaccess 0
NOTE: Changes to this setting require a reboot to take effect.
DisableLastAccess = 0

I don't know if the SHChangeNotifyRegister API gets around this (presuming you're just polling last write instead of being actively informed).