Running SAS programs from within UE

Running SAS programs from within UE

4

    Jun 16, 2021#1

    Hi all

    I would like to implement this UltraEdit power tip: Edit, Run, and Debug your SAS Programs in UltraEdit

    I have tried to set this up, but am getting stymied by spaces in file paths.
    I have got everything working up to this section in the .bat file:

    Code: Select all

    %EXESAS% ^
    -CONFIG %CONFIGSAS% ^
    -autoexec %AUTOEXECSAS% ^
    -sysin %1 -log %2 –nodms
    
    But when I run it, it fails. Below is what UE is writing in the output window (note the incorrect handling of the spaces, and note that I have protected file locations and simplified the output by replacing irrelevant portions with the ellipsis)

    This is the output window in UE:

    Code: Select all

    c:\Users\...\AppData\Roaming\IDMComp\UltraEdit\runSAS.bat" "C:\Users\...\OneDrive - The University …\...\test.sas" "C:\Users\...\OneDrive - The University …\...\test.log ...
    This is the Command window (with line breaks added to make it easier to follow):

    Code: Select all

    C:\Users\...\OneDrive - The University ...\...\Develop Run and Debug your SAS Programs in UltraEdit_files>
    "c:\Program Files\SASHome\SASFoundation\9.4\sas.exe"  
    -CONFIG "c:\Program Files\SASHome\SASFoundation\9.4\sasv9.cfg"  
    -autoexec "c:\Users\...\OneDrive - The University ...\...\Develop Run and Debug your SAS Programs in UltraEdit_files\autoexec.sas"  
    -sysin "C:\Users\...\OneDrive" - The University …\!USyd Projects\Google "Trends\Data_management\Cov\test.sas" 
    -log "C:\Users\...\OneDrive" ûnodms
    So the problem seems to be coming in with the %1 and %2 parameters that UE is supplying (according to the tip). I tried placing those in quotes in the .bat file, but that didn't work.
     
    Does anyone have any idea how to get those file locations sent through properly to the command window?

    Thanks in advance

      Jun 16, 2021#2

      Hi

      Some more searching (didn't pick this up yesterday) led me to this page: File path and name with spaces in batch file for loop

      Where I discovered the fix for the first part: add ('type %1') and ('type %2')

      The script still fails, now indicating in the SAS error log:

      Code: Select all

      ERROR: Unrecognized SAS option name ÛNODMS.
      ERROR: (SASXKRIN): KERNEL RESOURCE INITIALIZATION FAILED.
      ERROR: Unable to initialize the SAS kernel.
      Any ideas about how to send through the nodms SAS option?

      Also, it seems as if the program file is now fed through correctly, but still not the log file. See the Command window output below:

      Code: Select all

      C:\Users\...\OneDrive - The University ...\...\Develop Run and Debug your SAS Programs in UltraEdit_files>
      "c:\Program Files\SASHome\SASFoundation\9.4\sas.exe"  
      -CONFIG "c:\Program Files\SASHome\SASFoundation\9.4\sasv9.cfg"  
      -autoexec "c:\Users\...\OneDrive - The University ...\...\Develop Run and Debug your SAS Programs in UltraEdit_files\autoexec.sas"  
      -sysin ('type C:\Users\...\OneDrive" - The University ...\!USyd Projects\Google "Trends\Data_management\Cov\test.sas') 
      -log ('type C:\Users\...\OneDrive') ûnodms
      Any help will be appreciated. 

      Thanks

      6,601547
      Grand MasterGrand Master
      6,601547

        Jun 16, 2021#3

        The file batch_file.txt can be downloaded from the power tip page. It contains the lines:

        Code: Select all

        #Location of config and autoexec files
        set CONFIGSAS=<path & file to your config file>
        set AUTOEXECSAS= <path & file for your autoexec file>
        
        #Location of sas.exe. Note %1 & %2 parms will be set by UltraEdit Tool.
        #For 64 bit, use: C:\Program Files\SAS\SASFoundation\9.2(32-bit)\sas.exe
        Set EXESAS= <path & file to sas.exe>
        "%EXESAS%" ^
         -CONFIG "%CONFIGSAS%" ^
         -autoexec "%AUTOEXECSAS%" -sysin %1 -log %2 –nodms
        
        Mistake 1: # is used instead of rem used for comment lines.

        There mixed somebody Windows batch with Linux/Mac bash syntax as # is definitely not interpreted by Windows command processor cmd.exe as comment line. There should be used the command rem (or the invalid label :: which I do not recommend for usage in a batch file).

        Mistake 2: The character left to option nodms is not a hyphen-minus character.

        The character left to option nodms is not a hyphen-minus character with hexadecimal code value 2D, but a Windows-1252 encoded en-dash with hexadecimal value 96 (hexadecimal 2012 in Unicode) which is interpreted with code page 437 or code page 850 as Latin capital letter U with circumflex.

        That wrong character is responsible for the error message: ERROR: Unrecognized SAS option name ÛNODMS.

        Mistake 3: The ampersand & is interpreted as AND operator by cmd.exe outside a double quoted string.

        So please use in your batch file:

        Code: Select all

        @rem Location of config and autoexec files
        set CONFIGSAS=C:\Program Files\SASHome\SASFoundation\9.4\sasv9.cfg
        set AUTOEXECSAS=%UserProfile%\OneDrive - The University ...\...\Develop Run and Debug your SAS Programs in UltraEdit_files\autoexec.sas
        
        @rem Location of sas.exe. Note %1 and %2 parms will be set by UltraEdit Tool.
        set EXESAS=C:\Program Files\SASHome\SASFoundation\9.4\sas.exe
        "%EXESAS%" -CONFIG "%CONFIGSAS%" -autoexec "%AUTOEXECSAS%" -sysin %1 -log %2 -nodms
        
        I does not really make sense to use a multi-line command line here with usage of ^ to escape the line terminations. That just makes processing of the batch file a bit slower.

        It makes also no real sense for me to define three environment variables with the full qualified file names of the SAS executable, config and autoexec files which are used only once in the batch file.

        And if the log file should be created always in the directory of the file being processed by SAS with file extension .log instead of file extension of active file in UltraEdit, there is no need to use "%p%n.log" in the UltraEdit tool configuration to pass the name of the active file with path with file extension .log to the batch file. It is easier to use in this case in the batch file a single line with:

        Code: Select all

        "C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\sasv9.cfg" -autoexec "%UserProfile%\OneDrive - The University ...\...\Develop Run and Debug your SAS Programs in UltraEdit_files\autoexec.sas" -sysin %1 -log "%~dpn1.log" -nodms
        Well, in real a batch file would not be needed at all. The UltraEdit tool configuration could contain this command line with %1 being replaced in tool configuration by "%f" and "%~dpn1.log" by "%p%n.log" resulting in the UltraEdit tool configuration command:

        Code: Select all

        "C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\sasv9.cfg" -autoexec "%UserProfile%\OneDrive - The University ...\...\Develop Run and Debug your SAS Programs in UltraEdit_files\autoexec.sas" -sysin "%f" -log "%p%n.log" -nodms
        However, if a batch file processed by cmd.exe should be used just to run sas.exe which could be done also directly by UltraEdit, a better batch file code would be:

        Code: Select all

        @echo off
        setlocal EnableExtensions DisableDelayedExpansion
        rem Definition of the full qualified file names of config and autoexec files.
        set "CONFIGSAS=full path and file to your config file"
        set "AUTOEXECSAS=full path and file for your autoexec file"
        
        rem Definition of the full qualified file name of sas.exe.
        rem For 64 bit use: C:\Program Files\SAS\SASFoundation\9.2(32-bit)\sas.exe
        set "EXESAS=full path to\sas.exe"
        
        rem Note: %1 and %2 reference the parameters passed by the UltraEdit tool
        rem to this batch file. The second argument is optional and can be omitted
        rem in the UltraEdit tool configuration.
        if "%~1" == "" echo ERROR: File to process by SAS is not specified.& goto :EOF
        set "LogFile=%~2"
        if not defined LogFile set "LogFile=%~dpn1.log"
        @echo on
        "%EXESAS%" -CONFIG "%CONFIGSAS%" -autoexec "%AUTOEXECSAS%" -sysin %1 -log "%LogFile%" -nodms
        @endlocal
        
        I will inform IDM support by email about the serious mistakes made by the contributor Keith Adams to get them all fixed.

        I have the favor to ask you for running a command in a Windows command prompt window and tell me the output of the command.

        Code: Select all

        %SystemRoot%\System32\reg.exe QUERY "HKLM\Software\Microsoft\Windows\CurrentVersion\App Paths\sas.exe" /ve
        What is output on running this command line in a command prompt window?

        There can be output the error message:

        Code: Select all

        ERROR: The system was unable to find the specified registry key or value.
        But there could be output also the default string value of the queried registry key which would be the C:\Program Files\SASHome\SASFoundation\9.4\sas.exe (or the short 8.3 file name) in your case on SAS being registered during installation as recommended by Microsoft according to the documentation about Application Registration. I could suggest a batch file to use which can find out itself the full qualified file name of sas.exe on being registered as recommended by Microsoft.
        Best regards from an UC/UE/UES for Windows user from Austria

        4

          Jun 17, 2021#4

          Thank you for the tips.
          Your script to run it directly from the tool produces this output to the output window:

          Code: Select all

          "C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\sasv9.cfg" -sysin "C:\Users\...\OneDrive ...\Data_management\Cov\test.sas" -log "C:\Users\...\OneDrive ...\Data_management\Cov\test.log" -nodms ...
          'C:\Program' is not recognized as an internal or external command,
          operable program or batch file.
          It seems as if the correct information is being sent through? But still not getting the right result.
          The single line batch file produces this in the command window (I had to change it to "%1" to get this):

          Code: Select all

          C:\Users\jrau7673\AppData\Roaming\IDMComp\UltraEdit>
          "C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" 
          -CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\sasv9.cfg" 
          -autoexec "C:\Users\...\OneDrive - The University ...\Develop Run and Debug your SAS Programs in UltraEdit_files\autoexec.sas" 
          -sysin "C:\Users\...\OneDrive" - The University ...\Google "Trends\Data_management\Cov\test.sas"
          -log "C:\Users\...\OneDrive" - The University ...\Google "Trends\Data_management\Cov\test.log"
           SAS produces this error log:

          Code: Select all

          ERROR: Unrecognized SAS option name .LOG".
          ERROR: Unrecognized SAS option name "TRENDS\DATA_MANAGEMENT\COV\TEST.LOG".
          ERROR: Unrecognized SAS option name PROJECTS\GOOGLE.
          [size=50]ERROR: Unrecognized SAS option name UNIVERSITY.[/size]
          ERROR: Unrecognized SAS option name THE.
          ERROR: Unrecognized SAS option name .
          
          ERROR: Unrecognized SAS option name "TRENDS\DATA_MANAGEMENT\COV\TEST.SAS".
          ERROR: Unrecognized SAS option name PROJECTS\GOOGLE.
          [size=50]ERROR: Unrecognized SAS option name UNIVERSITY.[/size]
          ERROR: Unrecognized SAS option name THE.
          ERROR: Unrecognized SAS option name .
          ERROR: (SASXKRIN): KERNEL RESOURCE INITIALIZATION FAILED.
          ERROR: Unable to initialize the SAS kernel.
          So the spaces still seem to be creating problems.

          The command line you requested to return the registry key produces:

          Code: Select all

          HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\App Paths\sas.exe
          
              (Default)    REG_SZ    C:\Program Files\SASHome\SASFoundation\9.4\sas.exe
          Thanks

          6,601547
          Grand MasterGrand Master
          6,601547

            Jun 17, 2021#5

            1. Direct execution of sas.exe by UltraEdit

            Please let me know the version of UltraEdit used by you. There are some older versions of UltraEdit which have problems with tool (executable or script) to run containing in its full qualified file name a character which requires double quoting the full file name.

            I tried with UE v28.10.0.26 a tool with command being defined with the command line:

            Code: Select all

            "C:\Program Files (x86)\WinRAR\Rar.exe" a -idcdp "%p%n.rar" "%f"
            I named a text file UltraEdit Tool (Test) File!.txt for testing this tool configuration, opened this file in UltraEdit, executed the tool and I got in the directory of the text file the file UltraEdit Tool (Test) File!.rar and displayed in the output window:

            Code: Select all

            "C:\Program Files (x86)\WinRAR\Rar.exe" a -idcdp "C:\Temp\UltraEdit Tool (Test) File!.rar" "C:\Temp\UltraEdit Tool (Test) File!.txt" ...
            
            Creating archive F:\Temp\UltraEdit Tool (Test) File!.rar
            
            Adding    F:\Temp\UltraEdit Tool (Test) File!.txt                         OK 
            So everything worked fine and as expected by me with 32-bit UltraEdit v28.10.0.26 running on Windows 7 x64.

            2. Execution of the single line batch file via tool

            There is something wrong on using "%1" instead of just %1 in the batch file with the single command line. %1 references the first argument string passed to the batch file exactly as passed to the batch file. As in UltraEdit tool configuration "%f" is used hopefully by you, the first argument string is already enclosed in " and therefore the usage of "%1" would result in execution of ""C:\full path\file name.sas"" resulting in an invalid arguments list for sas.exe. It is safe to use in a batch file always "%~1" which references the first argument string with surrounding double quotes removed by the Windows command processor on first argument string enclosed in double quotes at all and enclose the argument string in the batch file in double quotes to pass the argument string (file name) always correct to sas.exe. However, it is not possible to pass a full file name containing a space or one of these characters &()[]{}^=;!'+,`~ to a batch file without being enclosed in ".

            I can see on your output window content :
            1. The config file is specified correct with one " at beginning of full file name and one " at the end.
            2. The autoexec file is specified correct with one " at beginning of full file name and one " at the end.
            3. The sysin file is not specified correct. There should be also just " at beginning of full file name and one " at the end, but there are multiple double quotes in "C:\Users\...\OneDrive" - The University ...\Google "Trends\Data_management\Cov\test.sas". So for cmd.exe and for sas.exe this string is interpreted as at least six argument strings:
              1. "C:\Users\...\OneDrive"
              2. -
              3. The
              4. -
              5. University ...\Google
              6. "Trends\Data_management\Cov\test.sas"
              That should be impossible on using in the tool configuration "%f" and in the batch file just %1.
              Does the name of active file in UltraEdit on tool execution contain one or more Unicode characters?
              Yes, then it would be better to use in tool configuration just %F so that UltraEdit passes the short 8.3 file name of active file to the batch file which never contains a space or a Unicode character or any other character which would require enclosing the full file name in double quotes. cmd.exe is not designed for argument strings with Unicode characters.
            4. The log file is not specified correct, too. There are also four double quotes instead of just two double quotes. That should be impossible on using "%~dpn1.log".
              I begin to think that you are not really using in batch file and in the tool configuration command field what you have written here or the used version of UltraEdit has really a bug with passing the full name of active file correct to the Windows command processor respectively the batch file.
            3. Application registration of sas.exe

            Thank you for that information. SAS is registered as recommended by Microsoft. So it is possible to use a command line in the batch file to query the full qualified file name of sas.exe from Windows registry on not defined at all in the batch file by the user or the path of sas.exe is (no longer) correct specified in the batch file. I will take that into account on my suggestion for a better batch file.
            Best regards from an UC/UE/UES for Windows user from Austria

            4

              Jun 18, 2021#6

              UltraEdit Text Version 28.10.0.26
              Windows 10 Enterprise v1909 64 bit

              I decided to simplify things.
              I have created a dedicated test folder and file:
              c:\Users\...\Documents\File directory with a space\Test file.sas

              I also put the autoexec file in there, although really, I don't need the autoexec file at the moment.
              So I decided to first remove the -autoexec component, because I reason that once we get this working, we can then always put it back and it should work. In that way, if I do need it in the future, it will be fine.

              So here are the results.

              1. Running directly from the tool.

              This was my command line (No AUTOEXEC and also removed the -NODMS option):

              Code: Select all

              "C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\sasv9.cfg" -sysin "%f" -log "%p%n.log"
              And this is the output:

              Code: Select all

              "C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\sasv9.cfg" -sysin "C:\Users\...\Documents\File directory with a space\Test file.sas" -log "C:\Users\...\Documents\File directory with a space\Test file.log" ...
              'C:\Program' is not recognized as an internal or external command,
              operable program or batch file.
              This is still the same as yesterday. Why am I getting such different results to you?

              2. Running the single line batch file.

              This was the tool command line:

              Code: Select all

              "c:\Users\jrau7673\AppData\Roaming\IDMComp\UltraEdit\runSAS.bat" "%f" "%p%n.log"
              This is the exact contents of the single line batch file:

              Code: Select all

              "C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\sasv9.cfg" -sysin %1 -log "%~dpn1.log"
              This is the output:

              Code: Select all

              "c:\Users\...\AppData\Roaming\IDMComp\UltraEdit\runSAS.bat" "C:\Users\...\Documents\File directory with a space\Test file.sas" "C:\Users\...\Documents\File directory with a space\Test file.log" ...
              
              C:\Users\...\AppData\Roaming\IDMComp\UltraEdit>"C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" -CONFIG "C:\Program Files\SASHome\SASFoundation\9.4\sasv9.cfg" -sysin "C:\Users\...\Documents\File directory with a space\Test file.sas" -log "C:\Users\...\Documents\File directory with a space\Test file.log"
              That actually produced a program run!

              6,601547
              Grand MasterGrand Master
              6,601547

                Jun 19, 2021#7

                It is really good that at least the solution with the batch file works now.

                I don't understand why the direct execution of sas.exe does not work on your PC. I tried to reproduce that issue with copying rar.exe with new name sas.exe into folder C:\Program Files\SASHome\SASFoundation\9.4\ created by me before. I copied next a file with name Test file.sas into folder C:\Users\Mofi\Documents\File directory with a space\ also created by me. I opened this file also in 32-bit UltraEdit v28.10.0.26. I added a tool configuration with the command:

                Code: Select all

                "C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" a -idcdp "%p%n.rar" "%f"
                I let the working directory field empty in tool configuration. I executed the configured "DOS" user tool while having in background running free Sysinternals (Microsoft) tool Process Monitor with a filter added with Path begins with C:\Program (with a trailing space). The captured output as displayed in UltraEdit output window was:

                Code: Select all

                "C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" a -idcdp "C:\Users\Mofi\Documents\File directory with a space\Test file.rar" "C:\Users\Mofi\Documents\File directory with a space\Test file.sas" ...
                
                Creating archive C:\Users\Mofi\Documents\File directory with a space\Test file.rar
                
                Adding    C:\Users\Mofi\Documents\File directory with a space\Test file.sas      OK 
                So everything worked as expected as I could see also in Process Monitor. I saved the Process Monitor log also into a CSV file attached to this post.

                I could see that 32-bit UltraEdit executed the 32-bit Windows command processor to run sas.exe being in real rar.exe with the command line:

                Code: Select all

                C:\Windows\SysWOW64\cmd.exe /s /c ""C:\Program Files\SASHome\SASFoundation\9.4\sas.exe" a -idcdp "C:\Users\Mofi\Documents\File directory with a space\Test file.rar" "C:\Users\Mofi\Documents\File directory with a space\Test file.sas""
                That is a 100% correct command line.

                I suppose your user account name always replaced by ... in all your posts contains a character on which this command line does not work with your full paths. It is unfortunately impossible to run all command lines with %ComSpec% /s /c "user tool command line" due to special parsing of the argument string(s) by cmd.exe after option /c.
                program_files_sas_execution.csv (7.51 KiB)   0
                CSV file with the file system accesses with a path beginning with C:\Program as recorded by Process Monitor on execution of sas.exe being in real rar.exe.
                Best regards from an UC/UE/UES for Windows user from Austria