SFTP on z/OS (MVS) with UE v19.10

tmb012345 · Jul 15, 2013#12013-07-15T12:38+00:00

Hardware
Windows 7 Based PC
zEC12 Mainframe

Software
Windows 7 x64
OMVS (Unix System Services)
z/OS MVS v1.12
Co:Z
UltraEdit v19.10

Problem
We recently implemented SFTP (Secure FTP via SSH) on our mainframe and in doing so we broke our FTP functionality in UltraEdit. Unfortunately, the last time it was working was under UE v10.10c and this was just FTP not SFTP. After talking with IDM and my coworkers we decided that we would try the new version of UltraEdit and see if there was any difference. There actually was but it still isn’t quite right.

The issue now is we can SFTP to our MVS side but the members of the PDS it shows are only those members which do not have any statistics associated with them on the mainframe. This is a problem because most of the files we are going to be editing with UE from the mainframe will have statistics. So far IDM has not been able to find a solution. Has anyone else run into this issue and if so what did you do to solve it?

Jul 30, 2013#22013-07-30T17:24+00:00

So after playing around a bit I think we have made a little progress. This applies only to v19.10.

If you go into the FTP Account Manager, click on the Server Tab and change the Name Index Option to 9 it will show you your data sets correctly if you are looking at "//XXX". However, when you change directory to a fully qualified directory like "//XXX.LIB.JCL" it displays a Window Logo icon. Now, if you leave that setting at -1 (this is default), and look at "//XXX" it will display the volume name as though it were the name. Now if you change directory to "//XXX.LIB.JCL" you will see all of your members listed and available for opening.

Why it works like this:
The Name Index is "used to specify the column number where the name of the file is stored. The default for this value is "-1" and this should not be changed unless you're sure of the value that should be used." So, unlike on the mainframe where each character is in a column, this value specifies the whole column as seen below.

Code: Select all

                      COL1      C2          C3    C4        C5   C6    C7    C8   C9      C10
7/30/2013 9:30:30 AM: Volume   Referred     Ext  Tracks    Used Recfm Lrecl BlkSz Dsorg  Dsname
7/30/2013 9:30:30 AM: PR3833  2013/07/29    1      15        1   FB   80    3120   PS    XXX.EOM.STATS

Now without seeing IDM's code I don't know why the default is -1 but you have to take that into account when specifying it. So instead of C10 we get C9 (C9 was just a educated albeit a lucky educated guess). If you compare the fully qualified listing below with the previous listing you will see why -1 works.

Code: Select all

                       C1            C2      C3              C4             C5
7/30/2013 9:42:45 AM: Name           Size  Created         Changed          ID
7/30/2013 9:42:45 AM: EOMSTAT

The fully qualified listing is totally different. Now the Name is C1 so the default works just fine if -1 implies "Whatever is in the first column." It indeed does work. It appears that UE is doing a full ls command (ls -l) and so gets back the attributes as well as the file name. What is needed is a way to send just the short ls command which gets only the file names.

Mofi · Jul 31, 2013#32013-07-31T05:46+00:00

I have not replied up to now to this topic as I'm having no experience with mainframes and their listings. I cannot give you a definite answer which works on your question regarding command ls, but maybe a little hint.

On tab Sever there is LIST Command Filter where the parameter(s) for list command can be defined.

What happens if you enter here for example * ?

Does this result in sending "ls *" or in sending "ls -l *" ?

tmb012345 · Jul 31, 2013#42013-07-31T12:17+00:00

First, I went and looked and on my accounts I already have setup the box for the LIST command Filter was greyed out and I couldn't enter anything so that right there is a clue that it probably won't work. However, I created a new account and was able to add the '*' to the filter. Unfortunately, it gave me the exact same display as before so I am figuring it sent the ls -l* command.

Mofi · Jul 31, 2013#52013-07-31T13:55+00:00

If you enable at Advanced - Configuration - FTP the setting Show FTP log in output window and open the output window before connect to the server, you can see all the commands UltraEdit sends to the server and what the server replies. This should help you to find out what is going on and which list command is used.

tmb012345 · Jul 31, 2013#62013-07-31T14:40+00:00

I did that. That is where the code blocks in my update came from. Using SFTP to connect I see the listing come back in the output window but no commands. Using FTP I do see commands and the one I saw was LIST.

UPDATE: I connected to our server once through FTP and once through SFTP (These are two different sides of the mainframe. FTP will go straight to MVS. SFTP will get to MVS via Co:Z via Unix System Services). This goes back to what I said earlier about the two different listings I receive when in the home directory (//XXX) and the fully qualified directory (//XXX.XXXXXX.XXXXXX). So here is what I found: Using FTP UE sends the LIST command both at logon and when you change to the fully qualified directory. The listing is different for both situations. Using SFTP, UE gives slightly different listings. See below:

FTP Home Directory: 7/31/2013 11:24:29 AM: Volume Unit Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
SFTP Home Directory: 7/31/2013 11:51:06 AM: Volume Referred Ext Tracks Used Recfm Lrecl BlkSz Dsorg Dsname

FTP Full Qual. Directory: 7/31/2013 11:25:31 AM: Name VV.MM Created Changed Size Init Mod Id
SFTP Full Qual. Directory: 7/31/2013 11:51:16 AM: Name Size Created Changed ID

We also found that the NLST command could be used to display what we want. However, the clients that we have only send this command after mapping the ls command to it. None of them will let us enter it directly from the command prompt. UE appears to send the full LIST command which gives back all the information and not just the names.

Mofi · Aug 03, 2013#72013-08-03T16:17+00:00

Do you have ever tried to find out which options list command on your server to which you connect with SFTP offers and if there is an option which returns just what you want to see in UltraEdit? This would give you the possibility to specify this parameter for the list command.

Sorry, I have just read "Note that some servers (VMS, MVS, etc) do not handle filter arguments like Unix/Linux servers." on UltraEdit help page FTP Account Manager in paragraph about Remote - Use LIST command. It looks like it is a known fact that for those servers the string entered under LIST Command Filter is ignored. So it is most likely a waste of time to find out which options the list command offers at all.

Have you ever tried the option Local - Perl regular expressions on the tab Advanced in the FTP Account Manager dialog?

UltraEdit does not support using another command for getting a directory listing as far as I know. It would be possible on server to replace list command file by nlist command file to get the listing you want in UltraEdit. But most likely you don't want to that "hack".

UltraEdit uses for FTP/SFTP/Telnet the library from WEONLYDO Software. Perhaps in their knowledge base or forum you can find some useful hints on your problem with the listing of the names, for example MVS server support.

tmb012345 · Aug 05, 2013#82013-08-05T12:26+00:00

Well, one of other SFTP clients, the one that is working right now and the one that shows the right output, actually sends the NLST command to the server instead of the LIST command. The NLST command only returns file names and thus parses out all the extra stuff. The interesting thing is, it does this on connect. It doesn't even bother using the LIST command. NLST is a better option when using MVS apparently. As for the Perl and Send after login options, these are both busts. Neither one worked for us. As of right now the best option would be to have UE send the NLST command instead of LIST but that isn't going to happen without IDM updating their software.

UltraEdit, UltraCompare, UEStudio forums