Re: Sorting SMF records

> Small wrote:

><snip>

>     1. I tried to use IFASMFDP to check for invalid records.  That
> resulted in complete failure, since the program installed on the TK
> systems must be very old and has only one purpose (apparently), ...

><snip>

Can't find it currently, but believe I read that there were only 2 options, CLEAR or DUMP.  DUMP must include an automatic CLEAR afterward based on your experience.

> 2. I looked for an alternative on the CBT tape.  I found a newer
> version called SMFDUMP, howevr I wasn't able to assemble it, because 
> it is designed for OS/390 and up, requiring HASM, etc.  I could have 
> tried to adapt it for my purpose, but I decided not to try.

HASM alternatives:

www.z390.org
www.tachyonsoft.com
www.dignus.com

> 3. I looked for a FIXVBS program on the CBT tape, but couldn't find
> that name.  Instead I found COPYVBS, ...

><snip>

> This was interresting, because at first it seemed to work fine.
> (RC = 00, no ABEND).

><snip>
 
> When running this program on my 5,108  record archive it only copied
> about 28 records.  So now  had to figure out why it wouldn't go past
> that first track.  It took a while but I found that R10 was used as
> base for the input buffer, a well as some other area (which didn't
> change after the second EXCP, and therefore the program recognized
> EOF.  After making the base for that area R12, the program happily
> read and wrote all 5,108 records with no further problems.

Please send corrected copy of program to CBTTape (see bottom of www.cbttape.org) to contribute or update to CBT.

VBSFIX is in CBT file 316 along with SPANCPY and some other SMF stuff.

> 4. Now I tried the SORT again.  I found that the "CORE=" parameter
> yields an error message "IER067I - INVALID EXEC OR ATTACH
> PARAMETER".

'CORE=' has to have a value.  Your JCL below:

//S020         EXEC    SORT,COND=(0,NE),RGN=6144,
//             PARM='MSG=AP'

would need the following to allow SORT to use more than it's default genned 256K of memory to create the initial runs.

//S020         EXEC    SORT,COND=(0,NE),RGN=6144,
//             PARM='MSG=AP,CORE=6291456'

Your SORT PROC JCL below:

//SORT     PROC  CLS='*',RGN=4096
//*
//SORT     EXEC  PGM=SORT,REGION=&RGN.K,
//         PARM='MSG=AP'

would need the following otherwise SORT only uses 256K as genned for TK3.

//SORT     PROC  CLS='*',RGN=4096
//*
//SORT     EXEC  PGM=SORT,REGION=&RGN.K,
//         PARM='MSG=AP,CORE=4194304'

I have not attempted your jobstream or the utilization of a SORT proc as you have so there may be some snafu which you have encountered when you attempt to use the 'CORE=' parm.

> The DIAG parameter causes a lot of basically useless messages to be
> written to the log, and causes the S0C1 abend if any errors are
> found. If the DIAG parameter is not present the program ends with
> RC=16.

Normally you wouldn't use DIAG unless you were having problems.  The CORE utilization for the various passes of SORT are detailed in them however.  Increasing the size of SORT's CORE usage can shorten the run time of the SORT by creating larger initial runs and allowing fewer merge passes to take place.

> The last message "IER053A - OUT OF SEQ" is the culprit for the
> RC=16. By changing the SORT statement to properly reflect the SMF
> record fields of DATE and TIME, I was able to overcome this issue.
>   SORT  FIELDS=(19,16,BI,A,11,4,BI,A,7,4,BI,A)
>   END
> Was my original statement, copied from SYS1.SAMPLIB.  It's wrong!

The sample SORT in SYS1.SAMPLIB is setup to utilize exits E15 & E35.  You have dropped that from your sort.  If you run the SMFE15 & SMFE35 members of SYS1.SAMPLIB, then run the SMFSORT, you should not have a problem.  I didn't have 5,000+ records when I tried it, but it worked.

From z/OS V1R9 :
  http://publib.boulder.ibm.com/infocenter/zos/v1r9/index.jsp?topic=/com.ibm.zos.r9.ieag200/iea2g28060.htm 

//SYSIN DD      *
  SORT  FIELDS=(19,16,A,11,4,A,7,4,A),FORMAT=BI,SIZE=E4000
  MODS  E15=(E15,700,EXITLIB,N),E35=(E35,1500,EXITLIB,N)
  END
/*

which is the same as TK3 SYS1.SAMPLIB(SMFSORT).

><snip>

> 5. The SORT program had definite troubles reading records greater
> than 7K.

Can you elaborate.  A TAPE SORT permits record sizes up to around 24,000.

>  So I reblocked the archive to 7K on a 2314 volume.
> The sort program had no trouble writing large records, even larger
> the 32760. So I moved my sortout to a 3380, because using a 3350
> volume> caused some S002-0c, or S002-18 abends.

If you have your input on AWS or 2314 which can be zipped and emailed would appreciate a copy to test with. ( opplr limited to 10MB per attachment by hotmail.com )

> 6. Finally the sort ended reporting all 5,108 records had been read,
> and written!

Sounds like a good end result but jumping through hoops isn't so hot.

><snip>
 
> Thanks again for all your input, it is very much appreciated!

Thank you for posting your endeavors.  Many times we never hear from someone after they present a problem and solution(s) are proposed.

Jay Moseley's IEFU29 web page

www.jaymoseley.com/hercules/installmvs/iefu29.htm

mentions some other CBTTape SMF stuff;

File 94 SMF Dataset Audit Facility
File 523 General SMF Record Selection Program

Phil - 94 & 316 are distributed in the TK3 CBT DASD HLQ CBT429 - 523 is too new for TK3 inclusion