Developed a Data Mapping presentation for the VICS EDI conference

This was my first foray into public speaking. I was the Technical Lead for my company's EDI implementation, doing both supplier-side and retailer-side EDI. We ended up using the VICS EDI standard, and working on some of the standards committees for new document types they were working on. I ended up presenting on Data Mapping for two years and enjoyed it very much. (Got good reviews too!)

Found GNU/Linux in 1996: Props to Linus T. and Richard S.

What can I say? Linux changed my life. I needed to learn some basic Unix commands in order to support an application running on a Solaris server, and I thought that this here Linux thing I had heard about might be useful to help me learn Unix. I ended buying RedHat Linux (version 3.03) and installed it on a parts-box PC I scrounged together. I joined a local LUG back in the days when all the meetings seemed to be about figuring out how to get dial-up connections to our ISP's to work using PPP, or how to create a boot floppy with CD-ROM drivers on it. Install fests were the order of the day when it took the combined brain-power of 5 or more geeks to figure out how to get a PC to runlevel 5 without tearing your hair out. Sigh!! Kids these days just don't know how lucky they are: Why, when I was young, I had to carry my bits five miles just to get to the closest peering point. In the snow.

In all seriousness, coming from a Windows/Mainframe environment, the idea of Open Source software blew my mind. Being able to get my hands on programming tools like perl, which offered so much power with the immediate feedback of an interpreted language, really changed my professional life. As people developed ports of perl, python, and later ruby for the Windows platform, I found that more and more of what I learned on my home Linux system could be used to simplify and streamline all kinds of data extraction and manipulation tasks that I would have had to write Mainframe code to do (either in COBOL or using JES utilities like IEBGENR).

I have since put perl aside except when I am forced back to it. I like python and ruby a lot more for any substantial coding. Perl has two real problems for me: the lack of a clean syntax for object-oriented coding; and flattening of lists, which forces you to resort to references to make any kind of complex data structures.

Migrating Unix system to Mainframe, no doco: What to do?

I migrated a batch system written in shell scripts and Sybase stored procedures to Mainframe batch COBOL programs running under JCL against DB2 databases. (I know, this is not the normal direction for such migrations!) As part of this effort, I wrote a perl program to document the current shell scripts and stored procedures by creating hyperlinked listings of all the components. The top level was a list of the main batch shell scripts. Clicking on the script name would navigate to a listing of the script, with links to the shell scripts and Sybase stored procedures it called. Pages for the called scripts and stored procedures showed all scripts or stored procedures that called the current item. This allowed anyone studying the system to drill down to any desired level of detail and eased the conversion effort substantially.

IBM Report Writer not installed, Ohh Noo!

On the same system migration project, our COBOL installation lacked Report Writer, so all report layout and generation had to be done by hand.(No Crystal Reports for you!) I wrote a code generator in Python that let a user lay out a report in a text editor and then generated the COBOL code defining the working storage, boilerplate move statements to populate the report, and output statements needed to write the report out.

Upgrading to a new version of DB2: Do all our programs still work?

I had to figure out how to parallel test our migration from DB2 version 7 to DB2 version 8 on an AIX server running MicroFocus COBOL programs to do our nightly batch processing. Given a known starting point to restore from, we decided to capture the state of our tables after running all our programs against both versions of DB2 and then compare them to see if they were the same.

I wrote SQL to get the key columns for all our tables from the system catalog tables, then wrote a series of perl and bash scripts to:

• Generate DB2 export commands to export all our tables in key order based on the results of query against the system catalog tables.
• Run the exports for both version 7 and Version 8.
• Compare the results and report on the differences. (People did not grok diff output very well, so I had to dress it up a bit)

Finding orphans on DB tables where RI was not used.

Millions of rows in the parent table, and ten times that many children in multiple tables. How to find the orphans without making the lights go dim or tieing up the tables. (Using 'not exists' in a where predicate may not be the way to go.) I exported the keys for all the tables involved to flat files. I wrote a Ruby program to populate a Hash with the child keys, then deleted the parent keys from the Hash. Orphans are what is left.

Print 8000 pages a month and mail to 25 different recipients: Is there a better way?

Flat files for these reports were created on an AIX system, then uploaded to a Mainframe to be printed once a month. Accounting staff had to break out the reports and mail them out to different state agencies. I suggested and prototyped converting these files to PDF to eliminate printing and mailing costs. I used the iText java library to create a program to convert the flat files into PDF's complete with bookmarks. These reports are now burned to CD and mailed out.