New York Internets Meet the Mainframe

The Internet is a powerful two-edged sword for many state governments. It provides a slick way for agencies to move public information out to taxpayers, but it can play havoc with the systems and workflow of state IS departments. Nowhere is the conflict more pronounced than in mainframe shops, where a state's most valuable public data often remains locked behind the technical legacies of older, bigger computers.

The challenge of connecting the World Wide Web to state legacy systems is part of an innovative project in New York state, where a group of state-sponsored researchers, agency IS managers and computer industry executives has been cooperating for the better part of a year to produce a set of models for how New York can best use the Internet.

"Agencies are looking for a way to get from here to there, and they see [Web-to-legacy technology] as a good possibility," said Sharon Dawes, director of New York's Center for Technology in Government (CTG), a nonprofit technology research group that is spearheading the project. "They've got to bring all those old mainframe systems along into the future, yet they can't afford to trash them all and start over again. Web-to-legacy looks like a middle-of-the-road way."

CTG is examining two Web-to-legacy approaches as part of its Internet Technologies Testbed project. In one, it is partnered with the consulting firm Deloitte & Touche and the State University of New York to explore methods of direct real-time Internet access to SUNY legacy databases. In the second, it is working with the New York State Education & Research Network (Nysernet) and the New York Department of Labor to study a mainframe-to-Web migration for a job bank application.

Project Goal

In both cases, the partners want to answer a simple but far-reaching question: Can New York state government use the Web as a universal interface for the delivery of all or most services to New York state citizens?

"It's not just a question about the Web," said Theresa Pardo, CTG's project coordinator on the Internet test bed. "It's the balance between the existing environment in agencies thinking about Web service delivery, the technologies available and the associated costs. There's a solution out there if you're willing to pay the price. The question is, what is the price that makes the answer always yes."

SUNY Central Administration

Partners: CTG, the Deloitte & Touche consulting group/DRT systems and the State University of New York

Goal: Establish methods of direct real-time Internet access to SUNY legacy databases.

For SUNY, the most important requirement was that the legacy database, which sits on an IBM Corp. mainframe, be maintained. "It doesn't seem practical to replace their entire infrastructure," said Ann DiCaterino, CTG's project support manager. "The system in place is too complex and too embedded to expect a very quick change; perhaps it is not even very desirable in the long run to expect to change that entire system from the mainframe down to a smaller-machine environment."

Background: SUNY Central Administration manages financial and accounting systems for the university and is testing real-time Web access to two applications: a purchasing system on an IBM mainframe and a procurement card system on a Computer Associates International Inc. Datacom relational database.

Both applications are transaction-based and permit users to read and update files on the legacy system. Before the test-bed project, users accessed the mainframe via proprietary channels: SUNYNet, the university's campus network, and IBM's CICS on-line transaction system.

The project plans to use the Internet to enlarge the user base for the mainframe and to make users' jobs easier by employing a Web browser as an interface. On an organizational level, the project will enable business transactions to take place at a lower level in the bureaucracy, which can reduce the need for a central business office to collect and enter transactions on the mainframe.

"The underlying business process will shift from being more centralized to being closer to the actual people who are transacting the business," DiCaterino said, "so there's more immediacy, less time wasted and more real-time understanding of the money in those accounts."

Approach: At SUNY, the partners decided to provide real-time access to the mainframe rather than adopt a copy-transfer or replication approach, which would mean moving batch files from the mainframe to more Web-accessible, smaller machines. Connecting to the mainframe provides users with immediate access to live data. On the other hand, it represents a tougher technical challenge than does Web-to-client/server technology.

"With batch, you are getting stale data," said Ron Schrimp, Deloitte & Touche's project manager on the SUNY project. "Our approach is to leave your current investment in place, introduce a framework which can bridge the two environments, and we can give you real-time data."

Tools: The test-bed partners are prototyping several middleware packages, including StormCloud Development Corp.'s WebDBC, Allaire LLC's Cold Fusion Professional, Bluestone Consulting Inc.'s Sapphire/Web and Oracle Corp.'s WebServer to write applications between the mainframe and the Web server. In general, middleware applications can be written in standard languages such as C or C++, or they can be written using general-purpose tools such as Sapphire/Web or Cold Fusion. Also, tools specific to back-end databases, such as Oracle or Sybase, are available.

Commercial middleware packages promise better application development time over standard programming tools because they are more user-friendly. The downside is that programmers can be locked into a vendor's development environment. Or worse, they still have to resort to standard tools. "Why would I want to buy a middleware product that I want to be able to plug and play and then still have to do C++ development?" Schrimp asked.

In fact, standard tools are all but a necessity in doing Web-to-legacy work. For example, many Web server developers are incorporating Common Gateway Interface (CGI) scripts, which let Web servers talk to middleware and other programs, in order to cut transaction time. The problem is, most Web servers do not offer connectivity to CICS back ends. So the partners are writing their own CGI scripts. But the transaction speed loss is marginal -- maybe half a second.

"In some cases where you have very high-volume transactions, you might need a more specialized, quicker development environment," CTG's DiCaterino said. "But in most applications, it's probably not going to have a big impact."

Platforms: On the hardware side, mainframe-to-server connectivity proved nettlesome. Upgrading to the current version of the mainframe operating system would have taken a minimum of six months. Instead, the partners designed "ingenious" but costly hardware workarounds to connect the older mainframe to Windows NT.

"If you're operating on an operating system that was designed back in the 1980s, it's going to be very difficult for you to talk over the line to a client/server box," said Jeff Older, a Deloitte & Touche technical architect on the project.

Again, a universal interface was desirable, but trade-offs were considered. "It's a matter of weighing the cost of the workaround vs. the cost of upgrading the operating system," Schrimp said. "You find out quickly that you either continuously upgrade versions of the mainframe so that when supporting client/server products come out they can easily communicate, or you have to come up with workarounds."

Conclusion: The bottom line is that the price of SUNY's universal interface was maintaining legacy, an approach best suited to the organization. "The nice thing about Web-to-legacy is that we can leave the mainframe alone, and they can continue to do business," Schrimp said. "The approach introduces change at a slower pace and still maintains the business need."

America's Job Bank

Partners: CTG, the New York State Education & Research Network (Nysernet) and the New York Department of Labor

Goal: Study a mainframe-to-Web migration for a job bank application.

Background: America's Job Bank, started 15 years ago as a way for state employment offices to swap job information, is also connecting its legacy systems to the Internet. But rather than linking the Web directly to the mainframe, the application will migrate more slowly from the legacy system to a smaller, Web-compatible platform. The case is a classic example of the Internet creating end-user demand that ultimately drives an IS strategy.

Under the legacy approach, a state sends a batch file containing its job openings to the America's Job Bank service center at the New York Department of Labor via a proprietary remote job entry (RJE) network. The file is then added to other state files on an IBM MVS system to produce a national file. That new file is then compiled in the state's data format, shot back through the network and made available at the state employment office.

The first step of the project was to permit end users -- both state employment officers and the general public -- to search the system via the Internet.

Approach: Create a parallel network more compatible with Internet and client/server protocols. To build this second network, the partners took the output of the compiled batch from the MVS system, copied it over on an Oracle database on a nightly basis and then provided Internet access to the smaller system so searches could be executed.

"When we did that, we almost immediately had a request that both job seekers and employers be able to get direct access to input job information," said Denis Martin, senior director of software engineering at Nysernet, the project's hosting and development partner. "Before, employers had to go to the employment service, fill out paper forms, and then someone would key it into the system."

At the same time, state clients expressed an interest in using the Internet as the transport method for putting batch data into the system. "It raised the obvious question: 'Gee, I'm sending you my files via this RJE network that's costing a lot of money. Couldn't we use the Internet to do that?' " Martin said.

Eventually, the solution was for the Oracle database to become the primary database, feeding the mainframe database overnight in batch mode so it could run other jobs against the data. "One of the things that we are doing is data coordination, where job orders submitted by employers to the Web system are being fed back into the legacy system," Martin said. "Before, it was legacy feeding the Web exclusively; now it's legacy feeding the Web and Web feeding the legacy. It's like a symbiosis."

Today the partners are working to take the system even a step further: allowing states to integrate data into the national system by taking the mapping and conversion functions that happened in the mainframe batch process and moving them out to the states. To do so, they are experimenting with software agents that embed data format and transport protocols into the server.

Platforms: The Web system is running on a Digital Equipment Corp. Alpha host running OSF. The Web server used is Netscape. Data is stored on an Oracle back end. As a mechanism for building applications, the partners developed a database application programming interface that allows them to connect to different databases in the back end. The system has support for Oracle, Sybase, SQL Server and Microsoft Access.

The system has a call spec that all the custom programs go to when searching for data, bringing it back, converting it to Hypertext Markup Language and displaying it to the user. The bulk of the code sits on the Web server as executable C programs.

Tools: All the applications were developed and written in-house, says Martin, whose team looked at a number of tools, including Sapphire/Web and Cold Fusion.

"We found that while many of them have very positive benefits and features, none of them specifically met our requirements of being able to do HTML to very customized C programming to a flexible back-end database," Martin said. Such products "offer a pretty dynamic development environment for generating C and C++, but they assume that you don't want to write it, nor can you. And we can and therefore have more flexibility on our code when we write it."

Conclusion: Ultimately, "the mainframe approach vs. the Web-based approach really started to parallel," Martin said. "These were, if not divergent, at least parallel tracks; these were not going to come together anymore. In fact, if anything, they were going to stay this far apart."

While use of the system has been soaring -- more than 200 new employers and 380,000 hits a day -- trade-offs are once again involved. The upside is that the project allows for migrating away from the legacy systems altogether. The cost is that, in the interim, nearly parallel legacy and Web systems need to be maintained. "The reality of all legacy systems is that as you start any change to them and migrate from them to something else, you have to maintain their integrity and functionality," Martin said.

Nevertheless, the impact on the job bank's business processes is classic Internet: End users win, midrange processes flatten, and management is ahead. "By putting in a Web front end, they are getting more access by job seekers," CTG's DiCaterino said. "People don't have to make a trip into the unemployment office. Now they can look at hundreds of jobs simultaneously and do a comparison. The job seekers have much more control over their future."

PAUL MCCLOSKEY is the editor of civic.com.