Websphere MQ 10th Birthday

SHARE Conference 2004

The event was held on 24th February 2004 at the SHARE conference in Long Beach California, to an audience of SHARE conference attendees interested in MQ. The album from this event can be accessed here.

During the event, Anthony O'Dowd gave the following talk.

Hello Everyone! It is my sincerest pleasure to stand before you this afternoon to introduce the formal celebration of the tenth anniversary of "WebSphere MQ" at SHARE in Long Beach.

As those of you have met me before will acknowledge, I am one of life's more cynical passengers; I am so cynical in fact, that in a crowded room packed with dozens of beautiful and intelligent people (cast hand over audience), I would merely claim lucky coincidence. Fortunately, there is no such need for cynicism today both because of the audience and the product we come to celebrate - WebSphere MQ.

We are gathered here today to celebrate the union of messaging and queueing in one of IBM's most successful new product families of the last decade. And although I would love to stand before you as the father of the bride, without any disrespect to my wife and testicles, I cannot claim such financially successful progeny. More humbly, I stand before you more as a friend who has been extraordinarily lucky to be asked to be best man, even if only for a few minutes, to celebrate the bride and groom that we know colloquially and simply as "MQ".

As with most success stories, under the surface MQ has a rather more interesting heritage than you might imagine. It all started a long time ago of course, over twenty years, if you really want to know. Although MQ has been developed over the last decade in the Hursley Labs in sunny England, it is, ironically, a rare example of an American idea successfully adopted and developed by the British. In some small way it is a friendly payback for the Jet engine, the computer and more importantly the English language, all of which were invented "over there", but so subtly developed and refined "over here".

The transatlantic origins of MQ are twofold. The first parent of MQ originated on the West Coast in California, where in 1980, Rob Drew, Bill Franz and Dick Divendorff were collared to do some Advanced Technology work when they were all in the Los Angeles area. They were funded by IBM Research who were trying to get some projects going to support the concept of a "System Memory", or a shared store that would be the hub for a cluster of loosely coupled machines. Over time this became the Sysplex. At the time, the "system memory" would contain a queue manager, and work would come into and out of task-specific processors. The AdTech team begged favours from many IBM labs, including Santa Teresa, Hursley, Raleigh and Poughkeepsie and established relationships which were later to prove vital.

Drew and Divendorff departed LA and got assignments with CICS in the technical planning group where they worked on the CICS dispatcher initially and then started a CICS cluster effort. A couple of years later, Graham Barber asked how long it would take to build a queue manager on CICS, to which Divendorff said "One year and one year to test it." (Nothing changes, eh?) This sizing was sufficiently large to discourage putting it into the CICS plan, so the concept of a separate stand alone messaging system based on the previous AdTech work was resurrected. Divendorff's previous involvement with the DB2 DSCF team provided a robust infrastructure from day one. His colleague, Mohan, from Almaden Research had great ideas on advanced logging and commit protocols from his work on the "Aires" project, which Dick was only too pleased to incorporate, given the slightly disappointing performance characteristics of the AdTech effort. Due to time constraints, the API was kept small and simple. Looking at the z/OS queue manager today, this history still shows through, giving the z/OS queue manager resilience beyond its tender 10 years.

At about the same time as Dick and company arrived in Hursley, a less well known, and seriously less glamorous second parent of MQ was struggling -- OS/2 Presentation Manager, whose mission was owned by Hursley because of it's GDDM heritage. Now PM had many faults, but there was nothing wrong with its heart, at which lay a messaging system. Many of you will know that at the core of every GUI lies messaging; it is the most effective way to get the disparate applications on your desktop to comunicate with each other. Out of the ashes of PM, came a product codenamed "HighNoon", which never really saw the light of day, but architecturally allowed the message processor to reside on any platform irrespective of where the message had originated. And prototype work performed on the remnants of "HighNoon" produced a core application programming interface and concept for cross platform messaging. Unfortunately, in the flavour of messaging APIs of the day, it was verb rich, and did not have the elegance of the small verb set that Divendorff's team thought so important. Divendorff's team fought hard, and won, to maintain this simplicity.

And with the merged mission of these two distinct parents, the two children, code named "Victory" on MVS, after Nelson's flagship, and "Mayflower" on distributed after the Pilgrim Brothers' flagship, had scoped MQ to roughly what we see today. A cross platform, messaging and queueing API.

However, IBM had two problems. The first problem was that Victory was significantly more advanced than Mayflower, who wanted to create a code base that was truly portable and high performing. So partnering with a company named SSI, IBM layered the MQI on top of an existing FTP program which was available on several platforms. The Victory team had a problem as well, which was there had been no requirement to communicate outside MVS. So, with the formats and protocols designed for communication the Victory team cobbled together the CICS mover to support the dominant transport protocol in 1993, LU62. Late December 1993 finally saw the General Availability of Message Queue Manager (MQM) version 1.1.1 for MVS/ESA, and the dream was reality. Later in 1994, IBM's distributed versions of MQ started to be released and the time taken over the reference port meant that both IBM and partners could rapidly grow platform support with full function and high performance. Messaging was about to become a first rate concept in IBM.

With these firm foundations, development has continued successfully for 10 years at Hursley Labs in England. MQ has seen significant functional and performance improvements, and today is rightly regarded as the industry leader. The Hursley team has worked hard over the last decade and will continue to do so to keep MQ ahead of the pack and serving its users. Over the last few years, we've seen messaging and queueing become the foundation for business integration. The Message Broker, code named "Argo" was created to give the vital structure to messages which is required to perform the effective routing and transformation necessary to integrate applications and extend the concept of any-to-any connectivity.

It is of course natural that in Britain the concept of queueing should be so readily embraced and cultured in an IBM program product. Britain is a country, remember, which gave the world its language, and which prides itself on its politeness, and therefore its ability to queue. (Mock) "After you", "No, after you, I insist", "Really, I say, how very kind of you." One could only have imagined what would have happened if the message queueing concept had been developed in, say, New York. What would have happened to polite, in-order delivery of messages? No - it is simply unimaginable.

And what about MQ's history at SHARE? It started as part of the Data Program in 1994 when Gary Ward presented high level overview and architecture concepts. Gary was instrumental in forming the MQ project as a separate entity in 1995, when he and Paul Williams did most available presentations. Then Stuart Jones became a stalwart presenter adding many of the more detailed technical sessions which users value so much at SHARE. He can't be here this afternoon, but asked me to say that once he did a whole day's presenting on his own. Ahhh, bless him - that's Stuart's idea of hard-work. More importantly, when MQ became its own project it always had the significant help of partners, such as Gary Ward from ADP, Ray Sun from Candle, Robin Wiley from Candle and Steve Nathan, who although with TelCordia for many years, I'm delighted to say has recently joined IBM in IMS level 2 support. (So get those ETRs in to Steve!) Sincerely though, we at IBM salute these folk for the significant help and advice that they have given us, and I'm sure will continue to give to help keep the MQ project a vital and thriving one. Moreover, this SHARE has seen the continuation of a Business Integration track in the MQ project, whose aim is to better align the portfolio of technologies required to achieve business integration. This integrated approach must surely help SHARE organize, and our SHARE members understand, the diverse range of technologies available to them.

And so we come to the reasons why MQ was, is, and will continue to be, successful. High on the list of technical success attributes must be simplicity and flexibility. At its core MQ allows users to connect different applications on different platforms to talk to each other using a simple messaging and queueing model irrespective of the underlying network protocol. MQ's simplicity and flexibility means that it is truly capable in the real world, heterogeneous environment in which our users live. Using MQ, users can get more from the applications they already have by effectively connecting them together, and moreover users are best positioned for ' any change to applications in the future. This change can range from a small enhancement of a departmental application to the kind of large scale reengineering seen in whole company mergers and acquisitions. MQ's simplicity and flexibility is appreciated by *all* of its users - the whole enterprise can use a common technical vocabulary based around messages and queues to specify, design and implement systems that really work well together.

While technical factors are always important, these are often not enough. IBM has been helped by several partners with the success of MQ. IBM realized very early that it could not do MQ all on its own, and by "beating the drum" (an early marketing theme) IBM and its partners, many of whom I'm delighted to see here at SHARE, were able to offer a full range of technologies and services to support IBM's MQ infrastructure. This included tooling for systems administration and monitoring, deploying the distributed reference port to platforms where partners had better skill than IBM, and services offerings to help enterprise architecture, design and implementation. This gave users a real choice and made MQ a de facto open standard. In today's world there is a lot of talk about run anywhere, but in 1994 MQ was the first major IBM program product, for example, to run on HP. And today, IBM and its partners, which over the years have included SSI, Candle, Willow, Hitachi, Unisys and others, have taken MQ and made it available on a staggering total of 39 platforms with high quality, high performance implementations.

So what happened to the original "Victory" and "Mayflower" design and development teams? From "Victory", Dick Divendorff left IBM in the mid l99Os to work for Microsoft where he still works. Rob Drew left IBM shortly afterwards to join Charles Schwabb, where he still works. Mohan still beavers away on astoundingly clever database technologies in IBM. This just leaves Dermot Flaherty who still works in IBM Hursley in the MQ Strategy department. From the distributed design side of the family, Taf Anthias left IBM in the mid 1990s and is now at Cisco, whereas Beth Hutchison although no longer part of MQ is still at IBM Hursley is working on Web Services. Of the original development teams on all platforms, only one remains on "Victory", and her name is, ironically, "Hazel Fix", which, if you know Hazel, is not an order you'd give to her face!

So what for the future? Well, it is vitally important for MQ to keep focussed on its users, as MQ comes from deep seated user requirement to perform effectively in a heterogeneous world. The message queueing architecture is simple and flexible enough for MQ to go from strength to strength - to allow users to integrate every single message oriented technology in a simple way. And that's the key I think. In a world of incredible complexity, MQ must keep itself simple for the sake of the users, and keep getting the message through. Focussing on solving the user's problems in a simple and efficient way will continue to keep MQ a key technology for the next decade and beyond.

Ladies and Gentlemen, before our "Weakest Link" quiz, let us wish MQ Happy Birthday in the time honoured way. (You'll sing better if you stand)

"Happy Birthday MQ
Happy Birthday MQ
Happy Birthday dear M-Q-
Happy Birthday MQ"