Cleaning up with SOAP

How can you get different languages and object frameworks to communicate over the Internet without using complex bridges? SOAP is a solution based on XML and HTTP.

Introduction

The web is a busy place. Every day new dot-coms roll out web applications, whilst behind the iron curtains of a myriad corporate firewalls key business applications make their way onto corporate Intranets. It may seem a beautiful picture of an expanding world of new software developments, but there’s a worm in the heart of the apple…

Whilst the web may be built on top of distributed applications and component architectures, every time a web developer creates a new application he may well be reinventing the wheel. How many times has an authentication and user tracking system been invented, and how many different payment gateways are there out there? Each fresh development is a waste of programming time and resources. Component technologies allow applications to be reused – both inside organisations and across the whole developer community.

If web applications are to be come widely used – and seen as a respectable method of application development – the same techniques are going to be required in web application development. Functions used in web applications need to be easy to exchange, and be able to support multiple applications. Luckily there’s now an answer: SOAP.

What is SOAP?

SOAP is the Simple Object Access Protocol. It defines a set of rules that allow HTTP and XML to be used to access services, objects and servers in a platform-independent manner. In a world of standards wars that often remind the outside observer of the religious wars of the Middle Ages, SOAP is perhaps the first truly agnostic component technology – able to link all operating systems, languages and component models.

HTTP and XML are widely accepted technologies. HTTP is the basis of the World Wide Web, and as a result is found just about everywhere – from 8-bit systems to PDAs to PCs and servers to mainframes. The same is rapidly becoming true of XML. A quick search on the web and you’ll find a plethora of XML parsing engines, most of which are free for use in you applications. Whether you use COM in Windows or Java on UNIX, there’s an XML parser and a set of HTTP servers and clients ready for you to use.

SOAP is based on the work done by Dave Winer of Userland, developers of the Frontier groupware and web content management system, and is an implementation of the proposed XML-RPC standard. SOAP defines a message format in XML that allows objects to communicate, as well as allowing remote creation and invocation.. At the time of writing SOAP 1.0 was an IETF draft document, and was on route to becoming an Internet standard. The SOAP 1.0 specification document describes it as “an RPC mechanism using XML for client/server interaction across a network using the following mechanisms: HTTP as the base transport, XML documents for encoding of invocation requests and responses”.

SOAP also has an advantage over other distributed application communication protocols – it’s easy for it to traverse firewalls. You can use any of the standard HTTP ports for a SOAP connection, so your connections can pass quickly through a pre-configured firewall, unlike the dynamic port assignations used by protocols like DCOM. If you want to make a secure connection, SOAP is designed to use HTTPS as well as HTTP. This means that you can use SOAP to implement secure e-commerce solutions, as well as making sure that sensitive information is transferred confidentially. Whilst it’s perhaps best considered as a tool for delivering request-response messages, SOAP can also be used to implement fire-and-forget style operations, acting as an event source for a remote object. Using this, you could use it to trigger a response in a billing system, or use it to monitor events in a distributed control system.

Using SOAP in your applications

Putting together a SOAP application is easy enough, and the published SOAP specification gives you most of the information you’ll need to get started with SOAP. You’ll also find useful information on the DevelopMentor website (http://www.develop.com/soap) and Microsoft’s MSDN Online (http://msdn.microsoft.com). The DevelopMentor site includes details of Java, Perl and COM implementations of SOAP.

Putting together a SOAP message is easy enough. If a method used by a SOAP invocation is known, then an XML message can be defined and used to request a response. The following code snippet is a SOAP message used to see if a software package is in stock, and how many copies are available, in an online store. The method used has been defined as:

So, to see if a copy of a specific game, say Quake, was in stock, an application would create an HTTP message as follows. The ApplicationNamespaceURI will be the XML namespace of the SOAP application on the server:

Sending this type of message is easy enough, given the number of HTTP components available. You’ll need an XML parser at the far end to work with the message, as well as to parse the response from the server. A standard response to the example message will be as follows:

This returns a successful HTTP response, indicating to the application that the operation has succeeded, and that the TCP connection to the server should be closed by the calling application. The return message must append the Response to the method element name, as well as adding a <return> element, containing the method’s return value. You can also use SOAP to return error messages, so that distributed applications will be able to garbage collect, as well as handle application errors. You’ll need to consider how to handle network errors and timeouts in your code, as these can be a problem for distributed applications, no matter how reliable the application code and the server…

SOAP can also transfer parameters to an application as well as methods values. You can define a set of headers that can be delivered at the same time as a SOAP call, which can then be parsed by the server. By using a SOAP:mustUnderstand element, you can control program flow – so that applications can ignore headers without affecting their operation. If mustUnderstand is set to a value of “1”, then your application has to parse the headers, otherwise the call must fail.

One key feature of SOAP is the methods used to submit messages. SOAP is designed to use both the standard POST HTTP method, as well as the newer and richer M-POST. M-POST is a new (and as yet only a draft before the W3C) HTTP extension that allows your SOAP applications to send headers that aren’t handled by a standard HTTP POST operation. M-POST is intended to allow firewalls to control the passage of XML and other more complex data-types than the standard HTTP types. You can use M-POST to make sure that the only data of type text/xml passed through a firewall is SOAP.

SOAP as a tool for a new type of ASP

One feature that SOAP brings to the table is the ability to create a new model of ASP. Application Service Providers are really a reinvention of the old computing bureaus, where applications were outsourced to third-party organisations. Turning them into web applications has increased their reach, and offered them to a wider audience – providing services for everything from large industries to the individual.

However, ASPs by their very nature are large, monolithic applications, designed to handle complex tasks. You’ll find them delivering office productivity suites, or project management, or even extending a desktop application well beyond the desktop. With organisations like SAP and Oracle developing ASP platforms, a key market is being ignored: the application developer. There’s big money to be made from large scale ASPs in monthly and yearly rental and access fees. However, it’s often the case that an organisation’s requirements are a lot simpler than a complex ERP application.

SOAP gives companies the ability to sell the functionality of the components they use on their web sites. With an XML description of the available interfaces, and the ability to accept XML data and deliver it, web applications can quickly become SOAP components in their own right. So what does this actually mean for the developer? If the promise of SOAP is anything to be believed it will be a whole new world.

A world built around SOAP makes things very different – and a lot easier. If you want to create an intranet web application that needs a currency exchange function, you could use the Financial Times’ web site as a component – or if you preferred Thomas Cooks’. Other applications could take advantage of functions included in Expedia or LastMinute.com, or even MSN and AOL. If you felt your site had functions that other developers would want to use in their applications, you could create SOAP descriptors and interfaces, and sell it to anyone who wanted to use it. One of the first sites to offer a SOAP component will be Microsoft’s Passport.com, which provided user authentication services to MSN and Hotmail. You will soon be able to access its authentication and payment wallet systems – stopping you from reinventing the wheel.

However, one big problem exists. How do you charge for a SOAP component? Three options exist: one-time charges, regular rental payments, and usage-based charges. A secure, and trusted, payment authority will be needed to encourage the use of SOAP in distributed applications. Companies like Component Source could move from being a way of buying COM components and Java Beans, to acting as brokers for SOAP web applications. There’s a big opportunity here for the first SOAP brokerage site, giving developers an online catalogue of components and functions, ready to be used in their applications.

Enterprise application integration in Visual Basic

SOAP isn’t just a tool for the web developer. It’s also a valuable tool for enterprise application integration – allowing large-scale legacy applications to be linked to new tools and user interfaces, simply by creating SOAP interfaces to existing processes, rather than hooking up complex EAI environments. Already SOAP tools and libraries have been developed for a wide range of different languages, from C++ to Java. The work originally done by the XML-RPC team has meant that there’s an existing body of software and components that could be quickly converted to support the final SOAP specifications. However, it’s Microsoft’s commitment to SOAP that’s very surprising.

I recently had a preview of an early alpha of the forthcoming Visual Studio 7.0 at a conference in Islington. Whilst Visual Studio 7.0 will be a significant release for Microsoft, as it will be their first development suite of for Windows 2000, it’s perhaps more significant for it’s overwhelming support for SOAP. Both Visual Basic and Visual InterDev will use SOAP as the main component communication technology, relegating DCOM to the background. Using Visual Basic 7.0 you will be able to connect to a site offering a SOAP component, pull in the details of the available interfaces and suddenly find it part of the object set available in the Visual Studio IDE. It’ll even be recognised by the IntelliSense code completion tools… Similarly, Visual InterDev will allow you to call SOAP components from server side ASP scripts in exactly the same way as they can call COM objects today.

SOAP is also used as the basis for Microsoft’s BizTalk XML EAI tool, which adds features that aren’t in the basic SOAP specification. These include Quality of Service and routing information. If you’re developing your own SOAP environment you can add this type of functionality by creating your own XML messages to be included in a SOAP message, taking advantage of the innate extensibility of any XML document.

This is an interesting approach for Microsoft to have taken, using a widely accepted open standard in place of RPC over the proprietary DCOM protocol – allowing Visual Basic applications to include components written in pure Java, or hosted in CORBA frameworks, or even existing solely as web pages. It’s also an excellent opportunity for web developers to start exposing key site functions as SOAP components. The use of SOAP in one of the mostly widely used development tools is not to be ignored, and it will encourage more developers to use distributed components in their applications.

It’s also a driver for the development of scalable and reliable web server environments. There’s no point in relying on a critical SOAP component if the server you’re connecting to is in fact a single P90 with a set of flaky network cards running an old copy of Slackware Linux that was badly installed by a sysadmin who left the company three years ago… Successful SOAP components that attract large numbers of users will need to be hosted on well designed web farms and application servers, running enterprise operating systems and built on carefully selected hardware and network components.

Is this the way ahead?

It’s easy to label SOAP as a panacea for all that ills the software development world. Certainly it promises much in terms of interoperability between different development languages and ways of working – as well as finally giving us a bridge between the worlds of COM and CORBA, and Visual Basic and Java. However it’s not as rich an inter-process communications model as DCOM or Java, and certainly could be a lot slower. There’s going to be a requirement for fast native XML parsers if SOAP is going to be a success!

However, the advantages look set to out-weigh the disadvantages. The ability to create distributed applications based on HTTP and XML is going to be a boon to the web developer, and the prospect of a profitable means building trusted ASPs operating and selling SOAP components will drive developments. SOAP is a very important development for the web applications architect. It’s up to us to make sure that it’s used correctly, and fulfils its promise.

Summary

• SOAP is a new standard for remote object access over the Internet
• It is based on XML and HTTP
• Implementation is straightforward, and relies on XML messages and HTTP headers
• SOAP will allow web applications to be componentised and distributed, and even sold in a new model of ASP.
• You can use SOAP to inter-operate DCOM, CORBA, Java, Perl – in fact any language with an XML parser and a set of HTTP tools.