Developing Content Negotiated Web Applications Using ASP and IIS.

ASP, IIS and Visual InterDev

Developing web applications used to be a complex task, writing code that read in CGI environment variables, and produced hand crafted web pages. Debugging your code could take a long time, as you went through each statement displaying results on either a web page or a UNIX system console. An electronic commerce shop-front could take weeks to develop, even by a skilled PERL developer. Even embedding code inside standard HTML pages using server-side-includes was a tricky process, opening up security holes in most systems at the same time.

The problems didn’t stop there, as users familiar with database-driven online services like AOL and CompuServe wanted a personalised online experience. This was difficult for early web applications to deliver, as the asynchronous nature of HTTP connections meant that preserving session and state information was a complex process, often involving the use of hidden fields in HTML form objects.

Time in Internet development moves very fast indeed, and over the last couple of years web server technologies have marched on at a terrific pace. Life for the application developer has been made a lot easier by the latest offerings from Microsoft and Netscape, which have full-fledged internal object models that can be used by built-in scripting languages. At heart, Netscape’s LiveWire and Microsoft’s Active Server Pages are very similar, containing tools for handling user sessions and connections to external databases, and both can be implemented using both companies dialects of JavaScript - though you may find VBScript easier to use with ASP.

It’s in the development tool stakes that things are rather different. Netscape’s commitment to cross platform development has lead to them producing their LiveWire authoring tool Visual JavaScript as a Java application, which is still rather slower than a native application will be - especially if you’re using Windows as a development platform. Even so, it’s a lot easier than hand coding applications in a text editor, especially when it comes to working with server side JavaBeans.

Active Server Pages were introduced in IIS 3.0, and combine two key parts of Microsoft’s Internet strategy:

• The first was Microsoft’s web server API, ISAPI, which was designed to allow developers to produce applications and DLLs that could plug into the IIS web server, and use the contents of input forms to modify the HTML delivered to web browsers. Web application pioneers Allaire with Cold Fusion and HAHT with HAHTSite soon took advantage of ISAPI, and produced web application environments based around it.
• The second was ActiveX, which was originally intended to be a Windows equivalent of the Java applet. A windowless version of OLE, ActiveX can be used to develop application components that can run on a web server, and can be called from within an ASP web application. As client side ActiveX was poorly received, due to security considerations, using ActiveX components to build server applications means that users don’t need to worry about intrusive downloads, and that web application developers can use existing Visual Basic and Visual C++ skills to develop tools to help develop web applications.

Microsoft have built their web application development system into the Visual Studio tool suite. Visual InterDev has something of a chequered history, first appearing as “Blackbird”, and designed to develop interactive content for the then proprietary Microsoft Network. MSN quickly moved over to web technologies, and Blackbird followed, now renamed Internet Studio. Public betas appeared in late 1996 with demonstrations of impressive database driven extranet applications, and InterDev was finally given its name in the “Boston” betas of Visual Studio.

If you’re going to develop Active Server Pages web applications, then InterDev is definitely the tool for you. Built into the familiar Visual Studio IDE, it gives you a framework in which you can build ASP web pages using VBScript or JavaScript. A series of “Design Time ActiveX Controls”, which are reusable wizards in all but name, help you build database connections, by giving you an Access-like view of any ODBC-compliant SQL database. You can then build queries using drag and drop techniques, before testing them and the embedding them in calls to Advanced Data Objects (an update of the familiar RDO and dbOLE technologies).

Visual InterDev also includes full documentation of the Internet Information Server scripting object model. This includes a large number of tools that help you produce personalised web pages, from session objects that allow you to maintain state between pages for your users, as well as tools for reading and writing cookies, or using URL encoding to pass information in the URL of a page, so that you can use ASP applications together with higher end web applications development tools like the Java-based NetDynamics or even with existing CGI programs and scripts.

Unfortunately, there’s little or no debugging facilities in the current release of Visual InterDev, 1.0. However, this will change as Microsoft start to bring out their next Visual Studio tools (and give everything the same version number - meaning InterDev will jump from version 1.0 to version 6.0!). The first downloadable betas of InterDev 6.0 are now out, from http://www.microsoft.com/vinterdev/, and include documentation for the IIS 4.0 object model, which adds transactional database access, and also add server-side debugging tools that will work with the latest releases of Microsoft’s ASP scripting engine.

Using InterDev, you can work with the Internet Information Server object model to design and develop ASP-based web applications that can implement user sessions

Content Negotiation: personalising web sites.

Web applications and sites are notoriously static, and the connectionless HTTP protocol used to deliver HTML pages does nothing to help. Each connection to a server by a web browser is seen as a separate transaction, operating in isolation from all previous and future connections, and often spawning off a fresh process.

Users tend to have a high expectation of web sites and applications, and expect to see a site that is tailored to their needs, and to their web browser. A site designed to look its best for Netscape Navigator 2.0 will look primitive when viewed in the current batch of 4.0 browsers, and a site designed for a 4.0 browser is often impossible for an earlier tool to display. You can find examples of content negotiated sites at www.microsoft.com, where pages are tailored to browser type and at MyYahoo, where the news stories are controlled by a user profile.

The solution to these problems is content negotiation. By using server-based dynamic page generation techniques, pages can be delivered that are tailored to pre-set preferences (or even to take into account how a user has used a site before), or that change in response to a user’s actions as they interact with a site. It is possible to produce content negotiated sites using traditional CGI-techniques, but these can be slow (even if you use server-side includes rather than external CGI programs). To get the best performances, you need to use the latest server scripting techniques. Content negotiation should always be transparent to the user, and in the best examples, users will only ever need to log into a content negotiated site once.

You can develop a content negotiated site using high cost tools like Inso’s XML driven DynaBase, or using Vignette’s StoryServer site content management tool - or even by publishing Lotus Notes databases via Domino. However, if you’ve got a copy of IIS 3.0, then the tools for building an effective and fast content negotiated web application are built into Microsoft’s ASP technology.

So what are the typical uses of content negotiation, and how can you implement them using Microsoft’s ASP technology? Three possible solutions are detailed below, showing how you can tailor content for web browsers, how you can personalise a web site, and how to deliver appropriate responses to incorrect HTML form submissions.

Spotting the browser.

Even though the pundits say that the browser wars are over, and that Microsoft have won, there are still a lot of different web browsers at large out on the net. A quick visit to any site that logs and displays the browsers visiting will show a wide range of different web browsers, most of which have very different capabilities - a good example can be found at http://www.ukonline.co.uk/public-bin/brlog/read. Even popular browsers like Netscape Navigator differ from version to version, with different object models and HTML rendering engines. There are still a lot of copies of Netscape Navigator 1.0 and Internet Explorer 2.0 in use, even though they don’t offer the latest features. However, they are small, and fast – unlike the latest huge downloads.

If you’re planning on developing a site that needs to work with a lot of different browsers, and must deliver a consistent page layout and navigation system to each of them, whilst at the same time offering a site that looks good in the latest tool, then you’ll need to use some form of content negotiation to deliver the pages to the browsers. ASP offer tools that will help you deliver the correct HTML to browsers. Using these techniques you can produce browser-tailored sites without investing in expensive content management tools like Vignette’s StoryServer or Inso’s DynaBase.

The first, and most important of these is the Browser Capabilities Component. You can use this to see what each browser that visits your application is capable of displaying, from frames and tables to JavaScript and VBScript - and even if users are coming in from AOL!. Composed of two parts, the BROWSCAP.INI file and BROWSECAP.DLL, you can modify the BROWSCAP.INI to add new browsers, so you can tailor content for new services like Web TV. Each entry in the INI file is defined by an HTTP_USER_AGENT string. These are a standard string delivered by the browser to a server during an HTTP transaction, and can be used to detect the browser used by the BROWSCAP.DLL server object. A typical INI file entry is shown below:

This is for a major version of a browser. You can also define capabilities for minor versions and variants, which inherit all the capabilities of the parent definition, but save space in the INI file only by showing changes and differences. It’s a good idea to define an entry for a default browser, and make all the properties False, to prevent problems with unidentified tools!

Getting information from the BROWSCAP.INI file in an ASP application is easy. All you need to do is create an instance of the Browser Capabilities Component:

Unlike most ASP objects, the Browser Capabilities Component has no methods or collections - just the properties you have defined for it in the BROWSER.INI file. You can then operate on it to determine which capabilities are available by querying the object. If a property isn’t defined, then the default return is always False. You need to explicitly define any features you wish to be used by your ASP applications. So, if you want to test if a browser supports JavaScript, as you may be about to use client-side form content checking or want to script an ActiveX control, just add an ASP script section to your page:

You can then use the Browser Capabilities Component with ASP to produce conditional blocks which determine what content is delivered to each browser. Designers can then produce page templates for you that you can include in your application, delivering the best possible layout for each browser.

A web site of my own: personalising the web application.

Content negotiation isn’t just for handling browsers. You can also use it to personalise pages in a web site, and to control how users work with an application. Using these techniques a news application can deliver a “Daily Me”, where each story fits a users’ profile, or an Intranet application can indicate whether important business information has been read or not. You can also use content negotiation to display the contents of  shopping basket as a user passes through your electronic commerce applications

ASP makes developing personalised pages easy (and even easier if you purchase Microsoft’s Site Server package, which includes a personalisation server and a commerce engine). You can use the database integration to store user preferences between sessions, whilst client-side cookies can be used to store user IDs in a users web browser. During sessions, ASP uses server side session objects linked to browser session cookies (which are destroyed when the user shuts their browser down) to handle the transfer of large amounts of data between the pages of an application.

Microsoft’s server object model contains two objects, with methods that allow you to read and write cookies. You can read the contents of a cookie using the request object, and can write a new cookie using the response object. It’s important to remember that cookies can only be written at the start of an ASP page - before the HTML parser in the server reaches the <HEAD> tag - though they can be read at any point in a page. Whilst traditional server side applications could only run in the body of a page, you can create ASP applications that complete running before any page content is displayed by implementing all server side code before the <HEAD> tag.

The session object is one of the simplest and most powerful tools in the hands of an ASP application developer. Using this large amounts of information an be passed between the various pages of an application, without data being transferred in hidden fields in web pages (which can add significant bandwidth overheads).

Once created, a session object is bound to a user session, and has a limited life-span. As session objects use server memory, periodic garbage collection is needed to prevent memory from being permanently allocated to expired sessions. You can read and write to a session object as if it were a keyed list, and once created data will persist until deleted or updated, or the session object is destroyed.

Ooops-a-daisy! - controlling user input from the server.

Building a web application is more than producing forms, and linking them to back end systems. One major problem for the application developer is how to handle error conditions. Often web applications just echo back the error statement, with little or no explanation, leaving users frustrated.

Using ASP it’s possible to trap errors by examining the output of a database query before displaying it to the user. The following code snippet is part of a page that processes data submitted through a form on the previous page.

If no data is returned, the user is shown the form again. You can pre-fill it with the data already submitted if you want. If a response has been received, the user is shown an alternate form, in which they can pick a dynamically generated selection from a drop down list.

You can use this technique to build complex applications in a single page, where a user is guided to producing a valid query before being shown a results set. When you are dealing with large database driven applications, where there may be several million data elements, using ASP in this way can help reduce search times, as well as reducing load on your web server and databases.