Programmer to ProgrammerTM  
Wrox Press Ltd  
   
  Search ASPToday Living Book ASPToday Living Book
Index Full Text
 
ASPToday Home
 
 
Home HOME
Site Map SITE MAP
Index INDEX
Full-text search SEARCH
Forum FORUM
Feedback FEEDBACK
Advertise with us ADVERTISE
Subscribe SUBSCRIBE
Bullet LOG OFF
                         
      The ASPToday Article
January 7, 2002
      Previous article -
January 4, 2002
   
 
   
   
   
Sharing Data Across Apps Using XML Data Islands   Will Smith  
by Will Smith
 
CATEGORY:  XML/Data Transfer  
ARTICLE TYPE: Tutorial Reader Comments
   
    ABSTRACT  
 
Article Rating
 
   Useful
  
   Innovative
  
   Informative
  
 7 responses

When developing complex web applications using ASP, it is often necessary to access data that is stored in a separate application or on a separate network. The power of Microsoft’s new .NET framework and Web Services makes it easy to share data between applications hosted in separate environments, however, for many developers, upgrading to .NET may not be an option. Many organizations have spent lots of time and energy in rolling out Windows NT 4.0 or even Windows 2000 server installations recently, and as such, it willmay be some time beforeinstalling/upgrading to .NET.they are ready to upgrade again. In this article, we will explore the possibilities of using XML Data Islands and HTTP as a means of sharing data across multiple standard ASP applications.

   
                   
    Article Discussion   Rate this article   Related Links   Index Entries  
   
 
    ARTICLE

When developing complex web applications using ASP, it is often necessary to access data that is stored in a separate application or on a separate network. The power of Microsoft's new .NET framework and Web Services makes it easy to share data between applications hosted in separate environments, however, for many developers, upgrading to .NET may not be an option. Many organizations have spent lots of time and energy in rolling out Windows NT 4.0 or even Windows 2000 server installations recently, and as such, it may be some time before they are ready to upgrade again. In this article, we will explore the possibilities of using XML Data Islands and HTTP as a means of sharing data across multiple standard ASP applications.

The primary focus of this article will be to discuss one particular method for sharing data between separate applications using a variety of technologies including ASP, HTTP and XML. We'll start by examining the problem at hand and discussing the techniques used in uncovering a unique solution. Then I'll share some simple ASP code that will demonstrate these techniques and eventually work up to a more comprehensive solution that will hopefully allow you to move towards creating your own customized solution.

The Problem

Imagine a situation where there is some data that currently lives in a data store of some kind and is accessible to an internal application but not to any other. An example of this might be data that is stored in a SQL Server database that is behind a firewall or data stored in a legacy mainframe and therefore not accessible from outside it's internal network. The solution is to create an ASP page that will retrieve the desired data and return it as XML via HTTP to any application that wishes to use it. One advantage to this solution is that the data can be filtered by the application so that it has complete control over what data is sent back to the requesting application or client. This is in contrast to the alternative method of opening up access to the data source itself through the firewall, which can pose a great security risk.

Why XML?

When sharing data between applications, many options are available for deciding which format to deliver the data in. XML is a relatively new technology in the world of web development and is quickly becoming a standard for many web-enabled desktop and server applications. It is a standard proposed by the World Wide Web Consortium (http://www.w3.org/XML/?WROXEMPTOKEN=83465Zpwg63rNUDFHs6wAEhlVA) and as many of you know, is fully supported in many of today's most popular development environments, especially Microsoft's .NET Framework.

Sending data in a more traditional or legacy format such as a tab-delimited text file could achieve desirable results for our purposes here, however, the ability for XML to package data into a hierarchal format means that more complex data sets can be stored and delivered more effectively. This hierarchal format uses parent-child relationships as well as attributes-value pairs to facilitate easy migration of even the most complex data. This is in contrast to the more conventional methods of data delivery, which can force the developer to reduce data into rather archaic two-dimensional, 'row-column' formats, therefore potentially rendering the data useless. The full power of XML is still evolving. Other features such as using custom Data Type Definitions (DTDs) can even further enhance an XML document. With support rapidly growing for this emerging standard, it is easy to see why it is important to adapt to this new and powerful technology, which is sure to be around for some time.

What is a Data Island?

A data island is a self-contained document containing data formatted as XML. It is called an island because it is rather isolated in that, by itself, it doesn't do any good. Data is rarely stored as XML. XML is simply used as a packaging and delivery vehicle for the data. Once an application receives data in XML, it must be parsed or picked apart before it can be used. The process of taking this chunk of data and either storing it or displaying it involves parsing it according to XML standards or transforming the data using an XSLT Style sheet.

The MSXML Component

In order to process the XML, we will rely on Microsoft's MSXML Component. This free component is geared towards giving developers the ability to work with XML documents in a variety of ways. If you are unfamiliar with this component, especially the DOMDocument object, you can find documentation and even download the component from Microsoft at http://msdn.microsoft.com/xml/?WROXEMPTOKEN=83465Zpwg63rNUDFHs6wAEhlVA. We will use the DOMDocument object to load the XML data and eventually transform it to provide HTML output, which can then be sent back to the client. In addition to using the DOMDocument object for loading and transforming XML, we will use it to build new XML documents that an application can return to the client. The MSXML component also provides a great mechanism for sending/receiving messages to and from any HTTP compliant server. This functionality comes in the form of the ServerXMLHTTP and XMLHTTP objects. These objects are capable of sending and receiving HTTP messages. Using these objects, you can open a connection to a web server and request a document from the server. The object is then capable of reading any response data received as a result of the request. Although both objects perform virtually the same tasks, the ServerXMLHTTP object is used in a web server environment such as ASP. Therefore this article will concentrate on this object alone.

Now let's take a look at some sample code. While looking at these samples, it is important to keep in mind that we will be dealing with both sides of the transaction. One side involves the application where the data is stored, or the hosting application. The code here will be responsible for establishing a connection to the database, retrieving a subset of data and returning it as an XML Data Island. The second application will be responsible for making a request for the data, accepting the response from the first application then parsing the data to return it to the browser. We'll start by looking at a simple example of how the hosting application can access its data and return it as XML via HTTP.

Example 1 - Exposing Data Using ASP

Our first example involves gathering data from SQL Server in the pubs database and returning it as an XML Data Island via the ASP Response stream.

Option Explicit

'Print out the ContentType Header
Response.ContentType = "text/xml"

Dim Conn, RS, strXML
Dim RS_Titles
Dim SQL
Dim strTitleID, strTitle, strPrice, strDate

'Create and Open the Database Connection
Set Conn = Server.CreateObject("ADODB.Connection")
Conn.Open "driver={SQL Server};server=localhost;uid=sa;pwd=;database=pubs"

'Generate SQL Statement
SQL = "SELECT title_id, title, price, pubdate FROM titles ORDER BY title"

'Create and open the Recordset
Set RS_Titles = Server.CreateObject("ADODB.Recordset")
RS_Titles.Open SQL, Conn, 0, 1

If NOT RS_Titles.EOF Then
    
    'Start the XML Document
    strXML = strXML & "<?xml version=""1.0""?>" & vbCrLf
    strXML = strXML & "<data>" & vbCrLf
    
    'Loop through all titles
    Do While NOT RS_Titles.EOF
        strTitleID = GetString(RS_Titles(0).Value)
        strTitle = GetString(RS_Titles(1).Value)
        strPrice = GetString(RS_Titles(2).Value)
        strDate = GetString(RS_Titles(3).Value)
        
        'Write the current row
        strXML = strXML & "<row>" & vbCrLf
        strXML = strXML & "<title_id>" & strTitleID & "</title_id>" & vbCrLf
        strXML = strXML & "<title>" & Server.HTMLEncode(strTitle) & "</title>" & vbCrLf
        strXML = strXML & "<price>" & strPrice & "</price>" & vbCrLf
        strXML = strXML & "<date>" & strDate & "</date>" & vbCrLf
        strXML = strXML & "</row>" & vbCrLf
        
        RS_Titles.MoveNext
    Loop
    
    'Finish the Document
    strXML = strXML & "</data>"
End If

'Print out the XML
Response.Write strXML

Response.End

'Cleanup
Set elData = Nothing

RS_Titles.Close
Set RS_Titles = Nothing

Conn.Close
Set Conn = Nothing

Function GetString(x)
    If NOT IsNull(x) Then
        GetString = Trim(CStr(x))
    Else
        GetString = ""
    End If
End Function

First, we want to set the proper Content-Type header so that the calling application (be it a browser, or another application) will know that the response is indeed XML. The next step is to use ADO to access the required data and populate a recordset. From there we loop through the recordset and generate the XML to output through the Response stream. If there are no records, then no data is returned.

This code is effective in that it returns a valid XML document containing the data. However, this method of string concatenation is rather slow in ASP and is not recommended. That is why it may be useful to use the MSXML DOMDocument object to create the XML document on the fly. The following example illustrates just that.

Example 2 - Exposing Data Using MSXML

Option Explicit

'Print out the ContentType Header
Response.ContentType = "text/xml"

Dim Conn, RS, strOutput, objXML
Dim RS_Titles
Dim SQL, strSearch
Dim elData, elRow
Dim strTitleID, strTitle, strPrice, strDate

'Create and Open the Database Connection
Set Conn = Server.CreateObject("ADODB.Connection")
Conn.Open "driver={SQL Server};server=localhost;uid=sa;pwd=;database=pubs"

'Create the XML DomDocument Object
Set objXML = Server.CreateObject("MSXML2.DOMDocument")

'Create the Root Element (data)
Set elData = objXML.createElement("data")
objXML.appendChild(elData)

'Generate SQL Statement
SQL = "SELECT title_id, title, price, pubdate FROM titles ORDER BY title"

'Create and open the Recordset
Set RS_Titles = Server.CreateObject("ADODB.Recordset")
RS_Titles.Open SQL, Conn, 0, 1

If NOT RS_Titles.EOF Then
    
    'Loop through all titles
    Do While NOT RS_Titles.EOF
        strTitleID = GetString(RS_Titles(0).Value)
        strTitle = GetString(RS_Titles(1).Value)
        strPrice = GetString(RS_Titles(2).Value)
        strDate = GetString(RS_Titles(3).Value)
        
        'Add the row element
        Set elRow = objXML.createElement("row")
        elData.appendChild elRow
        
        'Add the data for this record
        AppendNewElement "title_id", strTitleID, elRow
        AppendNewElement "title", strTitle, elRow
        AppendNewElement "price", strPrice, elRow
        AppendNewElement "date", strDate, elRow
        
        Set elRow = Nothing
        
        RS_Titles.MoveNext
    Loop
End If

'Print out the XML
Response.Write objXML.xml

Response.End

'Cleanup
Set elData = Nothing

RS_Titles.Close
Set RS_Titles = Nothing

Conn.Close
Set Conn = Nothing

Set objXML = Nothing

Sub AppendNewElement(strName, strValue, elParent)
    Dim elTmp
    
    Set elTmp = objXML.createElement(strName)
    
    elTmp.text = CStr(strValue)
    elParent.appendChild(elTmp)
    
    Set elTmp = Nothing
End Sub

Sub AppendNewAttribute(strName, strValue, elParent)
    Dim elTmp
    
    Set elTmp = objXML.createAttribute(strName)
    elParent.setAttributeNode(elTmp)
    If NOT IsNull(strValue) Then
        elTmp.value = strValue
    Else
        elTmp.value = ""
    End If
    
    Set elTmp = Nothing
    
End Sub

Function GetString(x)
    If NOT IsNull(x) Then
        GetString = Trim(CStr(x))
    Else
        GetString = ""
    End If
End Function

The code here performs in much the same way as the first example. However, when it comes time to build the XML Document, we build the XML Document with the help of a DOMDocument object that we have created. We start by creating the new object, then appending a new root element. ( data) Then we use that element to append children elements ( row) as we loop through the data. Each row element represents a single record in the database. Note that this method of using the component to build the XML for us is much more efficient than building the XML string using string concatenation.

Now that we've seen two examples of exposing data as XML, let's take a look at how to actually get the data from a separate application and display it to the user. This code will live in the application that is requesting the data.

Example 3 - Retrieving Data From an ASP Page

Option Explicit

Server.ScriptTimeout = 5

Dim objRequest, objXMLSource, objXMLStyle
Dim strXML, bWorked

'Create the HTTP Request Object
Set objRequest = Server.CreateObject("MSXML2.ServerXMLHTTP")

'Genereate the Request
objRequest.open "GET", "http://myhost/data.asp", False

'Send the Request
objRequest.send CStr(Request.Form)

'We should have a valid document at this point

'Create the XML Source Object
Set objXMLSource = Server.CreateObject("MSXML2.DOMDocument")
objXMLSource.validateOnParse = False
objXMLSource.resolveExternals = False

'Load the XML into the Object
objXMLSource.load objRequest.responseStream

'Create the XML Style Object
Set objXMLStyle = Server.CreateObject("MSXML2.DOMDocument")
objXMLStyle.load Server.MapPath("sheet1.xsl")

'Transform the object
Response.Write(objXMLSource.transformNode(objXMLStyle))

Set objRequest = Nothing
Set objXMLSource = Nothing
Set objXMLStyle = Nothing

In this example, our first step is to create the MSXML ServerXMLHTTP object. This object will give us the functionality we need in order to request the document from the hosting application and process the response. We then call the open method to create a connection to the server, as well as tell the component to not connect asynchronously. The next step is to call the send method, which will actually send the request to the server. Since we called the open method asynchronously, it will not return until a response has been received, or the component has timed out. So it is generally safe to assume that a valid response has been received before continuing.

We'll create a new DOMDocument object that can be used to store the response XML in. The load method of the DOMDocument object will load XML from the supplied source, which in this case is a stream object from the ServerXMLHTTP object. After the XML has been loaded, we create a second DOMDocument object that will contain the style sheet used to transform the data. Then we call the transformNode function, which returns the result of the transformation as HTML.

Example 4 - Retrieving Data From a Secure Page

When exposing data in such a manner via HTTP, security can be a concern. Since the ASP page is served just as any other document on the web site, it is possible for anyone to simply access the file. The user would not be able to modify any data, but it would give the user a complete view of the data, assuming they knew the URL. If we wanted to secure the data, we could do so by implementing a Basic Authentication scheme using Windows NT/2000 and IIS. The following example illustrates ASP code that accesses an XML Data Island while providing security credentials.

'Generate the Request, not asyncronous
objRequest.open "GET", "http://myhost/data.asp", False, "username", "password"

This is as simple as passing a username and password to the open method. This username and password will be sent for requests requiring authentication. Keep in mind; this will only work with Basic Authentication schemes.

Example 4 - Using HTTP POST

The easiest way for us to get data from a remote web server is to simply request a known ASP document and receive the XML data as a response. However, there may be times where it is necessary to send some information to the host first, then retrieve the response based on the data sent. A good example of this is when you want the user to be able to search this data. The user would enter the search terms into a form, and then the code would be responsible for returning a valid set of results. The following code illustrates posting data to an ASP document using the ServerXMLHTTP component.

'Generate the Request, not asyncronous
objRequest.open "POST", "http://myhost/data.asp", False

'Add the Content-type Header
'Content-Type: application/x-www-form-urlencoded
objRequest.setRequestHeader "Content-Type", "application/x-www-form-urlencoded"

'Send the Request
objRequest.send "Search=" & Request.Form("Search")

The code here performs the same basic operations as the previous examples of Retrieving data, however the POST method is specified instead of GET and an extra content header is added to the request. The Content-Type header must be sent so that ASP will be able to handle the request properly. Without that header, the data would not be interpreted correctly, as the request would be mal-formed.

In addition, the actual data that is to be posted to the server is passed to the send method. This data should follow the HTTP standards, in the form of <name>=<value>&<name2>=<value2>&<name3>=<value3> and so on. The component will automatically add the Content-Length header, and set the value to the length of the data passed.

It is important to note that this component will also support the PUT method, which will allow you to send a complete XML document (or any text for that matter) to the remote server according to the HTTP 1.1 standards.

Real-World Example

ThinkGeek! (http://www.thinkgeek.com/?WROXEMPTOKEN=83465Zpwg63rNUDFHs6wAEhlVA) is an online retailer that sells a specialized line of clothing and merchandise geared towards a techno-savvy audience. Their ever-growing inventory of innovative products has forced them to devise new ways to inform their loyal patrons of their newest products as soon as they're available. With this in mind, they have published a regularly updated XML data file containing data on the 50 or so newest products, which are available through their web site. They encourage customers to use this file in conjunction with their own web sites to spread the word about their new offerings. Being a regular shopper of their fine web store, I decided it would be fun to integrate this data into my own web site. You can view the final product here: http://www.chee-tos.org/geek.asp?WROXEMPTOKEN=83465Zpwg63rNUDFHs6wAEhlVA. I've included the code along with this article.

Conclusion

Now that you are a little more familiar with the concepts discussed in this article, you should be able to enhance the code shown, and add some new features to meet your needs. One possibility would be to integrate a caching solution for larger data files. Modifying the HTTP Request and checking for 'fresh' content can accomplish this. If the file has not been modified since the application last retrieved it, it could simply use a cached version. Similarly, consider the possibility of storing a cached version that can be used when the host server is unavailable. Also, with the power of SQL Server 2000 and its support for returning XML from queries, it is possible to bypass a few steps and return a query result directly to the client.

 
 
   
  RATE THIS ARTICLE
  Please rate this article (1-5). Was this article...
 
 
Useful? No Yes, Very
 
Innovative? No Yes, Very
 
Informative? No Yes, Very
 
Brief Reader Comments?
Your Name:
(Optional)
 
  USEFUL LINKS
  Related Tasks:
 
 
   
 
 
       
  Search the ASPToday Living Book   ASPToday Living Book
 
  Index Full Text Advanced 
 
 
       
  Index Entries in this Article
 
  • appendChild method
  •  
  • Basic Authentication
  •  
  • Content-Type header
  •  
  • ContentType property
  •  
  • data
  •  
  • data islands, XML
  •  
  • definition
  •  
  • DOMDocument object
  •  
  • dynamically generating
  •  
  • HTTP protocol
  •  
  • load method
  •  
  • MSXML
  •  
  • MSXML component
  •  
  • Open method
  •  
  • POST method
  •  
  • problems with
  •  
  • Put method
  •  
  • Recordset object
  •  
  • Response object
  •  
  • security
  •  
  • Send method
  •  
  • ServerXMLHTTP object
  •  
  • setRequestHeader method
  •  
  • sharing data
  •  
  • system level security techniques
  •  
  • transformNode method
  •  
  • Write method
  •  
  • XML data
  •  
  • XML data islands
  •  
  • XML datasets, processing
  •  
  • XML documents
  •  
  • XMLDOMDocument object
  •  
  • XMLHttpRequest object
  •  
     
     
    HOME | SITE MAP | INDEX | SEARCH | REFERENCE | FEEDBACK | ADVERTISE | SUBSCRIBE
    .NET Framework Components Data Access DNA 2000 E-commerce Performance
    Security Admin Site Design Scripting XML/Data Transfer Other Technologies

     
    ASPToday is brought to you by Wrox Press (http://www.asptoday.com/OffSiteRedirect.asp?Advertiser=www.wrox.com/&WROXEMPTOKEN=83465Zpwg63rNUDFHs6wAEhlVA). Please see our terms and conditions and privacy policy.
    ASPToday is optimised for Microsoft Internet Explorer 5 browsers.
    Please report any website problems to webmaster@asptoday.com. Copyright © 2002 Wrox Press. All Rights Reserved.