Wrox Press
ASPToday
       933 Articles
  in the Solutions Library
  Log off
 
 
 
ASP Live!  
 
ASPToday Subscriber's Article Péter Csáki Péter Holpár
Using HTTP Modules to Map Addresses in a Portal System
by Péter Csáki, Péter Holpár
Categories: .NET Framework, Site Design
Article Rating: 4.5
Published on August 29, 2002
 
Content Related Links Discussion Comments Index Entries Downloads
 
Abstract
The role of HTTP modules in the .NET world is parallel to the function of the ISAPI filters of the pre-.NET age: they can preprocess HTTP requests and post-process HTTP responses.

In this article we will introduce HTTP modules and their usage by creating a URL mapping solution for a portal system. This solution will enable content managers of a site to specify user-friendly virtual URLs (such as http://www.latestnews.com/menonmars) for visitors to access long, hard-to-remember, query string based URLs (http://www.latestnews.com/default.aspx?EDRef=53&NewsRef=3244). In the past, one could implement similar applications developing ISAPI filters that were the territory of hardcore Visual C++ or Delphi developers, and were hard to debug due to their architecture.

Now, Visual Basic programmers are able to achieve the same result using VB .NET and the .NET Framework class library. After reading this article you can start to design and implement your own HTTP modules to enhance your site with cool solutions that were unimaginable and unreachable for the Visual Basic programmers before .NET.

Besides the HTTP module sample we provide a simple administrative tool to manage the URL mappings and the content of the cache.
 
 
Article

Introduction

A common defect in several portal systems is that their pages can only be addressed using complex URLs that contain complicated query strings. Most often a single .asp or .aspx page does the job by using different parameters in the URL. Just have a look at the online .NET sample IBuySpy Portal (http://www.ibuyspyportal.com/?WROXEMPTOKEN=809724Z8y2lKNrebv4PObUjYYy) where DesktopDefault.aspx is used to display most of the portal pages while the tabindex and tabid parameters in the query string specify exactly what content should be shown to the visitor.

Imagine a fictitious news company, LatestNews, has an Internet site - http://www.latestnews.com/?WROXEMPTOKEN=809724Z8y2lKNrebv4PObUjYYy - and a regular TV news program. The company would like to attach a unique, easy-to-remember URL for the main news discussed on the TV and show this URL on the screen at the end of the item. For example, in the case of a successful Mars expedition, this URL would probably be http://www.latestnews.com/menonthemars?WROXEMPTOKEN=809724Z8y2lKNrebv4PObUjYYy.

The portal system is similar to the one described above, so the real URL may be something like this: http://www.latestnews.com/default.aspx?EDRef=53&NewsRef=3244&WROXEMPTOKEN=809724Z8y2lKNrebv4PObUjYYy. The managers of the news company worry that this format is too difficult for an average viewer (wonder why?). The news editor should be able to maintain the mapping between virtual and real URL addresses, and it may be necessary to disable the direct, query string based access to the pages, and to allow only the usage of virtual URLs to hide the internal working of the web application from visitors.

In this article we will show you how to implement a .NET-based solution for this problem. We will use HTTP modules to achieve this goal, so the article starts with a short description of the theory of HTTP modules. After this, we will extend our base HTTP module to fulfill the requirements of the URL mapping system. We will show you the application applied on the IBuySpy Portal sample and the administration page that you can use to maintain you mappings. Finally, we provide detailed instructions about installing and configuring the solution in you environment.

System Requirements

To install the sample application you will need the following software components on your system:

  • Windows 2000 Server/Advanced Server, Windows 2000 Professional or Windows XP operating system
  • MS SQL Server 2000/ MS SQL Server 7.0 or MS Data Engine
  • .NET Framework (+ .NET Framework Service Pack 1) and Visual Studio .NET
  • Internet Explorer 5.0 or higher

To install the IBuySpy Portal sample we used to test the application, the requirements are a little bit stronger for the database server and for the browser:

  • MS SQL Server 2000 or MS Data Engine 2000 or higher
  • Internet Explorer 5.0 or higher

To fully understand the code you should have at least basic knowledge of:

  • Visual Basic .NET
  • ASP.NET
  • ADO.NET
  • SQL

Introducing HTTP Modules

The role of the HTTP module in the .NET world is parallel to the function of the ISAPI filters of the pre-.NET age (similarly, HTTP handlers are parallel to ISAPI extensions). If you are familiar with ISAPI filters you know that they can preprocess HTTP requests and post-process HTTP responses. HTTP modules extend this model so you can register your HTTP module to be triggered by events of an HTTP application. An instance of the triggering HTTP application is passed to the HTTP module, so you can access and modify its properties, including the Request and Response objects that are members of the HttpApplication.Context object.

The figure below shows the schematic HTTP pipeline of processing an HTTP response and passing back an HTTP request.

Developing ISAPI filters requires you to have knowledge of relatively low level programming languages, like C++ or Delphi, as the architecture and threading model of Visual Basic were inadequate to perform such tasks. The advent of the .NET Framework blurred the difference between the programming languages, so - of course - it is possible to use Visual Basic .NET to implement an HTTP module. To demonstrate this we chose this language to create our URL mapping application.

We don't find comparison data about the performance of HTTP modules vs. performance of ISAPI filters, but ISAPI filters should provide slightly higher performance on the same hardware and software environment. Since HTTP modules are implemented over an additional abstraction layer, namely the HTTP runtime of ASP.NET, each request must pass through the aspnet_isapi.dll ISAPI extension.

Debugging HTTP modules is easy (similar to debugging ASP.NET pages) in contrast to ISAPI filter bug-hunting, that usually requires your debugger application to be able to attach to a Windows process (the integrated debugger of MS Visual Studio is capable of doing this). Since ISAPI filters run very tightly integrated with system processes, a badly written ISAPI filter may cause your web server and even the operating system to crash, this is certainly not the case with the .NET HTTP modules.

Creating the Frame of an HTTP Module in Visual Basic .NET

HTTP modules should implement the System.Web.IHttpModule interface. This interface has two methods that you must implement:

  • Sub Init(ByVal Application As HttpApplication): You should register the event handlers in this method.
  • Sub Dispose(): You should place the clean-up code into this method.

The skeleton of the HTTP module should be something like this:

Imports System.Web

Public Class SimpleHttpModule
  Implements IHttpModule


  Public Sub Init(ByVal Application As HttpApplication) Implements IHttpModule.Init
    ' register the BeginRequest event
    AddHandler Application.BeginRequest, AddressOf Application_BeginRequest
  End Sub

  Public Sub Dispose() Implements IHttpModule.Dispose
    ' do the clean-up
  End Sub

  Private Sub Application_BeginRequest(ByVal Source As Object, ByVal e As EventArgs)
    ' casting the Source to an HttpApplication object
    Dim Application As HttpApplication
    Application = CType(Source, HttpApplication)
    ' get or modify HTTP application properties as adequate
  End Sub

End Class

Configure Your Web Application to Use the HTTP Module

You can register your HTTP module in the system.web section of the web.config file using an <add> directive.

<add name="ModuleName" type=".NET 
Class, Assembly [,Version=version number] [,Culture=culture] [,PublicKeyToken=token]" />

Where the name is a friendly name for the module and type refers to a comma-separated class/assembly combination. The assembly dll should be in the private \bin folder of the application or in the assembly cache.

Changing Application Mapping on the Web Server

Another, maybe less known, difference with the ISAPI filters is that, by design, HTTP modules are notified only if files with extensions associated to the aspnet_isapi.dll - like .aspx - are requested by the user. For example, if the URL request refers to the simple folder name of /portal/discussions the request will not be handled by the HTTP module. Since we would like our module to monitor these kinds of requests too, we have to make some changes in the configuration of the web server. The necessary configuration steps are detailed in the IIS setup section of this article.

You should be aware that by making these changes, you also change the way IIS handles default documents. To be exact, IIS will not find the default documents for you if you specify only the folder name. You must specify the full path of the document in your browser. A workaround to this problem is to make a URL mapping for the default document that maps the URL of the folder to the full URL of the default document. We will describe how to do this, later.

The URL Mapping Application

In this section we will show you the URL mapping HTTP module and the database used to store the mappings.

The Database

We don't need a very sophisticated database in this application. To store the mappings of the virtual and real URLs and the caching interval for the mapping the following simple table is enough:

We will store the following values for each URL mappings:

  • cVirtualURL: The URL as it is typed in the browser (for example, http://localhost/PortalCSVS/Discussions). By storing the site name as part of the URL it is possible to use a single database table for multiple sites.
  • cURL: The real URL relative to the web site (i.e. without the leading http:// and site name, for example: /PortalCSVS/Default.aspx). If you would like to hide the URL specified in the cVirtualURL, use an empty string in the cURL field.
  • nCachingInterval: An integer value to store the individual caching interval for the URL mapping. If it is a negative value, the default caching interval specified in the web.config file will be used.

We will query the database using simple SQL Select statements through ADO.NET from the HTTP module.

Caching

Caching is important to remove any unnecessary burden from the database server by storing already resolved and known mappings in memory on the web server instead of querying the SQL server for each web request. This is especially useful if your web server and database are running on different machines. We will implement our caching system using the Cache object of ASP.NET.

On the other hand, if you specified a too long cache time for an item, browsers would be redirected to the cached URL, even if you modified this mapping in the database in the meantime. The administration page we provide you, gives a solution to this problem so you shouldn't disable or clear the entire cache. For the details of this solution see the Using the administration page section later.

Extending the Skeleton

Let's examine how we should extend our HTTP module skeleton to develop the URL mapping applications. We should modify our Application_BeginRequest method.

First, we create references to the most often used objects:

    Dim Application As HttpApplication = CType(Source, HttpApplication)
    Dim Context As HttpContext = Application.Context
    Dim Request As HttpRequest = Context.Request
    Dim Response As HttpResponse = Context.Response
    Dim Cache As Caching.Cache = Context.Cache

After this we read the requested URL (SplitSlashes is a function that removes extra slashes from the end of the URL):

    cURL = SplitSlashes(LCase(Request.Url.AbsoluteUri))

Then we read configuration parameters from the web.config file:

      Dim AppSet = Context.GetConfig("appSettings")
      cConnStr = AppSet("URLMapperConnectionString")
      nDefaultCache = AppSet("DefaultCachingInterval")
      bCachingEnabled = (LCase(AppSet("CachingEnabled")) = "true")

If caching is enabled, we try to look up the URL in the cache:

      If Not (IsNothing(Cache(cURL))) Then

If default is stored in the cache, we do not modify the request. In this case the server tries to fulfill the original request.

        If (Cache.Item(cURL) = "default") Then
          Exit Sub

If disabled is stored in the cache, we send back an HTTP not found status, so users will experience a file not found message.

        ElseIf (Cache.Item(cURL) = "disabled") Then
          ' set HTTP status to not found (404)
          Response.StatusCode = HttpStatusCode.NotFound
          Response.End()
          Exit Sub

Otherwise we modify the path for the request to the cached one:

        Else
          ' redirect URL and finish
          Context.RewritePath(Cache.Item(cURL))
          Exit Sub
        End If
      End If

If caching is disabled, we remove all items already cached by the application. We shouldn't remove any item that is cached by the system, so we apply a filter:

      For Each CacheItem In Cache
        If (Left(CacheItem.Key, 7) <> "System." And Left(CacheItem.Key, 19) <> "ISAPIWorkerRequest.") 
Then
          Cache.Remove(CacheItem.Key)
        End If
      Next

If the requested URL is not found in the cache, we should query our database. First we check if the mapped URL is an empty string. This would mean that the direct access of this URL is disabled.

      DataAdapter = New System.Data.SqlClient.SqlDataAdapter
("SELECT nCachingInterval FROM URLMapper WHERE cVirtualURL = '" + cURL + 
"' AND cURL = ''", cnn)
      DataAdapter.Fill(ds)

If the dataset is not empty, it means we have a matching record in the database, so we should deny access to the requested page. If caching is enabled, we store this result in the cache too. The caching interval is from the database and if it is a negative value, the default caching interval is used.

    If ds.Tables(0).Rows.Count > 0 Then
      If bCachingEnabled Then
        nCacheFromData = ds.Tables(0).Rows(0)("nCachingInterval")
        ' use default caching interval if negative
        If (nCacheFromData < 0) Then nCacheFromData = nDefaultCache
        ' add item to the cache
        Cache.Add(cURL, "disabled", Nothing, Now.AddSeconds(nCacheFromData), 
Cache.NoSlidingExpiration, Web.Caching.CacheItemPriority.Normal, Nothing)
      End If
      ' set HTTP status to not found (404)
      Response.StatusCode = HttpStatusCode.NotFound
      Response.End()
      Exit Sub
    End If

The next step is to check if are other (non empty string) mappings for the requested URL. If there are, we should redirect the request to this file. We cache the result if caching is enabled. The caching interval is from the database and if it is a negative value, the default caching interval is used.

      DataAdapter = New System.Data.SqlClient.SqlDataAdapter
("SELECT cURL, nCachingInterval FROM URLMapper WHERE cVirtualURL = '" +
 cURL + "'", cnn)
      DataAdapter.Fill(ds)

    If ds.Tables(0).Rows.Count > 0 Then
      cRedirectingTo = ds.Tables(0).Rows(0)("cURL")
      ' redirecting URL
      Context.RewritePath(cRedirectingTo)
      If bCachingEnabled Then
        nCacheFromData = ds.Tables(0).Rows(0)("nCachingInterval")
        ' use default caching interval if negative
        If (nCacheFromData < 0) Then nCacheFromData = nDefaultCache
        ' add item to the cache
        Cache.Add(cURL, cRedirectingTo, Nothing, Now.AddSeconds(nCacheFromData), 
Cache.NoSlidingExpiration, Web.Caching.CacheItemPriority.Normal, Nothing)
      End If
    Else

If there is no mapping for the URL, the request is not redirected, instead the resource requested in the original URL is sent to the client. In this case we do nothing, except for caching the result again (using the default caching interval), if caching is enabled.

      If bCachingEnabled Then
        ' use default caching interval for URLs not in the database
        ' add item to the cache
        Cache.Add(cURL, "default", Nothing, Now.AddSeconds(nDefaultCache), 
Cache.NoSlidingExpiration, Web.Caching.CacheItemPriority.Normal, Nothing)
      End If
    End If

URL Mapping in Work

In this section we first show you the solution working in snapshots of the IBuySpy Portal and then introduce the administration interface, finally provide a description about how to install the sample in your environment.

The URL Mapping sample

The figure below shows you the Discussions page of the IBuySpy Portal accessed using the virtual URL path mapped to the original query string.

The next figure shows the result of disabling the URL of the About the Portal page. It is achieved by sending back a Not Found HTTP status from the HTTP module.

Using the administration page

The admin page included in the sample files provides you with four main functions. These functions are detailed below.

On the top of the admin page you will find the application configuration parameters as you specified them in the web.config file.

In the middle there is a table listing the cached items.

Virtual URLs are displayed in the left column and resolved URLs in the right one. disabled is shown if the direct access to the URL is disabled. default means that there is no mapping found in the database for the URL.

The list is created in the Display_Cache method by iterating through the cached items and filtering out items cached by the system.

  '
  ' display list of cached items
  '
  Public Sub Display_Cache()
    For Each CacheItem In Cache
      If (Left(CacheItem.Key, 7) <> "System." And Left(CacheItem.Key, 19) <> "ISAPIWorkerRequest.") 
Then
        Response.Write("<TR><TD>" & CacheItem.Key & "</TD><TD>" & CacheItem.Value & "</TD></TR>")
      End If
    Next
  End Sub

Under the list of cached URLs there is a simple URL mapping maintenance tool implemented using a datagrid and a few text boxes. You can use it to create, modify or delete mappings in the database.

Each time you delete or modify a mapping, the corresponding item is automatically cleared from the cache, before the deletion or modification itself takes place (in the DeleteCommand and UpdateCommand event handlers of the DataGrid):

Cache.Remove(LCase(ds.Tables(0).Rows(iIndex).Item("Virtual URL")))

This line of code guarantees, that there are no orphaned items in the cache, each cached mapping has its original in the database.

At the bottom of the page we implemented a cache manager. You can select one or more items, and remove them from the cache. It may be especially useful in the case of items that have no mappings in the database, where default is stored as the value of the cached item. The method described above for deletion and modification of mappings doesn't work for these items.

For example, if you rename a file that has its URL cached, and wouldn't like to wait for the default caching time to expire and the obsolete item to be removed automatically by the caching system, it's best to remove the item manually.

When the user presses the Remove button, we first remove all selected items from the cache:

    Dim i As Integer
    For i = 0 To CachedList.Items.Count - 1
      If CachedList.Items(i).Selected Then
        Cache.Remove(CachedList.Items(i).Text())
      End If
    Next

Then we should repopulate the list, since beyond the manually deleted items, there may be other changes in the cache, like items purged automatically by the caching system or new items cached since the last refresh of the admin page.

So we first remove all items from the list:

    Do While CachedList.Items.Count > 0
      CachedList.Items.RemoveAt(0)
    Loop

And then read the up-to-date state of the cache:

    For Each CacheItem In Cache
      If (Left(CacheItem.Key, 7) <> "System." And Left(CacheItem.Key, 19) <> "ISAPIWorkerRequest.")
 Then CachedList.Items.Add(CType(CacheItem.Key, String))
      End If
    Next

Application Setup

In this section we first provide a brief description about how you can download and install the IBuySpy Portal sample on your machine then give you instructions on setting up and configuring the URL mapping application.

Installing the IBuySpy Portal

IBuySpy Portal is a sample application that demonstrates .NET capabilities in creating e-commerce solutions. You can download it starting from the ASP.NET QuickStart Tutorial and selecting IBuySpy.com sample application. If you click View Sample, your browser will redirect to the ASP.NET Web site (http://www.asp.net/?WROXEMPTOKEN=809724Z8y2lKNrebv4PObUjYYy). From there you may choose an online version of the application by clicking Run online, but to try out the sample on your own machine you should click Download. You can download four versions of the portal sample. If you would like to view or modify the application in the Visual Studio .NET environment we suggest you download the VS.NET version, but if you don't need this then the SDK version should be enough for you. There is no significant difference in the size of the install kits. Both Visual Basic .NET and C# versions are available; you can choose what you prefer. We installed the C#, Visual Studio .NET version. System requirements are detailed on the web page. When the download is finished, start the .exe, and follow the instructions to complete the installation of the sample application.

Database setup

You should create a new database manually (let's call it URLMapperDB) on the SQL Server or you can use any existing one for this purpose (for example, the database of the IBuySpy Portal called Portal). Whichever option you choose, you should run the CreateURLMapperDB.sql script using the Query Analyzer from that database to configure the database for the sample.

The script completes the following tasks:

  • Delete any previous version of the URLMapper table
  • Create the URLMapper table and its fields
  • Create the primary key constraint

After these steps the script also creates three sample mappings:

  • A URL mapping with 3 minute caching intervals. This mapping creates an alias for the default document of the portal.
  • An URL mapping with 5 minute caching intervals. This mapping creates an alias, called 'discussions', for the Discussions page.
  • A hidden URL with 10 minutes caching interval. This mapping makes the About the Portal page inaccessible using the query string.

IIS setup

To enable the HTTP module to handle all requests, you should create a new application mapping in the IBuySpy application using the Internet Information Server Manager. You can find this web application under the Default Web Site/PortalCSVS (if you installed the C#, Visual Studio .NET version of the portal).

The new application mapping can be created in the property dialog box of the application, on the Virtual Directory tab, by pressing the Configuration. button. On the App Mappings tab of the Application Configuration dialog box you should click the Add button and then create the mapping as shown in the next figure:

This mapping enables GET HTTP requests to be bound to the aspnet_isapi.dll, without checking if the file specified in the URL is an existing one. If your application is not limited to just passing parameters in the query string, but also uses the POST method posting back form data to the server, then you should specify GET, POST in the Limit to: text box.

MS Internet Information Services 5.1, which is included in Microsoft Windows XP, requires the file extension specified in the Extension text box to start with a dot. This means that if you have Windows XP on your system you should specify .* (dot plus an asterisk) instead of the single * (asterisk) in the Extension text box. The effect of this mapping is the same as the one we described above in the case of IIS 5.0.

Configuring the application

You can configure the URLMapper application in the appSettings section of web.config. Here you can set the data source from where the application gets the URL mappings and caching properties.

The values you must set are:

  • CachingEnabled: Caching can be enabled (true) or disabled (false). It is recommended to enable caching to improve application performance.
  • DefaultCachingInterval: The default time in seconds for caching URLs and resolved mappings. This value is used always for the URLs without individual mappings (i.e. for the URLs not specified in the database). The optimal value is depend on the traffic of your site and the number of different URLs your visitors try to access.
  • URLMapperConnectionString: The SQL connection string to the database where the URLMapper table is present.

A web.config file is included in the sample files, you can customize it to match your system configuration.

Registering the HTTP Module

You can register the URLMapper HTTP module by including the following lines in the system.web section of your web.config file.

        <!-- Registering HTTP Module for the application -->
        <httpModules>
            <add name="URLMapper" type="URLMapper.URLMapper, URLMapper" />
        </httpModules>

Copying files

You should copy URLMapper.dll into the \bin folder of the IBuySpy web application (for the default install location it is C:\PortalCSVS\PortalCSVS\bin), and the AdminPage.aspx into the root of the portal (C:\PortalCSVS\PortalCSVS). Of course, in the case of a production server, you should store this page in a separate directory with more restrictive NTFS rights to prevent anonymous access the file.

Known Issues of the Application

We haven't implemented any kind of data validation on the administration page. If the length of the text specified in the URL fields does not exceed the length of the database fields and the caching interval is an integer number, you are allowed to type any values into these fields. The page does not check whether a well-formatted URL is typed. If you plan to use the URL mapping solution not only for learning purposes, we strongly recommend you implement a more sophisticated user interface and validation on this page.

This application version does not handle the issue that users may access your web site using its IP address instead its name. You should map both URLs - one with the IP address and other one with server name - to deal with this problem. Alternatively, you may use Url.PathAndQuery instead of Url.AbsoulteUri when getting request properties in the HTTP module to get the URL without the site name, but in this case you give up the possibility to store mapping for multiple web sites in the single database table we provided with this sample. In this case you may need to redesign the table to store site names in an additional column and use it as part of the primary key constraint.

Conclusion

HTTP modules are late ascendants of ISAPI filters. Although both of these technologies enable you to modify HTTP requests and responses on the fly, HTTP modules are more flexible and a lot easier to implement and debug, especially if you are not a hardcore Visual C++ programmer. In this article we have demonstrated this by creating a URL mapping solution using Visual Basic .NET. The URL mapping sample can be useful for you if you plan to advertise your portal site displaying user friendly URLs instead of long and magic query strings. We have built a caching mechanism into the application that exploits the built-in ASP.NET caching system. We have provided a simple administration page for you to make it easier to monitor the application configuration settings, the state of the cache and last but not least, to maintain URL mappings.

 

Please rate this article using the form below. By telling us what you like and dislike about it we can tailor our content to meet your needs.

Article Information
Author Péter Holpár & Péter Csáki
Chief Technical Editor John R. Chapman
Project Manager Helen Cuthill
Reviewers Phil Sidari, Susan Connery

If you have any questions or comments about this article, please contact the technical editor.

 
 
Rate this Article
How useful was this article?
Not useful Very useful
Brief Reader Comments: Read Comments
Your name (optional):
 
 
Content Related Links Discussion Comments Index Entries Downloads
 
Back to top