Introduction
A common defect in several portal systems is that their
pages can only be addressed using complex URLs that contain
complicated query strings. Most often a single .asp or
.aspx page does the job by using different parameters in the
URL. Just have a look at the online .NET sample IBuySpy Portal (http://www.ibuyspyportal.com/?WROXEMPTOKEN=809724Z8y2lKNrebv4PObUjYYy)
where DesktopDefault.aspx is used to display most of the
portal pages while the tabindex and
tabid parameters in the query string
specify exactly what content should be shown to the visitor.
Imagine a fictitious news company, LatestNews, has an
Internet site - http://www.latestnews.com/?WROXEMPTOKEN=809724Z8y2lKNrebv4PObUjYYy
- and a regular TV news program. The company would like to attach a
unique, easy-to-remember URL for the main news discussed on the TV
and show this URL on the screen at the end of the item. For example,
in the case of a successful Mars expedition, this URL would probably
be http://www.latestnews.com/menonthemars?WROXEMPTOKEN=809724Z8y2lKNrebv4PObUjYYy.
The portal system is similar to the one described above, so the
real URL may be something like this: http://www.latestnews.com/default.aspx?EDRef=53&NewsRef=3244&WROXEMPTOKEN=809724Z8y2lKNrebv4PObUjYYy.
The managers of the news company worry that this format is too
difficult for an average viewer (wonder why?). The news editor
should be able to maintain the mapping between virtual and real URL
addresses, and it may be necessary to disable the direct, query
string based access to the pages, and to allow only the usage of
virtual URLs to hide the internal working of the web application
from visitors.
In this article we will show you how to implement a .NET-based
solution for this problem. We will use HTTP modules to achieve this
goal, so the article starts with a short description of the theory
of HTTP modules. After this, we will extend our base HTTP module to
fulfill the requirements of the URL mapping system. We will show you
the application applied on the IBuySpy Portal sample and the
administration page that you can use to maintain you mappings.
Finally, we provide detailed instructions about installing and
configuring the solution in you environment.
System Requirements
To install the sample application you will need the following
software components on your system:
- Windows 2000 Server/Advanced Server, Windows 2000 Professional
or Windows XP operating system
- MS SQL Server 2000/ MS SQL Server 7.0 or MS Data Engine
- .NET Framework (+ .NET Framework Service Pack 1) and Visual
Studio .NET
- Internet Explorer 5.0 or higher
To install the IBuySpy Portal sample we used to test the
application, the requirements are a little bit stronger for the
database server and for the browser:
- MS SQL Server 2000 or MS Data Engine 2000 or higher
- Internet Explorer 5.0 or higher
To fully understand the code you should have at least basic
knowledge of:
- Visual Basic .NET
- ASP.NET
- ADO.NET
- SQL
Introducing HTTP Modules
The role of the HTTP module in the .NET world is parallel
to the function of the ISAPI filters of the pre-.NET age
(similarly, HTTP handlers are parallel to ISAPI
extensions). If you are familiar with ISAPI filters you know
that they can preprocess HTTP requests and post-process HTTP
responses. HTTP modules extend this model so you can register your
HTTP module to be triggered by events of an HTTP application. An
instance of the triggering HTTP application is passed to the HTTP
module, so you can access and modify its properties, including the
Request and Response objects that are members of the
HttpApplication.Context object.
The figure below shows the schematic HTTP pipeline of
processing an HTTP response and passing back an HTTP
request.
Developing ISAPI filters requires you to have knowledge of
relatively low level programming languages, like C++ or Delphi, as
the architecture and threading model of Visual Basic were inadequate
to perform such tasks. The advent of the .NET Framework blurred the
difference between the programming languages, so - of course - it is
possible to use Visual Basic .NET to implement an HTTP module. To
demonstrate this we chose this language to create our URL mapping
application.
We don't find comparison data about the performance of HTTP
modules vs. performance of ISAPI filters, but ISAPI filters should
provide slightly higher performance on the same hardware and
software environment. Since HTTP modules are implemented over an
additional abstraction layer, namely the HTTP runtime of ASP.NET,
each request must pass through the aspnet_isapi.dll ISAPI extension.
Debugging HTTP modules is easy (similar to debugging ASP.NET
pages) in contrast to ISAPI filter bug-hunting, that usually
requires your debugger application to be able to attach to a Windows
process (the integrated debugger of MS Visual Studio is capable of
doing this). Since ISAPI filters run very tightly integrated with
system processes, a badly written ISAPI filter may cause your web
server and even the operating system to crash, this is certainly not
the case with the .NET HTTP modules.
Creating the Frame of an HTTP Module in Visual
Basic .NET
HTTP modules should implement the System.Web.IHttpModule interface. This
interface has two methods that you must implement:
- Sub Init(ByVal Application As
HttpApplication): You should register the event handlers in
this method.
- Sub Dispose(): You should place
the clean-up code into this method.
The skeleton of the HTTP module should be something like
this: Imports System.Web
Public Class SimpleHttpModule
Implements IHttpModule
Public Sub Init(ByVal Application As HttpApplication) Implements IHttpModule.Init
' register the BeginRequest event
AddHandler Application.BeginRequest, AddressOf Application_BeginRequest
End Sub
Public Sub Dispose() Implements IHttpModule.Dispose
' do the clean-up
End Sub
Private Sub Application_BeginRequest(ByVal Source As Object, ByVal e As EventArgs)
' casting the Source to an HttpApplication object
Dim Application As HttpApplication
Application = CType(Source, HttpApplication)
' get or modify HTTP application properties as adequate
End Sub
End Class
Configure Your Web Application to Use the HTTP
Module
You can register your HTTP module in the system.web section of the web.config file using an <add> directive. <add name="ModuleName" type=".NET
Class, Assembly [,Version=version number] [,Culture=culture] [,PublicKeyToken=token]" />
Where the name is a friendly name
for the module and type refers to a
comma-separated class/assembly combination. The assembly dll should
be in the private \bin folder of the application or in the
assembly cache.
Changing Application Mapping on the Web
Server
Another, maybe less known, difference with the ISAPI filters is
that, by design, HTTP modules are notified only if files with
extensions associated to the aspnet_isapi.dll - like .aspx
- are requested by the user. For example, if the URL request
refers to the simple folder name of /portal/discussions the
request will not be handled by the HTTP module. Since we would like
our module to monitor these kinds of requests too, we have to make
some changes in the configuration of the web server. The necessary
configuration steps are detailed in the IIS setup section of
this article.
You should be aware that by making these changes, you also change
the way IIS handles default documents. To be exact, IIS will not
find the default documents for you if you specify only the folder
name. You must specify the full path of the document in your
browser. A workaround to this problem is to make a URL mapping for
the default document that maps the URL of the folder to the full URL
of the default document. We will describe how to do this, later.
The URL Mapping Application
In this section we will show you the URL mapping HTTP module and
the database used to store the mappings.
The Database
We don't need a very sophisticated database in this application.
To store the mappings of the virtual and real URLs and the caching
interval for the mapping the following simple table is enough:
We will store the following values for each URL mappings:
- cVirtualURL: The URL as it is
typed in the browser (for example, http://localhost/PortalCSVS/Discussions).
By storing the site name as part of the URL it is possible to use
a single database table for multiple sites.
- cURL: The real URL relative to
the web site (i.e. without the leading http:// and site name, for example: /PortalCSVS/Default.aspx). If you would
like to hide the URL specified in the cVirtualURL, use an empty string in the
cURL field.
- nCachingInterval: An integer
value to store the individual caching interval for the URL
mapping. If it is a negative value, the default caching interval
specified in the web.config file will be used.
We will query the database using simple SQL Select statements
through ADO.NET from the HTTP module.
Caching
Caching is important to remove any unnecessary burden from the
database server by storing already resolved and known mappings in
memory on the web server instead of querying the SQL server for each
web request. This is especially useful if your web server and
database are running on different machines. We will implement our
caching system using the Cache object
of ASP.NET.
On the other hand, if you specified a too long cache time for an
item, browsers would be redirected to the cached URL, even if you
modified this mapping in the database in the meantime. The
administration page we provide you, gives a solution to this problem
so you shouldn't disable or clear the entire cache. For the details
of this solution see the Using the administration page
section later.
Extending the Skeleton
Let's examine how we should extend our HTTP module skeleton to
develop the URL mapping applications. We should modify our Application_BeginRequest method.
First, we create references to the most often used objects: Dim Application As HttpApplication = CType(Source, HttpApplication)
Dim Context As HttpContext = Application.Context
Dim Request As HttpRequest = Context.Request
Dim Response As HttpResponse = Context.Response
Dim Cache As Caching.Cache = Context.Cache
After this we read the requested URL (SplitSlashes is a function that removes
extra slashes from the end of the URL): cURL = SplitSlashes(LCase(Request.Url.AbsoluteUri))
Then we read configuration parameters from the web.config
file: Dim AppSet = Context.GetConfig("appSettings")
cConnStr = AppSet("URLMapperConnectionString")
nDefaultCache = AppSet("DefaultCachingInterval")
bCachingEnabled = (LCase(AppSet("CachingEnabled")) = "true")
If caching is enabled, we try to look up the URL in the
cache: If Not (IsNothing(Cache(cURL))) Then
If default is stored in the cache,
we do not modify the request. In this case the server tries to
fulfill the original request. If (Cache.Item(cURL) = "default") Then
Exit Sub
If disabled is stored in the cache,
we send back an HTTP not found status, so users will experience a
file not found message. ElseIf (Cache.Item(cURL) = "disabled") Then
' set HTTP status to not found (404)
Response.StatusCode = HttpStatusCode.NotFound
Response.End()
Exit Sub
Otherwise we modify the path for the request to the cached
one: Else
' redirect URL and finish
Context.RewritePath(Cache.Item(cURL))
Exit Sub
End If
End If
If caching is disabled, we remove all items already cached by the
application. We shouldn't remove any item that is cached by the
system, so we apply a filter: For Each CacheItem In Cache
If (Left(CacheItem.Key, 7) <> "System." And Left(CacheItem.Key, 19) <> "ISAPIWorkerRequest.")
Then
Cache.Remove(CacheItem.Key)
End If
Next
If the requested URL is not found in the cache, we should query
our database. First we check if the mapped URL is an empty string.
This would mean that the direct access of this URL is disabled. DataAdapter = New System.Data.SqlClient.SqlDataAdapter
("SELECT nCachingInterval FROM URLMapper WHERE cVirtualURL = '" + cURL +
"' AND cURL = ''", cnn)
DataAdapter.Fill(ds)
If the dataset is not empty, it means we have a matching record
in the database, so we should deny access to the requested page. If
caching is enabled, we store this result in the cache too. The
caching interval is from the database and if it is a negative value,
the default caching interval is used. If ds.Tables(0).Rows.Count > 0 Then
If bCachingEnabled Then
nCacheFromData = ds.Tables(0).Rows(0)("nCachingInterval")
' use default caching interval if negative
If (nCacheFromData < 0) Then nCacheFromData = nDefaultCache
' add item to the cache
Cache.Add(cURL, "disabled", Nothing, Now.AddSeconds(nCacheFromData),
Cache.NoSlidingExpiration, Web.Caching.CacheItemPriority.Normal, Nothing)
End If
' set HTTP status to not found (404)
Response.StatusCode = HttpStatusCode.NotFound
Response.End()
Exit Sub
End If
The next step is to check if are other (non empty string)
mappings for the requested URL. If there are, we should redirect the
request to this file. We cache the result if caching is enabled. The
caching interval is from the database and if it is a negative value,
the default caching interval is used. DataAdapter = New System.Data.SqlClient.SqlDataAdapter
("SELECT cURL, nCachingInterval FROM URLMapper WHERE cVirtualURL = '" +
cURL + "'", cnn)
DataAdapter.Fill(ds)
If ds.Tables(0).Rows.Count > 0 Then
cRedirectingTo = ds.Tables(0).Rows(0)("cURL")
' redirecting URL
Context.RewritePath(cRedirectingTo)
If bCachingEnabled Then
nCacheFromData = ds.Tables(0).Rows(0)("nCachingInterval")
' use default caching interval if negative
If (nCacheFromData < 0) Then nCacheFromData = nDefaultCache
' add item to the cache
Cache.Add(cURL, cRedirectingTo, Nothing, Now.AddSeconds(nCacheFromData),
Cache.NoSlidingExpiration, Web.Caching.CacheItemPriority.Normal, Nothing)
End If
Else
If there is no mapping for the URL, the request is not
redirected, instead the resource requested in the original URL is
sent to the client. In this case we do nothing, except for caching
the result again (using the default caching interval), if caching is
enabled. If bCachingEnabled Then
' use default caching interval for URLs not in the database
' add item to the cache
Cache.Add(cURL, "default", Nothing, Now.AddSeconds(nDefaultCache),
Cache.NoSlidingExpiration, Web.Caching.CacheItemPriority.Normal, Nothing)
End If
End If
URL Mapping in Work
In this section we first show you the solution working in
snapshots of the IBuySpy Portal and then introduce the
administration interface, finally provide a description about how to
install the sample in your environment.
The URL Mapping sample
The figure below shows you the Discussions page of the IBuySpy Portal
accessed using the virtual URL path mapped to the original query
string.
The next figure shows the result of disabling the URL of the
About the Portal page. It is achieved
by sending back a Not Found HTTP status from the HTTP module.
Using the administration page
The admin page included in the sample files provides you with
four main functions. These functions are detailed below.
On the top of the admin page you will find the application
configuration parameters as you specified them in the
web.config file.
In the middle there is a table listing the cached items.
Virtual URLs are displayed in the left column and resolved URLs
in the right one. disabled is shown if
the direct access to the URL is disabled. default means that there is no mapping found
in the database for the URL.
The list is created in the Display_Cache method by iterating through
the cached items and filtering out items cached by the system. '
' display list of cached items
'
Public Sub Display_Cache()
For Each CacheItem In Cache
If (Left(CacheItem.Key, 7) <> "System." And Left(CacheItem.Key, 19) <> "ISAPIWorkerRequest.")
Then
Response.Write("<TR><TD>" & CacheItem.Key & "</TD><TD>" & CacheItem.Value & "</TD></TR>")
End If
Next
End Sub
Under the list of cached URLs there is a simple URL mapping
maintenance tool implemented using a datagrid and a few text boxes.
You can use it to create, modify or delete mappings in the
database.
Each time you delete or modify a mapping, the corresponding item
is automatically cleared from the cache, before the deletion or
modification itself takes place (in the DeleteCommand and UpdateCommand event handlers of the DataGrid): Cache.Remove(LCase(ds.Tables(0).Rows(iIndex).Item("Virtual URL")))
This line of code guarantees, that there are no orphaned items in
the cache, each cached mapping has its original in the database.
At the bottom of the page we implemented a cache manager. You can
select one or more items, and remove them from the cache. It may be
especially useful in the case of items that have no mappings in the
database, where default is stored as
the value of the cached item. The method described above for
deletion and modification of mappings doesn't work for these
items.
For example, if you rename a file that has its URL cached, and
wouldn't like to wait for the default caching time to expire and the
obsolete item to be removed automatically by the caching system,
it's best to remove the item manually.
When the user presses the Remove
button, we first remove all selected items from the cache: Dim i As Integer
For i = 0 To CachedList.Items.Count - 1
If CachedList.Items(i).Selected Then
Cache.Remove(CachedList.Items(i).Text())
End If
Next
Then we should repopulate the list, since beyond the manually
deleted items, there may be other changes in the cache, like items
purged automatically by the caching system or new items cached since
the last refresh of the admin page.
So we first remove all items from the list: Do While CachedList.Items.Count > 0
CachedList.Items.RemoveAt(0)
Loop
And then read the up-to-date state of the cache: For Each CacheItem In Cache
If (Left(CacheItem.Key, 7) <> "System." And Left(CacheItem.Key, 19) <> "ISAPIWorkerRequest.")
Then CachedList.Items.Add(CType(CacheItem.Key, String))
End If
Next
Application Setup
In this section we first provide a brief description about how
you can download and install the IBuySpy Portal sample on your
machine then give you instructions on setting up and configuring the
URL mapping application.
Installing the IBuySpy Portal
IBuySpy Portal is a sample application that demonstrates .NET
capabilities in creating e-commerce solutions. You can download it
starting from the ASP.NET QuickStart Tutorial and selecting
IBuySpy.com sample application. If you click View Sample, your browser will redirect to
the ASP.NET Web site (http://www.asp.net/?WROXEMPTOKEN=809724Z8y2lKNrebv4PObUjYYy).
From there you may choose an online version of the application by
clicking Run online, but to try out
the sample on your own machine you should click Download. You can download four versions of
the portal sample. If you would like to view or modify the
application in the Visual Studio .NET environment we suggest you
download the VS.NET version, but if you don't need this then the SDK
version should be enough for you. There is no significant difference
in the size of the install kits. Both Visual Basic .NET and C#
versions are available; you can choose what you prefer. We installed
the C#, Visual Studio .NET version. System requirements are detailed
on the web page. When the download is finished, start the
.exe, and follow the instructions to complete the
installation of the sample application.
Database setup
You should create a new database manually (let's call it URLMapperDB) on the SQL Server or you can
use any existing one for this purpose (for example, the database of
the IBuySpy Portal called Portal).
Whichever option you choose, you should run the CreateURLMapperDB.sql script using the Query
Analyzer from that database to configure the database for the
sample.
The script completes the following tasks:
- Delete any previous version of the URLMapper table
- Create the URLMapper table and
its fields
- Create the primary key constraint
After these steps the script also creates three sample
mappings:
- A URL mapping with 3 minute caching intervals. This mapping
creates an alias for the default document of the portal.
- An URL mapping with 5 minute caching intervals. This mapping
creates an alias, called 'discussions', for the Discussions page.
- A hidden URL with 10 minutes caching interval. This mapping
makes the About the Portal page
inaccessible using the query string.
IIS setup
To enable the HTTP module to handle all requests, you should
create a new application mapping in the IBuySpy application using
the Internet Information Server Manager. You can find this web
application under the Default Web
Site/PortalCSVS (if you installed the C#, Visual Studio .NET
version of the portal).
The new application mapping can be created in the property dialog
box of the application, on the Virtual
Directory tab, by pressing the Configuration. button. On the App Mappings tab of the Application Configuration dialog box you
should click the Add button and then
create the mapping as shown in the next figure:
This mapping enables GET HTTP
requests to be bound to the aspnet_isapi.dll, without
checking if the file specified in the URL is an existing one. If
your application is not limited to just passing parameters in the
query string, but also uses the POST method posting back form data
to the server, then you should specify GET,
POST in the Limit to: text
box.
MS Internet Information Services 5.1, which is included in
Microsoft Windows XP, requires the file extension specified in the
Extension text box to start with a
dot. This means that if you have Windows XP on your system you
should specify .* (dot plus an
asterisk) instead of the single *
(asterisk) in the Extension text box.
The effect of this mapping is the same as the one we described above
in the case of IIS 5.0.
Configuring the application
You can configure the URLMapper application in the appSettings section of web.config.
Here you can set the data source from where the application gets the
URL mappings and caching properties.
The values you must set are:
- CachingEnabled: Caching can be
enabled (true) or disabled (false). It is recommended to enable
caching to improve application performance.
- DefaultCachingInterval: The
default time in seconds for caching URLs and resolved mappings.
This value is used always for the URLs without individual mappings
(i.e. for the URLs not specified in the database). The optimal
value is depend on the traffic of your site and the number of
different URLs your visitors try to access.
- URLMapperConnectionString: The
SQL connection string to the database where the URLMapper table is
present.
A web.config file is included in the sample files, you can
customize it to match your system configuration.
Registering the HTTP Module
You can register the URLMapper HTTP
module by including the following lines in the system.web section of your web.config
file. <!-- Registering HTTP Module for the application -->
<httpModules>
<add name="URLMapper" type="URLMapper.URLMapper, URLMapper" />
</httpModules>
Copying files
You should copy URLMapper.dll into the \bin folder
of the IBuySpy web application (for the default install location it
is C:\PortalCSVS\PortalCSVS\bin), and the
AdminPage.aspx into the root of the portal
(C:\PortalCSVS\PortalCSVS). Of course, in the case of a
production server, you should store this page in a separate
directory with more restrictive NTFS rights to prevent anonymous
access the file.
Known Issues of the Application
We haven't implemented any kind of data validation on the
administration page. If the length of the text specified in the URL
fields does not exceed the length of the database fields and the
caching interval is an integer number, you are allowed to type any
values into these fields. The page does not check whether a
well-formatted URL is typed. If you plan to use the URL mapping
solution not only for learning purposes, we strongly recommend you
implement a more sophisticated user interface and validation on this
page.
This application version does not handle the issue that users may
access your web site using its IP address instead its name. You
should map both URLs - one with the IP address and other one with
server name - to deal with this problem. Alternatively, you may use
Url.PathAndQuery instead of Url.AbsoulteUri when getting request
properties in the HTTP module to get the URL without the site name,
but in this case you give up the possibility to store mapping for
multiple web sites in the single database table we provided with
this sample. In this case you may need to redesign the table to
store site names in an additional column and use it as part of the
primary key constraint.
Conclusion
HTTP modules are late ascendants of ISAPI filters. Although both
of these technologies enable you to modify HTTP requests and
responses on the fly, HTTP modules are more flexible and a lot
easier to implement and debug, especially if you are not a hardcore
Visual C++ programmer. In this article we have demonstrated this by
creating a URL mapping solution using Visual Basic .NET. The URL
mapping sample can be useful for you if you plan to advertise your
portal site displaying user friendly URLs instead of long and magic
query strings. We have built a caching mechanism into the
application that exploits the built-in ASP.NET caching system. We
have provided a simple administration page for you to make it easier
to monitor the application configuration settings, the state of the
cache and last but not least, to maintain URL mappings. |