The great thing
about the POP mail protocol is that it is a
well-documented open standard, making writing a mail
client to collect mail from a POP box a relatively
painless process. Armed with basic knowledge of POP, or
SMTP it is possible to write proxies, which do a variety
of useful things, such filter out spam or junk mail, or
provide an e-mail answering machine service.
Unfortunately, in trying to write a standalone client
for Hotmail, the world’s most popular web-based
mailing system, the fact that no POP gateway exists
rapidly becomes a problem.
Despite the lack of POP-support, connecting to Hotmail
without using a web-browser is possible. Outlook Express
allows users to retrieve, delete, move and send
messages, connecting directly to a standard Hotmail or
MSN mailbox. By using a HTTP packet sniffer, it is
possible to monitor communication between Outlook
Express and Hotmail, making it possible to determine how
the direct mailbox connection is made.
Outlook Express uses an undocumented protocol commonly
referred to as HTTPMail, allowing a client to access
Hotmail using a set of HTTP/1.1 extensions. This article
explains some of the features of HTTPMail, and how best
to connect to Hotmail using a VB.NET client. The sample
source code accompanying this article uses COM interop
to leverage XMLHTTP as the transport service. The
XMLHTTP component provides a complete HTTP
implementation, including authentication together with
ability to set custom headers before sending HTTP
requests server-side.
Connecting to the HTTPMail Hotmail Gateway
The default HTTPMail gateway for Hotmail boxes
is located at
http://services.msn.com/svcs/hotmail/httpmail.asp.
Although undocumented, the HTTPMail protocol is actually
a standard WebDAV service. As we are using VB.NET, we
could use the TCP and HTTP classes provided by the .NET
framework within the System.Net namespace. Since we are
working with WebDAV, it is simpler to use XMLHTTP to
connect to Hotmail under C#. Referencing the MSXML2
component provides an interop assembly, which may be
accessed directly. Note that in the code snippets
contained within this article, variables suffixed with
an underscore refer to member fields declared elsewhere
within the sample code:
' Get the
namespace.
Imports
MSXML2
...
' Create
the object.
xmlHttp_ =
new
XMLHTTP()
In order to connect to a secure server, the WebDAV
protocol requires HTTP/1.1 authentication. The initial
request sent by an HTTPMail client uses the WebDAV
PROPFIND method to query for a set of properties. These
include the URL of the Hotmail advertisement bar
together with the location of mailbox folders:
<?xml
version="1.0"?>
<D:propfind
xmlns:D="DAV:"
xmlns:h="http://schemas.microsoft.com/hotmail/"
<D:prop>
<h:adbar/>
<hm:contacts/>
<hm:inbox/>
<hm:outbox/>
<hm:sendmsg/>
<hm:sentitems/>
<hm:deleteditems/>
<hm:drafts/>
<hm:msgfolderroot/>
<h:maxpoll/>
<h:sig/>
</D:prop>
</D:propfind>
Sending the initial request via XMLHTTP begins by
specifying the WebDAV server URL, and generating the
initial XML request:
' Specify the
server URL.
Dim
serverUrl As
String = http://services.msn.com/svcs/hotmail/httpmail.asp
' Build the query.
Dim
folderQuery As
String = Nothing
folderQuery += "<?xml version='1.0'?><D:propfind xmlns:D='DAV:'
"
folderQuery += "xmlns:h='http://schemas.microsoft.com/hotmail/'
"
folderQuery += "xmlns:hm='urn:schemas:httpmail:'><D:prop><h:adbar/>"
folderQuery += "<hm:contacts/><hm:inbox/><hm:outbox/><hm:sendmsg/>"
folderQuery += "<hm:sentitems/><hm:deleteditems/><hm:drafts/>"
folderQuery += "<hm:msgfolderroot/><h:maxpoll/><h:sig/></D:prop></D:propfind>"
The HTTPXML component provides an open() method used to
establish a connection to a HTTP server:
Sub open(method As String, url As String, async As
Boolean, user As String, password As String)
The first argument specifies the HTTP method used to
open the connection, such as GET, POST, PUT, or PROPFIND.
To connect to the Hotmail gateway, we specify the
PROPFIND method to query the mailbox. This and other
HTTP methods are used to retrieve folder information,
collect mail items and send new mail. Note that the
open() method allows for the possibility of asynchronous
calls (enabled by default) which is preferred for a
graphical mail client. Since the sample code is a
console application, we set this parameter to false. For
authentication, we specify a username and password. Note
that under XMLHTTP, the component will display a login
window if these parameters are missing and the site
requires authentication. To connect to the Hotmail
gateway we open the connection, set the PROPFIND request
header to our XML-based query, and then send the request
with a null body:
' Open a
connection to the Hotmail server.
xmlHttp_.open("PROPFIND",
serverUrl, False, username,
password)
' Send the
request.
xmlHttp_.setRequestHeader("PROPFIND", folderQuery)
xmlHttp_.send(Nothing)
Parsing the Mailbox Folder List
The request sent to services.msn.com is
typically redirected several times. After load
balancing, we are finally connected to a free Hotmail
server, and authenticated. This redirection, with
appropriate authentication, is handled by the XMLHTTP
component. Once connected, the server will also ask us
to set various cookies, valid for the current session
(again this is all handled automatically by XMLHTTP).
Upon sending the initial connection request, the server
will return an XML-based response:
' Get the
response.
Dim
folderList As
String =
xmlHttp_.responseText
The returned response will contain, among other useful
information, the URL locations of the folders within the
mailbox. For example:
<?xml
version="1.0"
encoding="Windows-1252"?>
<D:response>
...
<D:propstat>
<D:prop>
<h:adbar>AdPane=Off*...</h:adbar>
<hm:contacts>http://law15.oe.hotmail.com/...</hm:contacts>
<hm:inbox>http://law15.oe.hotmail.com/...</hm:inbox>
<hm:sendmsg>http://law15.oe.hotmail.com/...</hm:sendmsg>
<hm:sentitems>http://law15.oe.hotmail.com/...</hm:sentitems>
<hm:deleteditems>http://law15.oe.hotmail.com/...</hm:deleteditems>
<hm:msgfolderroot>http://law15.oe.hotmail.com/...</hm:msgfolderroot>
...
</D:prop>
</D:response>
</D:multistatus>
In
the sample console application, the two mailbox folders
that we are interested in are the inbox and sendmsg
folders, used to retrieve and send mail items
respectively. There are various ways to parse XML under
C#, but since we are confident of our XML structure,
System.XML.XmlTextReader provides fast, forward-only
access. We initialise the XML reader by converting the
XML string data into a string stream:
' Initiate.
inboxUrl_ =
Nothing
sendUrl_ =
Nothing
' Load the
Xml.
Dim
reader As
New StringReader(folderList)
Dim
xml As
New XmlTextReader(reader)
The XML is parsed by iterating through each node,
picking out the hm:inbox and hm:sendmsg nodes:
' Read the
Xml.
While
xml.Read()
' Got an
element?
If
xml.NodeType = XmlNodeType.Element
Then
'
Get this node.
Dim
name As
String = xml.Name
' Got the
inbox?
If
name = "hm:inbox" Then
'
Set folder.
xml.Read()
inboxUrl_ = xml.Value
End
If
'
Got the send message page?
If
name = "hm:sendmsg" Then
'
Set folder.
xml.Read()
sendUrl_ = xml.Value
End
If
End If
End
While
Once the
URLs for the inbox and outbox which are valid for this
session have been determined, it is possible send and
retrieve and e-mail.
Enumerating Folder MailItems
Given the URL of a mailbox folder (such as the Inbox
folder) we can direct a WebDAV request to the folder's
URL in order to list mail items within the folder. The
sample console application defines a managed type
MailItem, used to store mail information for a folder
item. Folder enumeration begins by initalising an array
of MailItems:
'
Initiate.
Dim
mailItems As
New ArrayList
To request mail item data, such as the mail subject, and
the to and from addresses, we generate the following
XML-based WebDAV query:
<?xml
version="1.0"?>
<D:propfind
xmlns:D="DAV:"
xmlns:hm="urn:schemas:httpmail:"
xmlns:m="urn:schemas:mailheader:">
<D:prop>
<D:isfolder/>
<hm:read/>
<m:hasattachment/>
<m:to/>
<m:from/>
<m:subject/>
<m:date/>
<D:getcontentlength/>
</D:prop>
</D:propfind>
The
followig VB.NET code generates the XML query string:
' Build the
query.
Dim
getMailQuery As
String =
Nothing
getMailQuery += "<?xml version='1.0'?><D:propfind
xmlns:D='DAV:' "
getMailQuery += "xmlns:hm='urn:schemas:httpmail:' "
getMailQuery += "xmlns:m='urn:schemas:mailheader:'><D:prop><D:isfolder/>"
getMailQuery += "<hm:read/><m:hasattachment/><m:to/><m:from/><m:subject/>"
getMailQuery += "<m:date/><D:getcontentlength/></D:prop></D:propfind>"
This request is sent via XMLHTTP using the PROPFIND
method, similar to the way in which the mailbox folder
list was requested above. This time round we set the
request body to be the query, and folder information is
returned. Since we have already been authenticated for
this session, there is no need to resupply the username
and password on the XMLHTTP open() call:
' Get the
mail info.
xmlHttp_.open("PROPFIND",
folderUrl, False,
Nothing,
Nothing)
xmlHttp_.send(getMailQuery)
Dim
folderInfo As
String =
xmlHttp_.responseText
Following a successful request, the server will respond
with an XML stream containing information for each
MailItem contained within the folder.
<D:multistatus>
<D:response>
<D:href>http://sea1.oe.hotmail.com/cgi-bin/hmdata/...
</D:href>
<D:propstat>
<D:prop>
<hm:read>1</hm:read>
<m:to/>
<m:from>Mark
Anderson</m:from>
<m:subject>RE:
New Information</m:subject>
<m:date>2002-08-06T16:38:39</m:date>
<D:getcontentlength>1238</D:getcontentlength>
</D:prop>
<D:status>HTTP/1.1
200 OK</D:status>
</D:propstat>
</D:response>
...
Looking at the XML fragment above, we find that
contained within each <D:response> node are a set of
fields identifying the MailItem, including the <D:href>
tag, which will later allow us to retrieve the item. We
can again use System.XML.XmlTextReader to parse this XML
text stream. We first initalise the stream reader:
'
Holders.
Dim
mailItem As MailItem =
Nothing
'
Load the Xml.
Dim
reader As
New StringReader(folderInfo)
Dim
xml As
New XmlTextReader(reader)
Parsing Folder Information
In order the parse the XML in a single pass, we
create a new MailItem instance on opening the <D:response>
element, and store the instance when we reach the end of
the tag. In between, we extract and set MailItem fields:
' Read the
Xml.
While
xml.Read()
' Sections.
Dim
name As
String = xml.Name
Dim
nodeType As XmlNodeType =
xml.NodeType
' E-mail?
If
name = "D:response" Then
'
Start?
If
nodeType = XmlNodeType.Element
Then
'
Create a new mail item.
mailItem =
New
MailItem()
End
If
'
End?
If
nodeType = XmlNodeType.EndElement
Then
'
Store the last mail.
mailItems.Add(mailItem)
' Reset.
mailItem =
Nothing
End
If
End If
'
Got an element?
If
nodeType = XmlNodeType.Element
Then
'
Mail field.
If
name = "D:href" Then
'
Load.
xml.Read()
mailItem.Url = xml.Value
End
If
'
Mail field.
If
name = "hm:read" Then
'
Load.
xml.Read()
mailItem.IsRead = xml.Value = "1"
End
If
End If
End
While
' Load other
MailItem fields, etc...
The above
code (part of that found the sample console application)
enumerates the MailItems found within the given folder.
For each MailItem we extract the following fields:
| XML
Node |
Description |
| D:href
|
The URL used
to retrieve this HttpMail item. |
| hm:read |
Flag set if
this e-mail is read. |
| m:to |
Who the mail
was sent to. |
| m:from |
Who the mail
was sent from. |
| m:subject |
The mail
subject. |
| m:date |
Timestamp in
the format [date]T[time] |
|
D:getcontentlength |
The size of
this e-mail (in bytes). |
The sample code reads the XML nodes as set out above,
to extract information for each mail item found within
the returned Folder Info XML data stream.
Retrieving Folder Mail
Once MailItems in the folder have been
enumerated, the mail URL (valid for the given session)
for a MailItem can be used to retrieve the actual
e-mail. Sending a HTTP/1.1 GET request to the Hotmail
server, at the given URL, does this. The LoadMail()
function defined within the sample code takes a MailItem
instance, and returns the content of the mailbox e-mail:
' <summary>
' Loads the given mail item.
' </summary>
Public
Function LoadMail(ByVal
mailItem As MailItem)
As
String
'
Get the Url.
Dim
mailUrl As
String = mailItem.Url
' Open a
connection to the Hotmail server.
xmlHttp_.open("GET",
mailUrl, False,
Nothing,
Nothing)
' Send the
request.
xmlHttp_.send(Nothing)
' Get the
response.
Dim
mailData As
String =
xmlHttp_.responseText
' Return
the mail data.
Return mailData
End
Function
'LoadMail
Sending New Mail
In order to retrieve mail, the LoadMail()
method (see above) performs a HTTP/1.1 GET request.
Similarly, a POST is sent to the sendmsg URL in order to
send e-mail from the Hotmail box. The sample console
application also contains a SendMail() method which
maybe invoked once connected to the mailbox:
' <summary>
' Sends an e-mail.
' </summary>
Public
Sub SendMail(ByVal
from As
String,
ByVal fromName
As
String, ByVal [to]
As
String, ByVal
subject As
String,
ByVal body
As
String)
'
End
Sub
'SendMail '
We
begin by setting up a quote string (used later) as well
generating a mail time stamp:
' Quote
character.
Dim
quote As
String = """"
' Generate
the time stamp.
Dim
now As DateTime =
DateTime.Now
Dim
timeStamp As
String = now.ToString("ddd,
dd MMM yyyy hh:mm:ss")
The HTTPMail protocol follows an SMTP-like communication
scheme (See RFC 821). Outlook express sends out mail in
MIME format, but for demonstrations purposes we simply
send a plain text e-mail:
' Build the
post body.
Dim
postBody As
String =
Nothing
'
Dump mail headers.
postBody += "MAIL FROM:<" +
from + ">" + ControlChars.Cr + ControlChars.Lf
postBody += "RCPT TO:<" + [to] + ">" + ControlChars.Cr +
ControlChars.Lf
postBody += ControlChars.Cr + ControlChars.Lf
postBody += "From: " + quote + fromName + quote + " <" +
from + ">" + ControlChars.Cr + ControlChars.Lf
postBody += "To: <" + [to] + ">" + ControlChars.Cr +
ControlChars.Lf
postBody += "Subject: " + subject + ControlChars.Cr +
ControlChars.Lf
postBody += "Date: " + timeStamp + " -0000" +
ControlChars.Lf
postBody += ControlChars.Cr + ControlChars.Lf
' Dump mail
body.
postBody += body
To send the mail, we need to set the Content-Type
request header to message/rfc821, indicating that this
request contains a body which follows RFC 821. We POST
the request body generated above to the sendmsg URL
obtained during connection time:
' Open the
connection.
xmlHttp_.open("POST",
sendUrl_, False,
Nothing,
Nothing)
' Send the
request.
xmlHttp_.setRequestHeader("Content-Type",
"message/rfc821")
xmlHttp_.send(postBody)
Given a valid destination mailbox, Hotmail will send the
mail to our desired location.
Conclusion
Hotmail is the world's largest provider of
free, Web-based e-mail. However, the only non-web mail
client with direct access to Hotmail is Outlook Express.
Since the HTTPMail protocol is undocumented, other
vendors are discouraged from providing a similar
service. In this article we saw how to connect to a
Hotmail mailbox, enumerate inbox mail items, and send
and retrieve e-mail, using VB.NET and the XMLHTTP
component. Sample code accompanying this article
contains a .NET assembly, demonstrating that connecting
to Hotmail via HTTPMail can be as simple as working with
any other mail protocol such as POP3, IMAP4 or SMTP.