RSS Feed Project in VB.NET

Introduction
 
 The RSS Feed project is aimed as demonstrating writing VB code to consume RSS feeds from the Internet and putting the data from these RSS feeds into a database for you to use in your own applications. In my sample we are concerned with getting a link, title and description for each of the items in the feed. We will be consuming the RSS feeds provided by SlashDot.org and WiredNews.com.
 
Project Setup
 
To run the project you will need to create a Sql Server database and run the RssFeed.sql script. RssFeed.sql can be found in the support directory in the zip file. This will create the tables and stored procedures you will neeed. It also populates the WebNewsSource with the data needed to use the RSS feeds from SlashDot.org and WiredNews.com. You will also need to modify the connection stiring in the Rss Feed.exe.config to point to your database. The Rss Feed.exe.config needs to be in the same directory as the exe that you are running. When running the app from the IDE in debug mode the exe is in the bin\debug directory.
 
The Database
 
The database I designed for this application consists of two tables. WebNewsSource holds the information we need to connect to a RSS feed and the field names of the data that we want to keep (Link, title and description). The WebNewsItem table is the table that we will use to hold the data we retrive from the feeds.
 
 

 Defining a RSS Feed
 
An RSS Feed is an especially formated XML document. I used a database to hold the data that the application will need to know how to consume the RSS feed. The data is kept in the WebNewsSource table. The application will need to know the URL of the RSS feed; this data is kept in the RSSURL field. The RSSSelect field tells the application how to get to the nodes in the RSS Feed that we are interested in. The RSSTitle, RSSLink and RSSDescription fields tell the application which pieces of data we want to store in the Title, Link and ArticleDescriptions fields respectively in the WebNewsItem table.
 
 The Code
 
 The sample contains two classes. Feed.vb is class that will get the RSS feed as an XML document and go through it to get the data we are looking for. RSSItem.vb will read the RSS Feed information from the database and hold it. We need to use the System.Xml.XPath, System.Data, and System.Data.SqlClient namespaces.
 
 Imports System.Xml.XPath
 Imports System.Data
 Imports System.Data.SqlClient
 
 The GetFeed() method of the Feed class will get the RSS Feed and bring it back into a XPathDocument using a select that we defined in our database. It then loops through the nodes of the document passing each node to the ProcessNode method.
 
 Public Sub GetFeed()
 Try
 ' Bring back the Feed
 Dim doc As New XPathDocument(rssURL)
 Dim nav As XPathNavigator = doc.CreateNavigator()
 Dim iter As XPathNodeIterator = nav.Select(rssSelect)
 ' Loop through the nodes
 While iter.MoveNext()
 ' Get the data we need from the node
 ProcessNode(iter.Current)
 End While
 Catch ex As Exception
 Console.WriteLine(ex.Message.ToString())
 End Try
 End Sub 'GetFeed
 
 The ProcessNode method gets the child nodes of the passed XPathNavigator and loops through them. We are looking for three nodes (Link, Title and Description) as we defined them in the database. We then make sure that the link and title are at least 10 characters long and pass the WebNewsSourceID, link, title and description to the InsertNews method.
 
 Sub ProcessNode(ByVal lstNav As XPathNavigator)
 Dim title As String = ""
 Dim link As String = ""
 Dim description As String = ""
 ' Get the child nodes
 Dim iterNews As XPathNodeIterator = lstNav.SelectDescendants(XPathNodeType.Element, False)
 ' Loop through the child nodes
 While iterNews.MoveNext()
 ' Save the current Name
 Dim rssName As String = iterNews.Current.Name
 ' Is this the title?
 If rssName.ToUpper() = rssTitle.ToUpper() Then
 title = iterNews.Current.Value
 End If
 ' Is this the Link?
 If rssName.ToUpper() = rssLink.ToUpper() Then
 link = iterNews.Current.Value
 End If
 ' Is this the Description?
 If rssName.ToUpper() = rssDescription.ToUpper() Then
 description = iterNews.Current.Value
 End If
 End While
 ' Make sure the link and title are at least 10 characters long
 If link.Length > 10 And title.Length > 10 Then
 ' Update the database
 InsertNews(webNewsSourceID, link, title, description)
 End If
 End Sub 'ProcessNode
 
 The InsertNews method reads the connection string from the config file. Sets up a command to run the upWebNewsItem_Insert stored procedure and executes it.
 
 Sub InsertNews(ByVal webNewsSourceID As Integer, ByVal link As String, ByVal title As String, ByVal description As String)
 Try
 ' Create Connection
 Dim objConnection As New SqlConnection
 System.Configuration.ConfigurationSettings.AppSettings("ConnectString"))
 ' Set up the command
 Dim objCommand As New SqlCommand("upWebNewsItem_Insert", objConnection)objCommand.CommandType = CommandType.StoredProcedure
 ' Set up the parameters
 objCommand.Parameters.Add("@WebNewsSourceID", SqlDbType.Int).Value = webNewsSourceID
 objCommand.Parameters.Add("@Link", SqlDbType.VarChar, 255).Value = link
 objCommand.Parameters.Add("@Title", SqlDbType.VarChar, 500).Value = title
 objCommand.Parameters.Add("@ArticleDescription", SqlDbType.VarChar, 1000).Value = description
 objCommand.Parameters.Add("@ID", SqlDbType.Int)
 objCommand.Parameters("@ID").Direction = ParameterDirection.Output
 ' Execute the command
 objConnection.Open()
 objCommand.ExecuteNonQuery()
 objConnection.Close()
 Catch ex As Exception
 Console.WriteLine(("ERROR:" + ex.Message.ToString()))
 End Try
 End Sub 'InsertNewsvIntroduction

The RSS Feed project is aimed as demonstrating writing VB code to consume RSS feeds from the Internet and putting the data from these RSS feeds into a database for you to use in your own applications. In my sample we are concerned with getting a link, title and description for each of the items in the feed. We will be consuming the RSS feeds provided by SlashDot.org and WiredNews.com.
 
 Project Setup
 
To run the project you will need to create a Sql Server database and run the RssFeed.sql script. RssFeed.sql can be found in the support directory in the zip file. This will create the tables and stored procedures you will neeed. It also populates the WebNewsSource with the data needed to use the RSS feeds from SlashDot.org and WiredNews.com. You will also need to modify the connection stiring in the Rss Feed.exe.config to point to your database. The Rss Feed.exe.config needs to be in the same directory as the exe that you are running. When running the app from the IDE in debug mode the exe is in the bin\debug directory.
 
 The Database
 
The database I designed for this application consists of two tables. WebNewsSource holds the information we need to connect to a RSS feed and the field names of the data that we want to keep (Link, title and description). The WebNewsItem table is the table that we will use to hold the data we retrive from the feeds.
 
 
 Defining a RSS Feed
 
An RSS Feed is an especially formated XML document. I used a database to hold the data that the application will need to know how to consume the RSS feed. The data is kept in the WebNewsSource table. The application will need to know the URL of the RSS feed; this data is kept in the RSSURL field. The RSSSelect field tells the application how to get to the nodes in the RSS Feed that we are interested in. The RSSTitle, RSSLink and RSSDescription fields tell the application which pieces of data we want to store in the Title, Link and ArticleDescriptions fields respectively in the WebNewsItem table.
 
 The Code
 
 The sample contains two classes. Feed.vb is class that will get the RSS feed as an XML document and go through it to get the data we are looking for. RSSItem.vb will read the RSS Feed information from the database and hold it. We need to use the System.Xml.XPath, System.Data, and System.Data.SqlClient namespaces.
 
 Imports System.Xml.XPath
 Imports System.Data
 Imports System.Data.SqlClient
 
 The GetFeed() method of the Feed class will get the RSS Feed and bring it back into a XPathDocument using a select that we defined in our database. It then loops through the nodes of the document passing each node to the ProcessNode method.
 
 Public Sub GetFeed()
 Try
 ' Bring back the Feed
 Dim doc As New XPathDocument(rssURL)
 Dim nav As XPathNavigator = doc.CreateNavigator()
 Dim iter As XPathNodeIterator = nav.Select(rssSelect)
 ' Loop through the nodes
 While iter.MoveNext()
 ' Get the data we need from the node
 ProcessNode(iter.Current)
 End While
 Catch ex As Exception
 Console.WriteLine(ex.Message.ToString())
 End Try
 End Sub 'GetFeed
 
 The ProcessNode method gets the child nodes of the passed XPathNavigator and loops through them. We are looking for three nodes (Link, Title and Description) as we defined them in the database. We then make sure that the link and title are at least 10 characters long and pass the WebNewsSourceID, link, title and description to the InsertNews method.
 
 Sub ProcessNode(ByVal lstNav As XPathNavigator)
 Dim title As String = ""
 Dim link As String = ""
 Dim description As String = ""
 ' Get the child nodes
 Dim iterNews As XPathNodeIterator = lstNav.SelectDescendants(XPathNodeType.Element, False)
 ' Loop through the child nodes
 While iterNews.MoveNext()
 ' Save the current Name
 Dim rssName As String = iterNews.Current.Name
 ' Is this the title?
 If rssName.ToUpper() = rssTitle.ToUpper() Then
 title = iterNews.Current.Value
 End If
 ' Is this the Link?
 If rssName.ToUpper() = rssLink.ToUpper() Then
 link = iterNews.Current.Value
 End If
 ' Is this the Description?
 If rssName.ToUpper() = rssDescription.ToUpper() Then
 description = iterNews.Current.Value
 End If
 End While
 ' Make sure the link and title are at least 10 characters long
 If link.Length > 10 And title.Length > 10 Then
 ' Update the database
 InsertNews(webNewsSourceID, link, title, description)
 End If
 End Sub 'ProcessNode
 
 The InsertNews method reads the connection string from the config file. Sets up a command to run the upWebNewsItem_Insert stored procedure and executes it.
 
 Sub InsertNews(ByVal webNewsSourceID As Integer, ByVal link As String, ByVal title As String, ByVal description As String)
 Try
 ' Create Connection
 Dim objConnection As New SqlConnection
 System.Configuration.ConfigurationSettings.AppSettings("ConnectString"))
 ' Set up the command
 Dim objCommand As New SqlCommand("upWebNewsItem_Insert", objConnection)objCommand.CommandType = CommandType.StoredProcedure
 ' Set up the parameters
 objCommand.Parameters.Add("@WebNewsSourceID", SqlDbType.Int).Value = webNewsSourceID
 objCommand.Parameters.Add("@Link", SqlDbType.VarChar, 255).Value = link
 objCommand.Parameters.Add("@Title", SqlDbType.VarChar, 500).Value = title
 objCommand.Parameters.Add("@ArticleDescription", SqlDbType.VarChar, 1000).Value = description
 objCommand.Parameters.Add("@ID", SqlDbType.Int)
 objCommand.Parameters("@ID").Direction = ParameterDirection.Output
 ' Execute the command
 objConnection.Open()
 objCommand.ExecuteNonQuery()
 objConnection.Close()
 Catch ex As Exception
 Console.WriteLine(("ERROR:" + ex.Message.ToString()))
 End Try
 End Sub 'InsertNewsv