XPath Using HtmlAgilityPack and WebClient

XPath, the XML Path Language, is a query language for selecting nodes from an XML document. The following code illustrates how to extract XPath using HtmlAgilityPack and a webclient on the fly.

You need to add a reference for HtmlAgilityPack; I've used version 1.4.0.1.

You can refer to http://htmlagilitypack.codeplex.com/releases/view/44954  to download the .dll

In this article we have used it to get multiple nodes using SelectNodesByPattern, which is an extension method.

We've used this url (http://htmlagilitypack.codeplex.com/releases/view/44954) as shown in the browser and found the value of "1.4.0 Stable".



We've tried to fetch a value of all heading "h1" tags having class ='page_title', kindly refer to the following:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using HtmlAgilityPack;
using System.Net;
using System.IO;

namespace DescendantUsingXPath
{
   public static class Program
    {
        static void Main(string[] args)
        {
            //WebClient object
            WebClient x = new WebClient();

            //Convert given url data to bytearray using DownloadData
            byte[] byteArray = x.DownloadData(new Uri("http://htmlagilitypack.codeplex.com/releases/view/44954"));

            //Convert Byte Array into Stram
            Stream stream = new MemoryStream(byteArray);

            //Create new object of HtmlAgilityPack.HtmlDocument
            HtmlAgilityPack.HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument();

            //To load stream into html object

            htmlDoc.Load(stream);
 
           //To get the value of descendent of form ,I tried to get value of all h1 having class=page_title ,
             //it'd retrun all the combinations of heading h1 having class page_title
           
            HtmlNodeCollection inputNodes = htmlDoc.DocumentNode.SelectNodesByPattern(@"descendant::form;//h
[@class='page_title']");
            foreach (HtmlNode node in inputNodes)
            {
                string strValue = node.InnerText;
                Console.WriteLine(strValue);
                Console.ReadLine();
            }
        }

        /// <summary>
        /// Returns HtmlNodeCollection based on given pattern based XPath.
        /// Example: "//body;//form;//table[3];//input"
        /// </summary>
        /// <param name="node">HtmlAgilityPack.HtmlNode</param>
        /// <param name="pattern">String</param>
        /// <returns>HtmlNodeCollection</returns>
        public static HtmlNodeCollection SelectNodesByPattern(this HtmlNode node, string pattern)
        {
            if (node == null)
                return null;
            if (!pattern.Contains(";"))
                return null;

            string[] expressions = pattern.Trim().Split(new char[] { ';' });

            HtmlNode tempNode = node;
            int incr = 0;

            for (incr = 0; incr < expressions.Length - 1; incr++)
                if (tempNode != null)
                    tempNode = tempNode.SelectSingleNode(expressions[incr]);

            if (tempNode != null)
                return tempNode.SelectNodes(expressions[incr]);

            return null;
        }

    }

}

Output