High Speed, Efficient In-Memory And Persistent Remote Dictionary Server (Redis)

We will be coding a Windows Form application to show the start-up company information.

Here’s the definition from Redis website.

“Redis is an open source (BSD licensed), in-memory data structure store, used as database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperlog and geospatial indexes with radius queries. Redis has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster”.

Redis is one of the most popular NoSQL key-value and in-memory based data store. Its open source and easy to use software. There exists numerous client libraries for various programming languages. Although the key-value pairs are stored in-memory, Redis has a mechanism to persist data on disk and load the same on start of Redis server. The data is written into a file and is in a specific format i.e. “RDB” type. You might see a file named “dump.rdb” with the Redis installation directory; which is nothing but an optimized binary representation of the data that you intend to store in-memory.

Redis persistence explained in wiki.

“By default, Redis syncs data to the disk at least every 2 seconds, with more or less robust options available if needed. In the case of a complete system failure on default settings, only a few seconds of data would be lost.”

Setting up Redis on Windows

Download and install the latest Redis setup program located at the following link.

Starting the Redis Server

Redis server can be started either by running the executable file or starting the windows service “Redis Server”.

Navigate to the installation folder, default is “C:\Program Files\Redis” and double click on “redis-server.exe” to run the Redis server.





Redis Demo

The use case that we are dealing is to show up the start-up company information. We will be coding an Windows Form sample demo app named “Company Info” and allow the user to key in the company name (including wildcard characters) to display the list of matching company names and allow the user to select one, so we can connect to Redis and fetch the company related information based on the key which is nothing but the company name.

Before we start coding the application. The Redis client that we are making use of is, “csredis”. It’s open source and free to use. We are going to install the same from Nuget.



Company Library Class Diagram

The following is the snapshot of the Company Library class diagram. This is just for your reference to understand the various models that one can use in the application.



Redis Data Load

Let us now take a look in the data load program. At first, before running the “Company Info” Windows Form App, the data load program has to be executed. We are going to use a static file for loading the start-up company information and then we are going to iterate and make a call to Redis client to load the data in-memory.

The sample file that I’m using can be downloaded from the following link.

I happened to tweak the data file by replacing “$date” with “date” and “$oid” with “oid” so that we can properly de-serialize the JSON data using JSON library.

Please note – The data file is separately attached with this article. Please download the same, extract and cut/copy the “companies.json” file to “DataFile” folder of RedisDataLoad program and run the console application.

The following is the code snippet for loading the data file to Redis in memory data structure.

Here’s what we do.

1. Get the host from the application configuration.
   
   
2. Get the executable path so that we can format the data path.
   
   
3. Construct the data path by formatting the string with the executable directory path and the data file path.
   
   
4. Use “File” class and make a call to the static method, ReadAllLines passing in the data file path to read all the file contents.
   
   
5. Create a new instance of RedisClient by specifying the host.
   
   
6. Iterate through the companies and de-serialize the company item to get the company name. The file that we are reading has the company information in JSON. So we are going to make use of a popular JSON library named “Newtonsoft.Json” for de-serializing the company JSON string to CompanyInfo instance.
   
   
7. Make a call to “Set” method of redis instance by passing in the company name and company info JSON string. Note – We are taking the company name as a key here. Redis will hold all the company information in-memory. Later we are going to access the company info based on the key i.e. company name.

static void Main(string[] args)
{
    string host = ConfigurationManager.AppSettings["Host"].ToString();
    var exePath = Path.GetDirectoryName(Application.ExecutablePath);
    var dataPath = string.Format("{0}\\{1}", exePath, "DataFile\\companies.json");
    var companies = File.ReadAllLines(dataPath);
    Console.WriteLine("Started Loading Company Info.");
    Console.WriteLine(string.Format("{0} - {1}", "Begin Time", DateTime.Now.ToString()));
    using(var redis = new RedisClient(host))
    {
        foreach(var company in companies)
        {
            var companyInfo = JsonConvert.DeserializeObject < CompanyLib.CompanyInfo > (company);
            Console.WriteLine(string.Format("{0}-{1}", "Loading company", companyInfo.name));
            redis.Set(companyInfo.name, company);
        }
    }
    Console.WriteLine(string.Format("{0} - {1}", "End Time", DateTime.Now.ToString()));
    Console.WriteLine("Completed Loading Company Info.");
    Console.ReadLine();
}

You can see below, how the company JSON is being de-serialized. At this time, we are just taking the company name.


 
Redis sample demo app – Company Info

Here’s the snapshot of the Redis sample demo application for showing the start-up company information.



Let us dig into the code and try to understand the inner workings. The following is the code snippet of “Search” button click.

We are creating an instance of “RedisClient” by specifying the host to connect. Make a call to “Keys” method of redis client passing in the full or partial wild card based key, which will get all the unique keys from Redis. Once we obtain the keys, we can get clear the ListView and then iterate through the keys and add each of them to ListView.

private void btnSearch_Click(object sender, EventArgs e)
{
    using(var redis = new RedisClient(host))
    {
        var keys = redis.Keys(txtCompanyName.Text.Trim());
        if(keys.Length > 0)
        {
            listView1.Clear();
            foreach(var key in keys)
            {
                listView1.Items.Add(key);
            }
        }
    }
}

Now we know how the search works, but that’s just for displaying the company names. Let us take a look into the ListView item selection code. On ListView Selected Index Changed is where we will be getting the selected company name. Once we get the selected company, we can make a call to Redis to get the full company information by using company name as key.

private void listView1_SelectedIndexChanged(object sender, EventArgs e)
{
    if(listView1.SelectedIndices.Count <= 0)
    {
        return;
    }
    Clear();
    SetVisibilityTrue();
    ListView.SelectedListViewItemCollection selectedItemCollection = ((ListView) sender)
        .SelectedItems;
    var companyInfo = GetCompanyInfo(selectedItemCollection[0].Text);
    if(companyInfo != null && companyInfo.offices.Count > 0)
    {
        Address1.Text = companyInfo.offices[0].address1;
        Address2.Text = companyInfo.offices[0].address2;
        City.Text = companyInfo.offices[0].city;
        State.Text = companyInfo.offices[0].state_code;
        Zip.Text = companyInfo.offices[0].zip_code;
        Country.Text = companyInfo.offices[0].country_code;
        txtCompanyOverview.Text = companyInfo.overview.Replace("<p>", "")
            .Replace("</p>", "");
        SetCompanyAcquisitionInfo(companyInfo);
    }
}

The following is the code snippet to set the company acquisition information. The code is self-explanatory.

private void SetCompanyAcquisitionInfo(CompanyInfo companyInfo)
{
    if(companyInfo.acquisition != null)
    {
        if(companyInfo.acquisition.acquired_day != null) AcquiredDay.Text = companyInfo.acquisition.acquired_day.ToString();
        if(companyInfo.acquisition.acquired_month != null) AcquiredMonth.Text = companyInfo.acquisition.acquired_month.ToString();
        if(companyInfo.acquisition.acquired_year != null) AcquiredYear.Text = companyInfo.acquisition.acquired_year.ToString();
        if(companyInfo.acquisition.price_amount != null) Price.Text = companyInfo.acquisition.price_amount.ToString();
        if(companyInfo.acquisition.price_currency_code != null) PriceCurrency.Text = companyInfo.acquisition.price_currency_code.ToString();
    }
}

Here’s the code snippet to get the company information from Redis. All we have to do is, create an instance of RedisClient by specifying the host. Make a call to “Get” method of redis client by passing in the company name as key so that will return the full company info as JSON string. If we have a company info, we can de-serialize the same by using our company class library entity named “CompanyInfo” and we return the same to caller. 

private CompanyInfo GetCompanyInfo(string companyName)
{
    using(var redis = new RedisClient(host))
    {
        var companyInfo = redis.Get(companyName);
        if(companyInfo != null) return JsonConvert.DeserializeObject < CompanyInfo > (companyInfo);
    }
    return null;
}

Conclusion

Redis is a highly efficient In-memory data structure store available for free. Persistence is built in with Redis. Apart from managing key-values pairs in memory, Redis provides numerous features like pub-sub, automatic failover, etc. One can literally use Redis as a database, message broker or key-value cache. There are folks make use of Redis as a Session storage.