MongoDB Atlas search Index in C# - Build aggregation pipeline

It's about the end-to-end implementation of the Atlas search index in C#.

Introduction

MongoDB Atlas search Index in C# - Build aggregation pipeline with conditional filters, sort, and projections

In this blog, I will describe how to search the documents in a MongoDB collection by a text or phrase and by the conditional filters, and even how to build or execute an aggregation pipeline with sort and projection for the same.

First, you should know about the Atlas search index and aggregation pipeline in Atlas MongoDB. I will explain how we can build and execute an aggregation pipeline similar to the one below.

The pipeline definition is very much important. Look at a line of declaration, var searchStage, and how it is defined. I have mentioned the index name, whether to search by text or phrase, the parameters query, and the path. These things have been saved as constants.

public const string SEARCH_INDEX = "App-Search";
public const string SEARCH_PATH = "['appName','shortName','longName']";
public const string SEARCH_BY_TEXT = "text";
public const string SEARCH_BY_PHRASE = "phrase";

I have created a class RandomSearch which inherits the base class BaseSearch. You can skip this and have it all in one class for practice/study purposes.

internal class RandomSearch: BaseSearch {
    public RandomSearch(SearchAppDto dto) {
      SearchKeyword = dto.SearchKeyword.ToLower();
      KeywordFilter = Builders<CollectionRecord>.Filter.Empty;

      if (dto.SearchKeyword.ToUpper().Contains(Constant.APP_ID_PREFIX)) {
        KeywordFilter = Builders<CollectionRecord>.Filter.Where(s => s.AppId.ToLower().Contains(SearchKeyword));
      }
      if (SearchKeyword == Constant.TYPE1_APPS.ToLower()) {
        KeywordFilter = Builders<CollectionRecord>.Filter.Where(s => s.Type1.ToLower().Contains(Constant.TIER1));
      } else if (SearchKeyword == Constant.TYPE2_APPS.ToLower()) {
        KeywordFilter = Builders<CollectionRecord>.Filter.Where(s => s.Type2);
      }

      SortBy(dto);
    }

    public override PipelineDefinition<CollectionRecord, CollectionRecord> Pipline {
      get {
        string searchBy = Constant.SEARCH_BY_PHRASE;
        if (SearchKeyword.Contains(Constant.APP_ID_PREFIX.ToLower())) {
          searchBy = Constant.SEARCH_BY_TEXT;
        }
        

        //Note - single and double quote issue, backslash issue
        SearchKeyword = SearchKeyword.Replace('"', ' ').Replace("'", "").Replace(@ "\", "
          ");

          var searchStage = @ "{$search: {index:'" + Constant.SEARCH_INDEX + "'," + searchBy + ": {query:'" + SearchKeyword + "', path:" + Constant.SEARCH_PATH + "}}}";
          var pipeline = new EmptyPipelineDefinition < CollectionRecord > ()
            .AppendStage < CollectionRecord, CollectionRecord, CollectionRecord > (searchStage)
            .Match(PanelFilter);

          return pipeline;
        }
      }
      //.....Some code here as well
    }

In a base class, I have declared the filter, sort, pipeline, and projection definitions variables.

internal abstract class BaseSearch
{
   protected string SearchKeyword { get; set; }

   protected FilterDefinition<CollectionRecord> KeywordFilter { get; set; }
        
   public SortDefinition<CollectionRecord> Sort { get; set; }

   public virtual PipelineDefinition<CollectionRecord, CollectionRecord> Pipline { get; set; }
         = new EmptyPipelineDefinition<CollectionRecord>();        


   public ProjectionDefinition<CollectionRecord> Projection { get {
        return new[] { nameof(CollectionRecord.AppId), 
               nameof(CollectionRecord.AppName), nameof(CollectionRecord.AppState),
               nameof(CollectionRecord.ApplicationUrl)}
               .Projection<CollectionRecord>();
      }
}

We can specify the sort definition as specified below.

internal class RandomSearch: BaseSearch {
//..Some Code

protected virtual void SortBy(SearchDto dto) 
{
  Sort = Builders<CollectionRecord>.Sort.Ascending(i => i.LongName);

  if (dto.OrderBy == SortFilter.LongNameDesc.ToString())
    Sort = Builders<CollectionRecord>.Sort.Descending(i => i.LongName);

  else if (dto.OrderBy == SortFilter.MostUsed.ToString())
    Sort = Builders<CollectionRecord>.Sort.Descending(i => i.ActiveUserCount);

  //else if ()
  //You can keep checking
}

//..Some Code....
}

I am calling the repository class method from a service class mentioned below.

public async Task<SearchResultDto> SearchApplications(string userName, SearchDto searchModel)
{
  IEnumerable<CollectionRecord> entities;
  BaseSearch search;
  search = new RandomSearch(searchModel);
  entities = await _collectionRepository.AggregateAsync(search.Pipline, search.Sort, search.Projection);

  if (entities?.Count() > 0)
  {
    //applying some logic then returning the final result
    return await GetFinalizedResult(userName, searchModel, entities);
  }

  return new SearchResultDto();
}

And in the repository class, the aggregation pipeline is being executed.

public Task<List<CollectionRecord>> AggregateAsync(PipelineDefinition<CollectionRecord, CollectionRecord> pipeline,
  SortDefinition<CollectionRecord> sortDefinition = null, ProjectionDefinition<CollectionRecord, CollectionRecord> projection = null)
{

  if (projection != null && sortDefinition != null)
    pipeline = pipeline.Sort(sortDefinition).Project(projection);

  else if (projection != null)
    pipeline = pipeline.Project(projection);

  return Collection.Aggregate(pipeline).ToListAsync();
}

Summary

Hope most of the things have been cleared. I tried simplifying the Atlas Search index implementation with the filters, sort, and projections in C#.

Thank you so much!!