Augmenting MongoDB LINQ with low level mongo query.

2020, Aug 18    

.NET landscape change back in November 2017 when .NET Framework 3.5 was release which included the Language Integrated Query, which we all call today as “LINQ”. This release also included some new C#3 language feature that enabled us to write a SQL-like language in the middle of our C# code.

var q = from x in data
            where x % 2 == 1
            orderby x descending
            select x * x;

However, the alternate (and also more common today) way to write LINQ is using the functions with lambdas.

var q = data.Where(x => x % 2 == 1)
          .OrderByDescending(x => x)
          .Select(x => x * x);

LINQ release back in 2017 was revolutionary due to many reasons. It is a general-purpose query language, this means that it works with any types and collections, that means we can use collections of objects, databases, json, xml and our query syntax is always the same.

The reduction in amount of code to be written was substantial as all the loops were taken care of by LINQ, thus making it a lot easier to read and maintain.

As a C# developer in 2020, even the simplest bits of aggregation in collections is done by LINQ as it’s simple, concise and easy to maintain.

MongoDB and LINQ

MongoDB C# Driver has support for LINQ, this means that we can write the above style queries without having to care about how that gets translated and executed on our MongoDB Database.

var client = new MongoClient();
var db = client.GetDatabase("test");
var collection = db.GetCollection<Book>("test");

var result = await collection.AsQueryable()
    .Where(x => x.Title == "Matching title")
    .Select(x => x.Title)
    .ToListAsync();

As you can see above we’re calling a method .AsQueryable() on our MongoDB collection, which is then returning an IQueryable<Book>. This means we can do all our normal LINQ operations on it.

Not Everything is Supported

As you can imagine not everything is going to be supported in LINQ, this is due to LINQ being an abstraction, it is general purpose and not just designed for MongoDB. This is a limitation that other LINQ Providers have too such as Entity Framework.

For example if we change around the code above to find every book that has its title having more than 5 ‘a’ characters, The MongoDB LINQ Provider will throw an InvalidOperationException.

var result = await collection.AsQueryable()
    .Where(x => x.Title.Count(c => c == 'a') > 5)
    .Select(x => x.Title)
    .ToListAsync();
System.InvalidOperationException: '{document}{Title}.Count(c => (Convert(c, Int32) == 97)) is not supported.'

However if we ran the same code with just a List<T> instead of a MongoDB collection it would run perfectly fine.

This also works the other way too, where MongoDB supports more features than what LINQ can provide. For this we can’t use tell MongoDB to use a text search or geo location search by just using LINQ. This is where we need to augmenting our LINQ statement with some lower level MongoDB query.

Augmenting LINQ

Most LINQ providers have a way to inject in some custom support to enable some more native support for a feature. For example it’s common for people using EF Core to use the EF.Property static method to access shadow properties when executing queries using LINQ.

context.Blogs
    .OrderBy(b => EF.Property<DateTime>(b, "LastUpdated"));

MongoDB Driver like other LINQ providers supports a way to inject in more functionality.

We can build up extra queries with the MongoDB filter definition builders and then inject these in to a LINQ queries.

Below is an example of how to create a text search and then inject it in to our LINQ query that MongoDB Driver will execute.

var collection = db.GetCollection<Book>("test");

var filter = Builders<Book>.Filter.Text("Mary Doe");

var results = await collection.AsQueryable()
    .Where(_ => filter.Inject())
    .Select(x => x.Title)
    .ToListAsync();

We can take any filter definition and call the Inject extension method within a where statement and the MongoDB LINQ provider will do some magic to inject in that query at the right point.

This is great for querying for anything that is not native to LINQ such as geo queries, text queries.

Augmenting LINQ Caveat

The caveat to injecting in a native MongoDB query is that now the LINQ statement will only run on a LINQ provider that understands what to do when the Inject method is called within an where expression.

If we try to run the above LINQ statement on just a standard List<Book> type, you’ll be presented with an InvalidOperationException with a message of The LinqExtensions.Inject method is only intended to be used in LINQ Where clauses..

This can make it hard to test if you’re just testing you LINQ queries in isolation to the database.