Long time, no posts and whatnot. To break the ice, here’s something that’s fairly well documented, but not necessarily that obvious, that I learned today.
TL;DR: keep your MongoClients as singletons, like the documentation damn well tells you to do. A more detailed explanation follows.
Yesterday, I deployed an app to production. It was a fairly major deployment in that it involved upgrading the infrastructure: I had updated the app to use the MongoDB C# driver 2.x series, and also upgraded the actual production database from MongoDB 2.x to the latest 3.6. The app was fairly well tested, but of course, the one thing that very rarely gets tested for is production load.
I had performed the driver upgrade as a fairly mechanical search-and-replace type exercise, and while most operations were easy to replace, the one thing that was missing was the ability to disconnect from the server. I hit the documentation, and found out that it says the following:
It is recommended to store a MongoClient instance in a global place, either as a static variable or in an IoC container with a singleton lifetime.
However, multiple MongoClient instances created with the same settings will utilize the same connection pools underneath.
Reading a bit more on the topic confirmed that there was no need to disconnect the client, so all was well in the world. What I didn’t do, however, was to register the client as a Singleton — because I wanted to keep the changes to a minimum, and the docs stated that it wasn’t required, even if it was recommended.
In the next changeset, I added some telemetry: I wanted to log the duration of MongoDB operations, so I could use our monitoring to see if our Mongo operations got very slow. I found a post titled Monitoring MongoDB with Application Insights and followed its instructions. And here’s where things went wonky.
See, the article’s example uses a lambda function as the cluster configurator, which isn’t bad as such, but. The documentation I quoted above? In full context, it looks like this:
However, multiple MongoClient instances created with the same settings will utilize the same connection pools underneath. Unfortunately, certain types of settings are not able to be compared for equality. For instance, the ClusterConfigurator property is a delegate and only its address is known for comparison. If you wish to construct multiple MongoClients, ensure that your delegates are all using the same address if the intent is to share connection pools.
Combine that with the fact that my MongoClient registration was per-request, and ta-da, I’ve effectively disabled connection pooling with no possibility to dispose of the connections. So perhaps it wasn’t that surprising to see MongoDB log the following:
2018-01-18T08:17:35.340+0200 I NETWORK [listener] connection accepted from 127.0.0.1:52669 #4457 (4455 connections now open)
Fortunately, the fix was rather simple: move the client registration to be a singleton, and that’s it.
Mind you, there’s nothing wrong with the instructions in the post I linked to. Had I treated the client instance like the docs suggested, I wouldn’t have had any problems.
Moral of the story? If the documentation recommends something, it’s probably a good idea to do it, I guess.