在一个分片MongoDB集群上并发执行transaction时遇到许多MongoCommandException错误:
code 251, codename NoSuchTransaction:
Command find failed: cannot continue txnId 4 for session 38604515-2584-45a5-a17a-5eb5d34ea6c4 - = with txnId 5. Command find failed: cannot continue txnId 4 for session 38604515-2584-45a5-a17a-5eb5d34ea6c4 - = with txnId 6. Command insert failed: cannot continue txnId 31 for session 3ed7ea61-eae1-440f-8d95-b6e066b35b69 - = with txnId 34.
When I execute MongoDB transactions in parallel, I encounter lots of
MongoCommandException: code 251, codename
NoSuchTransaction:
Command find failed: cannot continue txnId 4 for session 38604515-2584-45a5-a17a-5eb5d34ea6c4 - = with txnId 5. Command find failed: cannot continue txnId 4 for session 38604515-2584-45a5-a17a-5eb5d34ea6c4 - = with txnId 6. Command insert failed: cannot continue txnId 31 for session 3ed7ea61-eae1-440f-8d95-b6e066b35b69 - = with txnId 34.
Lightsquid is a handy log analyzer for squid proxy. However, the project is not maintained since 2009.
Today, I found lightsquid doesn't work in 2021. Why?
MongoDB sharded cluster is the most complicated architecture. The deployment of sharded cluster in Kubernetes is relatively hard. We will go through the deployment process by MongoDB Ops Manager in this post.
Before start, please go through the Create a UserDB ReplicaSet first.
A MongoDB sharded
cluster consists of the following components: - shard:
Each shard contains a subset of the sharded data. Each shard can be
deployed as a replica
set. - mongos:
The mongos acts as a query router, providing an interface
between client applications and the sharded cluster. - config
servers: Config servers store metadata and configuration settings
for the cluster.
In this post, we are going to create a sharded cluster with 2 shards (3 instances replica set), 2 mongos and 3 config servers.
MongoDB Ops Manager Series:
Sharded cluster是MongoDB部署中最复杂的形式，因为Sharded cluster的组件较多，部署步骤也更为繁琐。在实际部署中还有几个部署ReplicaSet时没遇到的证书和TLS问题。阅读本文前，强烈建议先阅读 创建用户数据库(replicaset) 作为基础。
一个Sharded cluster由三部分构成：
本文以创建一个2个Shard（每个由3实例ReplicaSet构成），2个mongos及3个config server的sharded cluster为例，演示具体部署流程。本文可做为Kubernetes部署MongoDB集群的番外篇。
整个系列：
This is part5, we will use the generated certficates to enable user database TLS and AUTH.
MongoDB Ops Manager Series:
This is part4, we will create a self-signed CA certificate and three server certificates.
MongoDB Ops Manager Series:
Self-signed certificates is not recommended for production. It cannot prevent man-in-the-middle attack. Since our main purpose is to encrypt the communication messages instead of authentication. Self-signed certificates is acceptable.
This is part3, we will expose the user database pods to the public so that Mongo client is able to access it.
MongoDB Ops Manager Series:
So far, the user database can be accessed only inside the kubernetes cluster. The official blog's approach is to expose the pods by NodePort: # Connect to a MongoDB Database Resource from Outside Kubernetes
I don't know why the official blog recommend NodePort. Obviously, LoadBalancer is a better way to expose MongoDB. We will use LoadBalancer to expose userdb pods.
This is part2, we will create a user database that is a 3 instances ReplicaSet.
MongoDB Ops Manager Series:
The so called Application Database is the backend DB of
Ops Manager. It cannot be used to store user data. The user database is
called MongoDB Deployment. Note that the
deployment is different with Kubernetes
deployment.
For simplicity, we will create a replicaset instead of a sharded cluster. The configuration process is almost the same. Because sharded cluster consists of more components, the configuration is more complicated. Let's start with an easier one. :-)
Reference: Deploy MongoDB Sharded Cluster by Ops Manager
Unfortunately, creating user database cannot be directly done by the
Ops Manager web UI. Most of the operations should be performed by the
MongoDB Kubernetes Operator, aka the kubectl
command line. If you try to create it through web UI, you need to
manually configure variety of parameters and install Agents which I
don't know how to do it. So to my understanding, Ops Manager is a great
tool for visualization and automation tasks, but not good at cluster
manipulation. I think the reason is that the cluster specification is
too complicated to incorperated to the UI.
It's pretty easy to configure a MongoDB standalone instance (almost zero configuration). However, if you want to run a production-level MongoDB cluster, the configuration process is non-trivial. For a production cluster, replication/sharding/dyanmic scaling/backup/transport encryption/monitoring are required. Is there a nice tool to help us?
MongoDB cluster is a distributed system, which is well suited to run
in Kubernetes. However, the collaboration of MongoDB instances usually
need to manually run commands on each instance which is independent of
Kubernetes. Therefore,
MongoDB Enterprise Kubernetes Operator is developed to
mitigate the gap. Morever, MongoDB Ops Manager is a great
web portal to help these automation tasks.
The whole deployment and configure process is a little bit long, I wrote a series of small pieces to make each of them compact and easy to follow:
Prerequisites
This is part1, we first introduce the
MongoDB Ops Manager components, then install it into your
Kubernetes cluster.