MongoDB Cluster In Kubernetes(2): Create a UserDB ReplicaSet

Posted on Edited on In 评论: Views:

This is part2, we will create a user database that is a 3 instances ReplicaSet.

MongoDB Ops Manager Series:

  1. Install MongoDB Ops Manager
  2. Create a UserDB ReplicaSet
  3. Expose UserDB to Public
  4. Openssl Generates Self-signed Certificates
  5. Enable UserDB TLS and Auth

The so called Application Database is the backend DB of Ops Manager. It cannot be used to store user data. The user database is called MongoDB Deployment. Note that the deployment is different with Kubernetes deployment.

For simplicity, we will create a replicaset instead of a sharded cluster. The configuration process is almost the same. Because sharded cluster consists of more components, the configuration is more complicated. Let's start with an easier one. :-)

Reference: Deploy MongoDB Sharded Cluster by Ops Manager

Unfortunately, creating user database cannot be directly done by the Ops Manager web UI. Most of the operations should be performed by the MongoDB Kubernetes Operator, aka the kubectl command line. If you try to create it through web UI, you need to manually configure variety of parameters and install Agents which I don't know how to do it. So to my understanding, Ops Manager is a great tool for visualization and automation tasks, but not good at cluster manipulation. I think the reason is that the cluster specification is too complicated to incorperated to the UI.

The official blog created user database by a new generated API key. However, I always failed with this method. I bypass the issue by creating the user database by ops-manager-admin-key. Though it's not recommended, it works. Update 2020-12-23: Please follow the official approach below. The 401 unauthorized issue is caused by that the API Key Secret username is not the same with the admin username.

The Official Approach

Generate a Public API Key

Go to Ops Manager, "UserName -> Account -> Public API Access", click the top right-most "Account": Generate Public API key

Note that the <apikey> appears only once, please well remembered.

Create API Key Secret

Make sure that the user specified here is the same with the Ops Manager admin user!(created at Install MongoDB Ops Manager) 

$ kubectl create secret generic om-user-credentials --from-literal="user=xxx@xxx.com" --from-literal="publicApiKey=<apikey>" -n mongodb
secret/om-user-credentials created

Create Ops Manager Connection ConfigMap

Find Ops Manager url first: 

$ kubectl get om ops-manager -o jsonpath='{.status.opsManager.url}' -n mongodb
http://ops-manager-svc.mongodb.svc.cluster.local:8080

Create ops-manager-connection ConfigMap with the above URL： 

$ kubectl create configmap ops-manager-connection --from-literal="baseUrl=http://ops-manager-svc.mongodb.svc.cluster.local:8080"  -n mongodb
configmap/ops-manager-connection created

Create User Database userdb

Create the user database by kubectl, edit the userdb.yaml and fill in credential and configMap fields: 

apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: userdb
  namespace: mongodb
spec:
  members: 3
  version: 4.2.2-ent
  type: ReplicaSet

  opsManager:
    configMapRef:
      name: ops-manager-connection
  credentials: om-user-credentials

$ kubectl apply -f userdb.yaml -n mongodb
mongodb.mongodb.com/userdb created

Simple(Crude) Approach (Not Recommended)

Please follow the official approach above.

You might encounter the following error after apply userdb.yaml: 

$ kubectl get mdb -n mongodb
NAME     TYPE         STATE    VERSION     AGE
userdb   ReplicaSet   Failed   4.2.2-ent   86s
$ kubectl describe mdb -n mongodb
Name:         userdb
Namespace:    mongodb
Labels:       <none>
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"mongodb.com/v1","kind":"MongoDB","metadata":{"annotations":{},"name":"userdb","namespace":"mongodb"},"spec":{"credentials":...
API Version:  mongodb.com/v1
Kind:         MongoDB
Metadata:
  Creation Timestamp:  2020-12-10T08:19:02Z
  Generation:          1
  Resource Version:    52201207
  Self Link:           /apis/mongodb.com/v1/namespaces/mongodb/mongodb/userdb
  UID:                 8c04df70-5249-4c3b-8a28-3907890e4eb8
Spec:
  Credentials:  om-user-credentials
  Members:      3
  Ops Manager:
    Config Map Ref:
      Name:  ops-manager-connection
  Type:      ReplicaSet
  Version:   4.2.2-ent
Status:
  Last Transition:      2020-12-10T08:20:33Z
  Message:              Failed to prepare Ops Manager connection: Error reading or creating project in Ops Manager: Status: 401 (Unauthorized), Detail: You are not authorized for this resource.
  Observed Generation:  1
  Phase:                Failed
  Version:
Events:                 <none>

401 (Unauthorized) error, it seems that the generated key is not valid.

I found an API key in the Public API access page by chance. Although the whole key is partially shown in the Ops Manager, we can found the original key in the kubernetes secret: ops-manager-admin-key. Let's check the last bits of the ops-manager-admin-key in the Ops Manager: OpsManager Public API Key

Modify the last line of userdb.yaml： 

apiVersion: mongodb.com/v1
kind: MongoDB
metadata:
  name: userdb
  namespace: mongodb
spec:
  members: 3
  version: 4.2.2-ent
  type: ReplicaSet

  opsManager:
    configMapRef:
      name: ops-manager-connection
  credentials: ops-manager-admin-key
Then apply it. It works!

Again, this method is not recommended, but till now I don't find other alternatives.

A new statefulset userdb is creating, wait until the state becomes "Running": 

$ kubectl get mdb -n mongodb
NAME     TYPE         STATE     VERSION     AGE
userdb   ReplicaSet   Running   4.2.2-ent   13m

Refresh the Ops Manager UI, click All Clusters, you will find that there are 2 clusters with corresponding context and Project: MongoDB Ops Manager All Clusters

View userdb processes: UserDB Processes Found that both TLS and AUTH are disabled. The transport layer communication is not encryped, username/password is not required.

View userdb servers: UserDB Servers

Note that the server address is something like: userdb-0.userdb-svc.mongodb.svc.cluster.local, which is an internal address. Therefore, we cannot connect to the userdb through Mongo Client.

Now the user database is sucessfully created!

Next we will expose the user database to the public by creating services: Expose UserDB to Public