Mongodb Healthcheck Step by Step

Step by step healthcheck for mongodb.

Step1 : Is authorization enabled for security.

vi /path/mongodb/bin/mongo.conf security: authorization: enabled

Step2 : Incoming operations in real time.

mongostat

Step3 : Which collections spending time reading/writing.

mongotop

Step4 : Network traffic is monitored.

mongoreplay

Step5 : “Insert,delete,update,memory,connection” operations is monitored in real time.

mongostat --username=system --password=*** --authenticationDatabase=admin -o insert,update,delete,command,dirty,used,conn

Step6 : Network traffic is monitored between mongodb and replicaset

mongoreplay monitor -i 10 -e "port 27017"

Step7 : Mongodb error log is monitored.



Server log keys:
Access,command,control,geo,index,network,query,repl,sharding,storage,journal,write

Severity levels:
F - Fatal
E - Error
W - Warning
I - Informational (Verbosity level=0)
D - Debug (Verbosity level>0)

Check the MongoDB config file at /etc/mongod.conf or /yourMongoDBpath/mongod.conf, the logpath defined where to log.
logpath=/var/log/mongodb/mongod.log

cat /var/log/mongodb/mongod.log | grep -i error | cut -d ' ' -f3

Step7 : Resource usage is checked on unix server

top

Step7 : Server disk status is checked.
sar -d 2 5 
iostat 2 5

Step8 : Swap status is checked.
vmstat 2 5

Step9 : Unix system logs should be checked.
dmesg -T
tail -200f /var/log/messages

Step10 : Unix server filesystem disk is checked.
df -Ph

Step11 : Unix server memory is checked.
free -g

Step12 : Unix server uptime and load balance are checked.

uptime

Step13 : Unix server zombi process is checked.
ps -aux | awk '{print $8 " " $2 }' | grep -w Z

Step14 : Mongodb should be monitored on grafana . 

Grafana and prometheus were installed on monitoring server.
Mongodb_exporter was installed on mongodb server.

-Active connection 
-Available connection
-Memory
-Replica set lag
-Uptime
-Queued operations as read/write 
-Transaction as collections
-Command operations as "insert,update,delete"
-Page fault
-Last error
-Network io 
-Lock
-Error codes
-Vb...

Step15 : Mongodb connection in terminal.
mongo --host 192.168.12.67 --port 27017 --authenticationDatabase=admin
use admin 
db.auth("system","password****")

Step16 : Mongodb uptime is checked.
db.serverStatus().uptime

Step17 : Mongodb lock is checked in real time.
db.serverStatus().locks

Step18 : Mongodb connections are checked in real time.
db.serverStatus().connections

Step19 : Mongodb slow session is killed more than 100ms.
db.CurrentOp() 
db.killOp(3461)

Step20 : Mongodb stop operation if it is needed.
/mongopath/bin/mongo 
use admin 
db.shutdownServer()
or
mongod --shutdown

Step21 : Mongodb start operation if it is needed.
/$MONGO_HOME/bin/mongod --fork --config /mongopath/bin/mongo.conf --logpath /tmp/mongodb.log

or
cd /home/mongo/mongodb/mongodb/bin

export PATH=$PATH:/home/mongo/mongodb/mongodb/bin
./mongod --dbpath /home/mongo/mongodb/mongodb/data &
ps -eaf | grep mongo



Step22: Session unlock

cd /path/bin 
./mongo 
rs0:PRIMARY>show dbs 
rs0:PRIMARY>use cache;
rs0:PRIMARY>db.c.remove({"_id":""VVM_Test_Case:498});
WriteResult( {"nRemoved" : 1})
rs0:PRIMARY>

Step23 : Mongodb Backup 
mongodump --out /backup_path/`date +"%m-%d-%y"` --username system --password <password> --host <ip>:<port>

Step24 : Mongodb backup status
Mongo backup log can be analyzed.

Step25 : Mongodb version status 
db.serverStatus().version

Step26 : Replica set sync status 
db.printSlaveReplicationInfo()

Step27 : Fragmantation status 
db.chunks.find()

Step28 : Get summary for sharding 
db.printShardingStatus

Step29 : Lag status for replication 
rs.status()
rs.printReplicationInfo()

Step30 : Sysstat
db.stats()

Step31 : Sharding status
sh.status()

Step32 : Flush sql id
db.runCommand(
{
planClearCache: "orders"
}
)

Step33 : Query explain plan 
db.tablename.find(query).explain

Step34 : Drop create all indexes on collection
db.collection.reIndex()
db.accounts.reIndex()

Step35 : Compact Collection
db.runCommand({compact: collection-name})



Regards ,