Mongodb Healthcheck Step by Step

 

 

 

 

 

 

 

 

 

Step by step healthcheck for mongodb.

 

Step1 : Is authorization enabled for security.

vi /path/mongodb/bin/mongo.conf security: authorization: enabled

Step2 : Incoming operations in real time.

mongostat

 

 

 

Step3 : Which collections spending time reading/writing.

mongotop



Step4 : Network traffic is monitored.

mongoreplay



Step5 : “Insert,delete,update,memory,connection” operations is monitored in real time.


mongostat --username=system --password=*** --authenticationDatabase=admin -o insert,update,delete,command,dirty,used,conn

Step6 : Network traffic is monitored between mongodb and replicaset

mongoreplay monitor -i 10 -e "port 27017"



Step7 : Mongodb error log is monitored.



Server log keys:
Access,command,control,geo,index,network,query,repl,sharding,storage,journal,write

Severity levels:
F - Fatal
E - Error
W - Warning
I - Informational (Verbosity level=0)
D - Debug (Verbosity level>0)

Check the MongoDB config file at /etc/mongod.conf or /yourMongoDBpath/mongod.conf, the logpath defined where to log.
logpath=/var/log/mongodb/mongod.log

cat /var/log/mongodb/mongod.log | grep -i error | cut -d ' ' -f3







Step7 : Resource usage is checked on unix server

 

top



Step7 : Server disk status is checked.
sar -d 2 5 
iostat 2 5




Step8 : Swap status is checked.
vmstat 2 5




Step9 : Unix system logs should be checked.
dmesg -T
tail -200f /var/log/messages


Step10 : Unix server filesystem disk is checked.
df -Ph



Step11 : Unix server memory is checked.
free -g

 

 

 

Step12 : Unix server uptime and load balance are checked.

uptime


Step13 : Unix server zombi process is checked.
ps -aux | awk '{print $8 " " $2 }' | grep -w Z


Step14 : Mongodb should be monitored on grafana . 

Grafana and prometheus were installed on monitoring server.
Mongodb_exporter was installed on mongodb server.
-Active connection 
-Available connection
-Memory
-Replica set lag
-Uptime
-Queued operations as read/write 
-Transaction as collections
-Command operations as "insert,update,delete"
-Page fault
-Last error
-Network io 
-Lock
-Error codes
-Vb...


Step15 : Mongodb connection in terminal.
mongo --host 192.168.12.67 --port 27017 --authenticationDatabase=admin
use admin 
db.auth("system","password****")



Step16 : Mongodb uptime is checked.
db.serverStatus().uptime


Step17 : Mongodb lock is checked in real time.
db.serverStatus().locks


Step18 : Mongodb connections are checked in real time.
db.serverStatus().connections


Step19 : Mongodb slow session is killed more than 100ms.
db.CurrentOp() 
db.killOp(3461)

Step20 : Mongodb stop operation if it is needed.
/mongopath/bin/mongo 
use admin 
db.shutdownServer()
or
mongod --shutdown

Step21 : Mongodb start operation if it is needed.
/$MONGO_HOME/bin/mongod --fork --config /mongopath/bin/mongo.conf --logpath /tmp/mongodb.log
or
cd /home/mongo/mongodb/mongodb/bin

export PATH=$PATH:/home/mongo/mongodb/mongodb/bin
./mongod --dbpath /home/mongo/mongodb/mongodb/data &
ps -eaf | grep mongo



Step22: Session unlock

cd /path/bin 
./mongo 
rs0:PRIMARY>show dbs 
rs0:PRIMARY>use cache;
rs0:PRIMARY>db.c.remove({"_id":""VVM_Test_Case:498});
WriteResult( {"nRemoved" : 1})
rs0:PRIMARY>


Step23 : Mongodb Backup 
mongodump --out /backup_path/`date +"%m-%d-%y"` --username system --password <password> --host <ip>:<port>


Step24 : Mongodb backup status
Mongo backup log can be analyzed.


Step25 : Mongodb version status 
db.serverStatus().version


Step26 : Replica set sync status 
db.printSlaveReplicationInfo()


Step27 : Fragmantation status 
db.chunks.find()


Step28 : Get summary for sharding 
db.printShardingStatus


Step29 : Lag status for replication 
rs.status()
rs.printReplicationInfo()


Step30 : Sysstat
db.stats()

 

 

 

Step31 : Sharding status
sh.status()



Step32 : Flush sql id
db.runCommand(
{
planClearCache: "orders"
}
)




Step33 : Query explain plan 
db.tablename.find(query).explain



Step34 : Drop create all indexes on collection
db.collection.reIndex()
db.accounts.reIndex()


Step35 : Compact Collection
db.runCommand({compact: collection-name})



Regards , 

About Fatih Gençali

- I have supported as Oracle and Nosql & Bigdata Dba for more than 9 years. - I worked in 24x7 production and test environment. - I have 12C OCP certificate. - I have europass diploma supplement. - Saving operations - I have supported for nosql databases (mongo,cassandra,couchbase) - I have supported for ambari&mapr hadoop distributions - I have couchbase certificate. - I have supported databases that are telecommunication , banking, insurance, financial, retail and manufacturing, marketing, e-invoicing . - Providing aligment between prod , prp , stb , dev - Providing management and performance tuning for app and database machines (linux) - Performance tuning and sql tuning - Consolidations, Migration (expdp,xtts,switchover vb...) , installation, patch , upgrade , dataguard , shell script writing , backup restore , exadata management , performans management , security management ,goldengate operations - Resolving performance and security problems for databases and linux machines - I managed oracle 10g/11g/12c databases (dev/test/prp/snap/prod/stby) on Linux/HP/AIX/Solaris O.S - Pl/sql operations , supported shell script, (for aligments and others) - Providing highly available it (software-hardware) systems, especially database systems. - Managing and monitoring availabilities and operations of all systems . - Goldengate operations (oracle to oracle , oracle to bigdata (hdfs , kafka)) - Exadata operations (cell management,upgrade,switchover) - My work processes is according to itil. - Preparing automation for everything to reduce human resource requirement and routine [email protected] https://www.linkedin.com/in/fatih-gençali-22131131/

Leave a Reply

Your email address will not be published. Required fields are marked *