How to show the count of all NetWitness mongodb documents in all collections?

[#1]

Article Number

000039762

Applies To

RSA Product Set: RSA NetWitness Platform
RSA Product/Service Type: Mongodb
RSA Version/Condition: 11.x
Platform: CentOS
O/S Version: 7

Issue

Troubleshooting the mongodb how to show a count of documents in each collection for all databases.

Want to see which collection has the most document count.

Resolution

1. Login to NetWitness appliance that is running the mongod service.
2. Confirm that the mongod service is running.
   systemctl status mongod
   
   For example:

   
   [root@NW ~]# systemctl status mongod
   ● mongod.service - MongoDB Database Server
      Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled)
     Drop-In: /etc/systemd/system/mongod.service.d
              └─mongod-start-managed.conf
      Active: active (running) since Mon 2021-06-07 03:32:30 UTC; 1 weeks 6 days ago
        Docs: https://docs.mongodb.org/manual
     Process: 1876 ExecStart=/usr/bin/numactl --interleave=all /usr/bin/mongod $OPTIONS run (code=exited, status=0/SUCCESS)
     Process: 1873 ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodb (code=exited, status=0/SUCCESS)
     Process: 1868 ExecStartPre=/usr/bin/chown mongod:mongod /var/run/mongodb (code=exited, status=0/SUCCESS)
     Process: 1861 ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb (code=exited, status=0/SUCCESS)
    Main PID: 3192 (mongod)
      CGroup: /system.slice/mongod.service
              └─3192 /usr/bin/mongod -f /etc/mongod.conf run
   
   Jun 07 03:32:19 NW5ESAPRIM systemd[1]: Starting MongoDB Database Server...
   Jun 07 03:32:27 NW5ESAPRIM numactl[1876]: about to fork child process, waiting until server is ready for connections.
   Jun 07 03:32:27 NW5ESAPRIM numactl[1876]: forked process: 3192
   Jun 07 03:32:30 NW5ESAPRIM numactl[1876]: child process started successfully, parent exiting
   Jun 07 03:32:30 NW5ESAPRIM systemd[1]: Started MongoDB Database Server.

3. Run the following mongo query to show a count of the documents for each collection in all database.
   Can copy all of the below lines down to and including the final EOF line and paste into a bash command line, then press enter.

mongo admin -u deploy_admin -p netwitness --quiet <<EOF

show dbs

db = db.getSiblingDB('admin');
var dbs = db.adminCommand('listDatabases');

dbs.databases.forEach(function(database){
 print("Database: " + database.name);
 print("-----");
 if (database.name == "config") { print("Skipped"); }
 else {
  db = db.getSiblingDB(database.name);
  db.getCollectionNames().forEach(function(collection) {
   print("Collection '" + collection + "' documents: " + db[collection].count());
  });
 };
 print("");
});

exit
EOF

Substitute netwitness with the correct deploy_admin password.

For example:

[root@NW ~]# mongo admin -u deploy_admin -p netwitness --quiet <<EOF
>
> show dbs
>
> db = db.getSiblingDB('admin');
> var dbs = db.adminCommand('listDatabases');
>
> dbs.databases.forEach(function(database){
>  print("Database: " + database.name);
>  print("-----");
>  if (database.name == "config") { print("Skipped"); }
>  else {
>   db = db.getSiblingDB(database.name);
>   db.getCollectionNames().forEach(function(collection) {
>    print("Collection '" + collection + "' documents: " + db[collection].count());
>   });
>  };
>  print("");
> });
>
> exit
> EOF
admin            0.000GB
config           0.000GB
endpoint-server  0.003GB
local            0.000GB
admin
Database: admin
-----
Collection 'system.users' documents: 23
Collection 'system.version' documents: 2

Database: config
-----
Skipped

Database: endpoint-server
-----
Collection 'bookmark' documents: 11
Collection 'certificate' documents: 95
Collection 'command' documents: 134
Collection 'file' documents: 1451
Collection 'filecontexthistory' documents: 1367
Collection 'grouppolicy' documents: 1
Collection 'machinedetail' documents: 2
Collection 'machinefile' documents: 1
Collection 'machinefilehistory' documents: 0
Collection 'machinefilescore' documents: 0
Collection 'machinefilestage' documents: 0
Collection 'machinehistory' documents: 1
Collection 'machineidentity' documents: 2
Collection 'mftrecord' documents: 0
Collection 'relayconfig' documents: 1

Database: local
-----
Collection 'startup_log' documents: 15

The output shows the disk space size of all databases in the mongodb, the "show dbs" output.
Then for each database, there is a count of the documents in each collection.
Investigate the databases using the most disk space and the collections with the most entries.

Note: The config database is skipped as it is for internal use by the mongod service and this command has insufficient permission to access.