Log Path in SAW Nodes

[AtulChavan]

Hi,

I want to see last log present in SAW Node. Can anyone please share command which I can run on SAW nodes or Path of logs file.

Need your assistance on this.

[AkramHamed]

Hello Atul,

For the SAW logs, there exists plenty depending on which mapr service, as well as the rsa logs itself.

However, most common logs are:

Hadoop Logs

     /opt/mapr/hadoop/hadoop-{Version}/logs

Hbase Logs

     /opt/mapr/hbase/hbase-{Version}/logs

SAW Log file

     /opt/rsa/saw/logs/saw.log

MapR Logs (Warden, mfs, etc)

     /opt/mapr/logs

ZooKeeper Logs

     /opt/mapr/zookeeper/zookeeper-{Version}/logs

And of course the /var/log/messages

For a complete list of logs:

•opt/mapr/hadoop/hadoop-0.20.2/logs/hadoop—jobtracker-hostname.log

•/opt/mapr/hadoop/hadoop-0.20.2/logs/hadoop--tasktracker-hostname.log

•/opt/mapr/hbase/hbase-0.92.1/logs/hbase--master-hostname.log

•/opt/mapr/hbase/hbase-0.92.1/logs/hbase--regionserver-hostname.log

•/opt/mapr/hbase/hbase-0.92.2/logs/hbase--master-hostname.log

•/opt/mapr/hbase/hbase-0.92.2/logs/hbase--regionserver-hostname.log

•/opt/mapr/logs/adminuiapp.log

•/opt/mapr/logs/cldbdiskbalancer.log

•/opt/mapr/logs/cldb.log

•/opt/mapr/logs/cldbrolebalancer.log

•/opt/mapr/logs/createJTVolume.log

•/opt/mapr/logs/createsystemvolumes.log

•/opt/mapr/logs/createTTVolume.log

•/opt/mapr/logs/disksetup.uniqueid.log

•/opt/mapr/logs/hoststats.log

•/opt/mapr/logs/maprcli-mapr-uniqueid.log

•/opt/mapr/logs/maprcli-root-0.log

•/opt/mapr/logs/mfsinit.log

•/opt/mapr/logs/mfs.log

•/opt/mapr/logs/mount_local_fs.log

•/opt/mapr/logs/mrdisk.log

•/opt/mapr/logs/nfsmon.log

•/opt/mapr/logs/nfsserver.log

•/opt/mapr/logs/pam.log

•/opt/mapr/logs/timeskew.log

•/opt/mapr/logs/wardeninit.log

•/opt/mapr/logs/warden.log

•/opt/mapr/zookeeper/zookeeper-3.3.2/logs/zookeepercleanup.log

•/opt/mapr/zookeeper/zookeeper-3.3.2/logs/zookeeper.log

•/opt/rsa/saw/bin/wrapper.log

•/opt/rsa/saw/derby.log

•/opt/rsa/saw/logs/saw.log

Also, you can refer to the 3 node scenario, logs for each service image below.

Akram.

[AtulChavan]

Hi Akram,

Can you please highlight the path in which specifically event source's logs are present?  Also, Is there any PDF explaining all these things?

[AkramHamed]

Hello,

There is no way you can know where the latest log file, or any log file is exactly saved on the hadoop file system.

Files in SAW/HDFS are saved in files called Avro files, these files are flat files to be easily compressed, with all data mixed together. you can not known that logs for a certain event source are stored in this Avro file.

The avro files for raw logs are stored in the path /rsasoc/v1/logs/data  in the HDFS, this can be listed using the "hadoop fs -ls" command. or from the NFS mount point on WHC.

To wrap up, you can not determine which avro files contain which data, also one avro file contains different and variant types of data. you can only interact with Avro files using Hive, Pig or Reporting Engine to generate reports.

I hope this has helped you better understand.

Thanks,

Akram