RSA Security Analytics - SFTP agent sending duplicate logs

[#1]

Article Number

000032871

Applies To

RSA Product Set: Security Analytics
RSA Product/Service Type: SA Core Appliance
RSA Version/Condition: 10.5.1.0
Platform: CentOS
O/S Version: 6
 

Issue

Running saftpagent -v will show log messages as below:

Mon Mar 28 09:20:15 @xxx.xxx.xxx.xxx <7> %NIC-7-251021: SFtp Agent, SFtp Agent, -, -, -, -, Detail: 32380: Host yyy.yyy.yyy.yyy, yyy.yyy.yyy.yyy:FromFileRead: size=1846231; volume=98b4afc3; index_high=01470000; index_low=0001780b; file=emrs_hss_output_file3.log

Mon Mar 28 09:20:15 @xxx.xxx.xxx.xxx <7> %NIC-7-251021: SFtp Agent, SFtp Agent, -, -, -, -, Detail: 32380: Host yyy.yyy.yyy.yyy, ignore_file_index flag is not set, comparing file indexes and volume serial

Mon Mar 28 09:20:15 @xxx.xxx.xxx.xxx <7> %NIC-7-251021: SFtp Agent, SFtp Agent, -, -, -, -, Detail: 32380: Host yyy.yyy.yyy.yyy, yyy.yyy.yyy.yyy:File information mismatch--treating as a new file

Mon Mar 28 09:20:15 @xxx.xxx.xxx.xxx <7> %NIC-7-251021: SFtp Agent, SFtp Agent, -, -, -, -, Detail: 32380: Host yyy.yyy.yyy.yyy, yyy.yyy.yyy.yyy:New file found: emrs_hss_output_file3.log

Mon Mar 28 09:20:15 @xxx.xxx.xxx.xxx <7> %NIC-7-251021: SFtp Agent, SFtp Agent, -, -, -, -, Detail: 32380: Host yyy.yyy.yyy.yyy, yyy.yyy.yyy.yyy:Data length is positive: length=1846231

Mon Mar 28 09:20:15 @xxx.xxx.xxx.xxx <7> %NIC-7-251021: SFtp Agent, SFtp Agent, -, -, -, -, Detail: 32380: Host yyy.yyy.yyy.yyy, yyy.yyy.yyy.yyy:Opened temporary file: 1459171215_yyy.yyy.yyy.yyy_emrs_hss_output_file3.log.gz
 

Cause

The POS tracking file of the logfile may have been corrupted, causing the sftp agent to loose track of the logs inside it. 

Workaround

To reset the tracking of the sftp agent:

1. The agent keeps its tracking information in the sftpagent installation directory under a folder called POS. This will have inside a text file with the name of the log file(s) that was sent with the tracking position and file index.
2. Please copy the file out, erase the original in the directory (POS), and restart the sftp agent. It will recreate the POS file.  This would send the whole file again, but it would correct the position marker for subsequent log appends. Thus initially, we would get duplicates, but it should catch up eventually.