Monitoring proxied web traffic with SA

[RSAAdmin]

Hi, wondered if anyone had any views on this issue?

We have deployed SA South of our web proxies, therefore capturing all web traffic between the clients and the proxies. This presents some advantages, but also some problems.

The biggest problem is TCP session re-use. If an employee browses to an external website, SA captures and records that TCP session. If the user then goes on to browse to other sites, the TCP session to the proxy is re-used, as their computer considers all external websites to be on a single IP (the proxy).... the same TCP session is used for all browsing within a single browser instance, unless the idle timer expires.

Why is this a problem? A single SA session could contain a large number of alias.host entries, representing all websites visited and content downloaded within that TCP session. If one of those domains matches a risky domain defined within in a Live feed, it is not always obvious which domain is responsible. Similarly alerts will display the first alias.host value observed in the session, rather than the one that triggered the alert. The same goes for other metatypes, where SA reports something suspicious (e.g. obfuscated iframe), but it is not obvious which website/content triggered the event.

We could move our SA monitoring to Northside of the web proxies. This would then ensure that a new session is created for each website a user visits. Downsides of this:

Primarily, we would not be able to detemine the employees IP address (all sessions would originate from the web proxy)

We would not record any activity that bounces off the proxy to an internal website

We would not record any activity that is blocked by the web proxy (although this would be recorded in proxy logs)

Can anyone comment on this, has anyone else had a similar problem? Are there any solutions, other than moving the monitoring to North of the proxies? I've considered whether the TCP idle time could be reduced in order to reduce the likelihood of session re-use.

TIA

[RSAAdmin]

Yep, proxies can be tricky.  I've often suggested that both sides of a proxy get tapped, provided bandwidth is not too excessive, and sometimes custom parsers are required to extract the x:forwardedfor IP information from the sessions to properly populate the source IPs.

Tapping both sides will allow you to:

Identify when the proxy allows traffic to a known malicious host

Identify when the proxy disallowed traffic to a known malicious host

You will likely have to dedup some information with the implementation of some filters and know which side of the traffic you are looking at to properly make your rules for alerts and reports.

Not just proxies, but also NextGen firewalls that have proxy capability also sometimes poses these problems.

Or, if you have a log decoder, I would tap network on the outside of the proxy and capture the proxy logs to represent traffic on the inside.

[Anonymous]

Fielder wrote:

Or, if you have a log decoder, I would tap network on the outside of the proxy and capture the proxy logs to represent traffic on the inside.

This is exactly what we do currently and it requires slightly more investigation work but with SA it does make it pretty easy to drill into the specific alias.host and see both the packet and log together.

To comment on @dazzer this does make the packets look funny at the same time.  But now instead of clients going to the proxy it is the proxy going to a ton of websites.  But I think we have fine visibility into everything when we see both the proxy logs and the packets together.

I am also not positive because I have not started my research on aggregators yet, but they might be able to do the parsing work on the backend for you, then send it to SA and you will see the correct streams, client to site.  No idea if that is true but I imagine the aggregator could look through the packets and accomplish this.

[RSAAdmin]

Hi

Thanks for coming back on this. Some good suggestions, we don't currently have the log decoder capability, unfortunately (a shame, this sounds like it would be the best solution). Similarly we refrain from sending the 'x:forwarded- for' header in order to conceal our internal address structure.

I also considered tapping both sides, then retaining external pcaps and truncating internal. I think some trials are in order to see which is most beneficial (inside/outside) or both if SA can handle it.

[RSAAdmin]

Hi,

Thanks for your advice. I'm starting to think that we should definitely consider adopting a solution similar to yourself.

[Anonymous]

I have the same problem.

I am tapped on both sides of the proxy & that helps with attribution.

The real problem is with the session re-use. You end up with what I like to call "Super sessions" which Netwitness has trouble parsing fully unless you make some changes to your "Max Parser bytes" and "Max session size" settings.

Since this behavior coincides roughly with a proxy upgrade & I never saw the "Super session"  issue before,  I believe a proxy change should fix the issue, we just need to find it.

Ex. You know a user went to google.com yet the is no mention of google.com in alias.hosts  when doing an investigation on that host in the same time period.  However if you export the same sessions to pcap  from Investigator and in Wireshark,  it clearly shows the traffic to google.com.

NW will only parse so far into a session the visit to google.com never gets parsed and added to alias.hosts.

Usually a single "super"  session will have all sorts of unrelated web traffic mixed into the session and when you are viewing one of these session in Investigator, you can scroll and scroll and scroll through pages of meta before the getting to the next session.

You will also start getting "parse: max meta reached" alerts in investigator.

[RSAAdmin]

Hi,

We have the same issues. Having a decoder outside the proxy (ISA TMG):

1) Not able to determine client address. As people suggest, will try to add another decoder on the inside of proxy or will add log decoder to get the proxy logs, although I am not sure of how to map sessions to that log to get client ip (see part 2)

2) Getting super sessions which are mega large: lots of different traffic in one session, not all of it is getting parsed, increasing the assembler/parser values slows processing, and investigator isn't able to extract files or for these sessions or show the user friendly web/mail view. Did you manage to extract files from the mega sessions?

As I get it the non-transparent proxy architecture will get us these big sessions anyways or has someone conquered them?