AppSec Blog

Examine HTTP compressed gzip content

For incident handling, forensics or troubleshooting purposes, packet sniffing is often used to understand the information exchange between two hosts. HTTP traffic packets are often sniffed so that the full header and body can be revealed easily, especially on the server side. On the client side, most commonly used technique is to use a proxy tool like Webscarab to intercept the request. While making new exercise for my DEV422, I noticed some interesting behavior of how different programs handles the gzip compressed HTTP content.

Normally, if the server sends gzipped content to the client, it sends a special HTTP header indicating to the browser the content should be decompressed before rendering on screen, this is the header normally sent

Content-Encoding: gzip

When there is a sniffer or interception proxy in the middle, the condition is complicated, as these man-in-the-middle need to be able to decode to read the actual content.

Let examine how Webscarab handles gzipped content, here is an interception on response coming from server.

Webscarab gzip encodingWebscarab gzip encoding

Webscarab shows a "X-Content-Encoding: gzip" indicating the gzip process had already been performed. The browser does not have to take any action as the content is already decompressed by Webscarab.

Wireshark, on the other hand is a simple sniffer, it doesn't alter anything the user is able to view. Let's see what Wireshark can see while sniffing packets on the wire.

09-05-21-wireshark-full

Unlike Webscarab, Wireshark does not alter the headers and only shows what is actually transferred. At the bottom of the above screen cap, Wireshark is able to decode the gzipped data and show the HTTP data.

Everything is good so far, Webscarab and Wireshark essentially handled gzip data transparently by default.

Now, what if I follow TCP stream within Wireshark? This is a very common function used by a lot of users to view the full conversation between two hosts.

Wireshark follow TCP streamWireshark follow TCP stream

Wireshark Follow TCP Stream function does not decode gzip compression. The content is not decoded while displayed on screen.

While handling packet sniffer or using a man-in-the-middle manipulating HTTP responses, make sure you are aware that some compression encoding could be at play if things are not displaying properly on screen.

2 Comments

Posted July 8, 2009 at 7:05 PM | Permalink | Reply

Dr. Hacker

Thank you very much dear friend, although I think the viewing of gzip'ed content is handled properly by mergecap a plugin for wireshark.

Posted November 16, 2010 at 5:32 PM | Permalink | Reply

James Taylor

mergecap seems to understand gzip compressed capture files but it does not handle the decompression of gzip encoded HTTP streams. I think an option to un-gzip HTTP streams in the "Follow TCP stream" window would be a very helpful addition to Wireshark. Ideally, it would be possible to set a display filter to find packets containing specific things in the gzipped content too.

Post a Comment






Captcha


* Indicates a required field.