I am blown away with the amount of traffic being consumed when using Google’s mail in standard view!
Background
My flatmate and I recently sold our souls to the devil, in this case a local cellphone service provider, and acquired a 3G internet contract. Unfortunately, in South-Africa the amount of data a person can transfer is still being capped. Severely. Call it bureaucracy, monopoly, call it bloody greed if you ask me; it’s sickening. Either way we’re stuck in this situation, and our contract allows us a measly 2Gb per month which we have to share that between the two of us.
The awakening
In just two days I noticed on the ISP’s site that I already whacked 200Mb of what I have, 25% damnit! I very quickly realized that I need to keep a watchful eye on my usage, and I thought it would be interesting to see where goes what. The best thing would be to use software that I could setup to monitor all the packets sneaking around.
I went all over the net and installed so many ‘must-have’ applications it could drive a Sahara forrest monkey crazy.
And then…
Enter BWMeter
I found this awesome traffic/bandwidth meter called BWMeter. All the others I have tried are “so called” bandwidth monitors but they’re too simple, way too simple. Trust me, forget about the other crap out there and just get this.
BWMeter allows you to define ‘Filters’ which could be any kind of connection you want to monitor. Source, Destination, Protocol, Port, Schedule and General option which has settings like ‘Include upload’ / ‘Include download’ / ‘Include IP protocol overhead’. You can also set it to block all traffic or limit the speed on the filter.
Once you’ve setup a filter you can set graph windows, view hourly/weekly/montly/yearly stats on it. It allows these to be exported to a .csv file, ie your favorite spreadsheet. You can also monitor and follow the exact requests that flows through each filter, which means that you can see IP:Port requests and the size in bytes.
Alerts allows to define an action to be followed whenever certain criteria is met. For example, when the Download+Upload reaches 500Mb during the current month, play a sound, run a program, display a message, send an email and/or throttle or block all traffic that applies to the filter. Versatile.
Google Talk + Google Mail
Right, the moment of truth has arrived. I know Google mail uses mail.google.com. For GTalk I just do a quick ‘netstat -a’ in the command window. This reveals a connection to *.google.com:5222. I’ve setup two separate filters, one that monitors mail.google.com, and the other for *google.com:5222 (google talk). Google talk runs through a couple of KB, but it all depends on how much you talk. About an hour of reasonable talk cost me 130Kb. I can live with that.
Google Mail!! Eish, this is scarey. Logging into the standard view mode consumed 350Kb just to get to the point where everything is loaded. Another 100Kb to be fully logged out. And I didn’t even open any mail… rough stuff. Changing to basic HTML (use http://gmail.google.com/gmail/h/) showed a usage of around 50Kb consumed when fully logged in. That’s a huge difference.
Conclusion
This clearly indicates that our wonderful Web2.0 technologies do consume a lot more traffic than what we may realize. I did expect it to be more, but the difference is huge, around 5x more. Taking into consideration the current state of this country’s internet options and the attitude that our government has taken we clearly need to be smart about the way we use what we have. Use Lynx!
*Note
For those that think of getting technical on me, yes, I did clear the cache and restarted the browser for each test. Now smile and go pick your nose if you need to pick on someone…
I’m under a 1Gb limit with a family and an IT career.
Gotta switch ISPs real soon! We’ve just got reasonably cheap ADSL2+ in metro regions of Australia.
I spent some recent sick leave at home doing packet sniffing with WireShark (http://www.wireshark.org/) to track down the culprit that munched up my last months 1Gb in a few days. You know the answer … GMail. You can watch it run up the meter with nothing visible happening on the UI. Sorry but this has lots of performance tuning to be done. I suspect that it might be exacerbated by my insisting that I use GMail with SSL which may impact caching.
I’ve also just started an evaluation project that will use Google Web Toolkit (http://code.google.com/webtoolkit/) that GMail and other Google goodies are supposed to be built on. I guess I’d better include some bandwidth monitoring in my assessment
.
It’s good to know that there are other people that can confirm my findings.
To what extent did your sniffing go? (in terms of what data is so consistently being sent in the background?)
I’ll keep an eye on your blog for a post on the toolkit!
Interesting post, especially to see how bw-hungry our spiffy “Web 2.0″ apps are in comparison to the old school HTML-only apps. This all stems from using loose XML as a transport protocol, which tends to have a much larger overhead than other methods but is more extensible.
Considering that GMail is almost three years old now and doesn’t have an API outside of Google, it’s very much due for a rewrite. With Hotmail revamping their offerings, I’m sure we’ll see something soon. It’ll be interesting again to see what method of transport the “new” GMail will employ to address these issues: add more bloat or lean and mean?
Ah, yes, what you’re saying does make a lot of sense.
I would imagine that the “rewrite”, as you put it, would be imminent since Google so much likes taking on MS (and I kinda like the little battle they got going *hie hie*)
guys guys guys…. you call this a test?! Of course Gmail uses a lot of bandwidth.. but only at the begining, when it downloads the javascript that make the Gmail cool
after the login phase the bandwidth used should be smaller. I never tested anything, I just know how it works…
It might be a hassle but access gmail from your phone’s WAP browswer.
The problem with V’s comment is that my laptop isn’t constantly on, so gmail has to log in quite a lot over my 3G mobile connection. What’s unclear is why if the gmail engine code hasn’t changed it’s necessary to download so much every login. (I guess because the http caching model doesn’t support whatever is needed to cache javascript in a semantically correct manner.)
you said:
“about an hour of reasonable talk cost me 130Kb”
That’s impossible, presumably the voice data is going directly to and from your peer, not through the google server.
I would expect a hour of talk to use at least 30Mb depending on the quality and how much you were talking.
sorry for the confusion Sam, I think what I meant here is talk as in typing/chatting…
Hi Byron, back when I posted this I don’t even know if Gmail had the phone stuff, either way, things have come a long way and I’ve made peace with the bandwidth usage. I’ve done some development in GWT (Google Web Toolkit) and as V commented, it is only the first time when the js loads that it uses a lot. I think now, with Google Gears and/or Chrome, which integrates better we might see more interesting things
dfgfgfgfg
Greetings,
Thanks for the post!!!
I’m very, very interested in knowing if this still happens with the current version of Gmail. Above all, I would LOVE to see published, if you can, more about if/how bandwidth consumption varies over time.
More exactly: could you provide more info about what is exactly that takes so much bandwidth? The initial javascript code or the fact that gmail (as any AJAX application) continuously downloads and redownloads message list, headers, message bodies, etc, so it can “open” a single message in no time because it had been already downloaded “just in case” the user wanted to see it?
This is a very important distinction. If bandwidth is consumed by downloading javascript, it can be optimized and happens once per session or so. But if Gmail continuously downloads other stuff “just in case” is a more serious issue.
Thanks,
Marco
Sam Watkins,
Exactly, Mr Sam, You are on right path.
I think that the Gtalk takes 24-32MB per hour for voice .It may depends upon the voice usage, (May:- Note that voice transfer use RTP, It cannot depends the content. Its RTP is under comes UDProtocol).
Guys note that b=> Denotes bit. (Eg: 0 is a bit 1 is a bit)
and B=> Denotes Bytes. (Eg: 1001 1101 is a Byte)
Any one know How much bandwidth takes by the video chat in Gtalk.
I think its about 48-64MB per hour (Its my guessing, Video takes more band width then audio.
Is g mail provides any information about the bandwidth taken by the Gtalk???.