“Google Analytics is the enterprise-class web analytics solution that gives you rich insights into your website traffic and marketing effectiveness. Powerful, flexible and easy-to-use features now let you see and analyze your traffic data in an entirely new way. With Google Analytics, you’re more prepared to write better-targeted ads, strengthen your marketing initiatives and create higher converting websites.”

Everyone uses Google Analytics (GA), right? It’s a great product and even better it’s free. It is a win/win situation where there is no point in anyone running their own analytics unless they have something custom and specific that is not covered by GA.

Still as other google product its documentation is, let’s say not the best. A while ago I started working on a project and there was no web analytics in place. I asked them why is that? They said they have too big traffic to be accepted in GA. Hmm… I looked into it and I must admit I could not find much information that we were interested. Finally looked over the terms of services and there I found:
“2. FEES AND SERVICES . Subject to Section 15 herein, the Service is provided without charge to You for up to 5 million pageviews per month per account, and if You have an active Adwords campaign in good standing, the Service is provided without charge to You without a pageview limitation.”

So there is a limit, a tiny one I would say of 5mil pageviews per month (per account, not even per site). Our site was making about 45Mil pageviews at that time. Per day! So what if we wanted to use GA? We searched everywhere but could not find any commercial offering of GA or any other information. We asked the @googleanalytics on twitter but we were completely ignored.

What to do? Well, we just gave it a try and added the site and started tracking it in GA as any other site. Surprisingly, it worked just fine for a few months. Yesterday though, we received an email from the google analytics team (or should I say GA “robot”?) telling us that they have detected we have a high traffic, much higher than the allowed limit of 5mil pageviews per month, and from now on we are no longer going to have live reports but only daily updated reports. This is a limitation we can live with, but it would have been great if they would have given us some option to pay for some extra services. My client would have been happy to pay in the first place, but I assume this is something google doesn’t care at all and they just want to offer it as a free services. There is a great opportunity for such a product that could handle high traffic analytics and can do real-time and other goodies; we would be definitely interested. In the meantime if you have a site that makes more than 5mil pageviews per month (not so uncommon) you can definitely use GA; in the worst case they will restrict your updates to keep up with your traffic. For our site we tracked 1,608,074,379 Pageviews last month in GA  and it works just fine.

I’ve been using lighttpd for various projects with great results. A few days ago I had to optimize a server that was running lighttpd but has degraded its performance during the past weeks. I was able to see quite easily that the problem was IO bound as the number of files was growing very fast, while the folders were not arranged very well. Lighttpd was starting to slowdown while blocked on disk IO requests.  We needed a quick solution to buy us some time while we improve the backend files layout.

Having used nginx for some time now on some bigger sites that this one, my first thought for a quick solution was to increase the number of lighttpd workers as I have done with nginx. Lighttpd supports this, but looking at their documentation page we can see:
“DO NOT USE THIS OPTION IF YOU DON’T UNDERSTAND WHAT IT DOES
DO NOT REPORT ERRORS OR BUGS IF YOU DID NOT TEST WITHOUT THIS OPTION SET
THIS OPTION MOST LIKELY WILL NOT BOOST YOUR PERFORMANCE, ITS MOST LIKELY YOUR BACKEND”
wow… talk about a warning

Read the rest of this entry »

This is my first blog post of 2009 and I just wanted to do a quick review of my past year blogging and also compare it with 2007. My general feeling before doing this, was that even though I have tried to dedicate more time to blogging, work and personal things haven’t allowed me to do this as much as I wanted. But let’s see the raw numbers and then draw a conclusion.

I posted exactly 91 articles during 2008, meaning an average of 8 per month. This sounds very low from the rate I am trying to achieve, but still it is a good improvement compared with 2007 when I had posted only 42 articles. The enthusiasm (and the extra time available I had back then) from my first year of blogging – 2006 – with 139 posts is still far away. I am definitely aiming for 2009 to be my best year in blogging (and not only ) and this should be doable with a little more dedication.

Read the rest of this entry »