MDLog:/sysadmin

The Journal Of A Linux Sysadmin

Netcraft Web Server Survey? Or Should I Say Domain Parking Survey?

| Comments

You have most certainly heard of Netcraft and have seen their Web Server Survey results. It is probably the best known authority in this field and it is referenced in many sources with their popular Web Server Survey results. Everybody seems to be looking at Netcraft to see how are Web Servers are used, what is the trend (what percent did Apache won this month? or did it lose something this month? how is IIS doing? etc.) or just to check out their statistics released monthly. I have noticed a problem a few months ago, but being too lazy I have decided to let it go, believing that someone will stand up against this issue, or that Netcraft themselves will see the problem themselves and fix it… but seeing the same thing over and over in the last months I have decided to share my thoughts about their statistics.

On Netcraft’s June 2006 Web Server Survey we can read this month’s report, that was released a few days ago, and from their post I quote:

“Microsoft continues to gain share in the web server market, chipping away at Apache’s commanding lead. The number of hostnames on Windows servers grew by 4.5 million, giving Microsoft 29.7% market share, a gain of 4.25% for the month. Apache had a decline of 429K hostnames, and loses 3.5% to 61.25%.

Apache’s lead over Microsoft, which stood at 48.2% in March, has been narrowed to 31.5%, a shift of 16.7% in just three months.

The largest movement of sites from Apache to IIS was once again at Go Daddy, with over 1.6M hostnames moving from Apache to IIS this month. While those parked domains were a major factor in Microsoft’s gains, Windows also saw solid growth in active sites, hostnames that contain content and likely to represent developed web sites.“

I read this and say: wow… are those peoples seeing themselves what happens? Does this makes any sense to them? 1.6 million hostnames parked at Godaddy changed the stats dramatically like that… If so, then we do realize that in the 85,541,228 sites they are using for the web server survey there are much more domains parked… We have here included millions of other parked domains from other major domain parking providers like Sedo, Afternic, DomainSponsor, etc. that are much bigger than Godaddy. Why would I care about what web server the parked domains are using? (either if that is Apache or IIS or whatever).

Now I don’t want to talk about Microsoft market policy on this move, and how they have won 1,6M parked domains. Not even about Godaddy’s move, as they are free to choose their own platform they believe it is best for them. Not about the users, that have their domains parked now on IIS instead of Apache and must be probably trilled by this move. I want to talk about the relevance of including those domains in what is probably the most widely used web server survey. I would have felt the same even if the situation would have been reversed or even if this would have not involved Microsoft, and just some other obscure web server that one major parking domain provider has chosen and would change drastically the results.

Parked domains… what are they? They are basically either domains parked at some page until the owner has the time to develop something on them, or they are just domains bought to be parked forever in the attempt to bring some profit to the domain owner. I don’t have any problem with parking domains… Everybody does what they want to with their domains. But in my opinion parked domains are just not used domains and should be treated as such. But why does Netcraft includes millions of such domains in their stats? Why would I care about what those domains are using as web server? Maybe if this was a parking domain survey then it would have been interesting to see what the major parking providers are using as web server. But this is not the case.

This is just my personal opinion, but when I look at the most cited web survey on the Internet I would expect: to see only real results that would show me what peoples are really using as web servers. For this I would like to see:

  • why include millions of domains? I want to see only statistics of servers and not domains (there can be hundreds of domains hosted on the same server, so what is the relevance?)

  • I realize that getting a server only report is not so easy as there can be many IPs bound on the same physical server, but at least show us results based on the IPs and not domains… Why show me thousands (or as seen above millions) of domains that are hosted at the same place and obviously will be using the same web server.

  • If the above are too hard… then at least remove from the stats the parked domains that will make the biggest difference (it will not really show if I am using on my own server 2, 5, or 10 domains, but if I have millions then it would). This should not be so hard as there are only a few big parking services that can be easily filtered from the results and still keep the same statistics based on ‘sites’. But maybe Netcraft doesn’t want to do that, as this is showing to be profitable for many peoples… I don’t want to speculate and I don’t even care (but probably Godaddy are very happy that Microsoft cares about the Netcraft results)…

  • Maybe Netcraft already has such a report, but provides it only on it paid services. Still in that case I would not care about that. What is important is what most peoples are looking and even more what thousands of other sources are referring in their articles.

Now what can we do about this? Well I could change the web server response in my own sites to return instead of Apache, “Netcraft stop including parked domains in your stats”… but that would not make any change of course… How can it, when they are showing millions of parked domains? Any ‘real’ change (not as I joked above, but a real trend) would not be observed by that. So the only thing I see, is to hope that Netcraft will realize the problem and do something that will keep their reports the best on the Internet. If not, probably we will soon see other companies that will start doing similar reports, hopefully more useful that Netcraft’s report is to me right now.

Final note: this is just my personal opinion and might be different from other peoples that would like to see all the domains in the world (parked or not) included in such statistics. Regardless of that, if Netcraft will not do something about this, I will have to search for another company that provides such results but in a better way (if that exists).

Comments