If you are one of those Website owners who slaps his own back for achieving traffic of 10,000 readers + daily, read on. Be prepared for a shock.
Online security firm Incapsula has just released a report in which it has claimed that in 2014, bots accounted for at least 56% of all Website traffic. The only silver lining – the little pieces of software (bots) visited slightly less Websites this year as compared to the 60% last year.
This is, of course, traffic routed through Sites on the Incapsula network. The data presented in the report was based on a sample of over 15 billion human & bot visits occurring over a 90-day period. It was collected from 20,000 Websites on the Incapsula network having a minimum daily traffic count of at least 10 human visitors. Geographically, the observed traffic includes all of the world’s 249 countries (per country codes provided by an ISO 31n the I6-1 standard).
The Wikipedia says: An Internet bot, also known as web robot, WWW robot or simply bot, is a software application that runs automated tasks over the Internet. Typically, bots perform tasks that are both simple and structurally repetitive, at a much higher rate than would be possible for a human alone.
Announcing the finding of the report on its official blog, the security firm said it had noticed that the bulk of the decline reflected a drop in so-called “good bot activity”. Specifically speaking, these bots were associated with RSS services. Initial assumption of the Incapsula analysts was that the shift was related to the Google Reader service shutdown.
Upon further inspection they realized though that Feedfetcher bot—associated with the Google Reader service—was still as active as ever, while the decline in RSS bot activity was across the board. This broad downward trend is RSS bot activity the main reason for the approximately 10% drop in good bot activity.
A few months ago, Incapsula had conducted an analysis of Googlebot activity. It showed that Googlebots did not play favorites; they crawled both small & large Websites in a pre-determined frequency, regardless of anyWwebsite’s actual popularity.
The team of analysts then wanted to test if all other inhuman agents were also as “hype-immune.” To do that, it split the sample group into 4 subcategories, based on the number of daily human visitors. What it gleaned from this exercise was that smaller Websites tended to get a higher percentage of bot visitors, where such access accounts for approximately 60 to 80 % of all traffic.
Incapsula said in its post it was difficult to say what the absolute average number of daily visitors is, but it was safe to assume that most Websites got less than 10,000 visits a day. While bot traffic to larger Websites was about 50 percent, less, medium Websites—which represent the bulk of the Internet—were actually serving 2 to 4 bot sessions for every human visitor.
Bad Bots Threaten Big & Small Sites Alike
But the study also made it clear that malicious bots posde a categorical threat to all Websites, regardless of size. The average percentage of bad bots consistently hovered around the 30% mark, regardless of Website size or popularity. In absolute terms, Incapsula said malicious bot traffic grows in an almost exact proportion to a site’s human traffic.
Image Credit: Incapsula
Advertising Message
