Abstract
There is strong evidence to suggest that a significant proportion of traffic on Web servers, across many domains, can be attributed to Web robots. With the advent of the Social Web, widespread use of semantic Web technologies, and development of service-oriented Web applications, it is expected that this proportion will only rise over time. One of the most important distinctions between robots and humans is the pattern with which they request resources from a Web server. In this paper, we examine the arrival process of Web robot requests across Web servers from three diverse domains. We find that, regardless of the domain, Web robot traffic exhibits long range dependence (LRD) similar to human traffic. We discuss why, at least in some cases, LRD in robot traffic may not be generated by heavy-tailed response sizes as in the case of human traffic.
| Original language | American English |
|---|---|
| State | Published - Aug 1 2012 |
| Event | International Proceedings of Computer Science and Information Technology - Duration: Aug 1 2012 → … |
Conference
| Conference | International Proceedings of Computer Science and Information Technology |
|---|---|
| Period | 8/1/12 → … |
Disciplines
- Computer Sciences
- Engineering