Discovering New Trends in Web Robot Traffic through Functional Classification

Derek Doran; Swapna S. Gokhale

doi:10.1109/NCA.2008.47

Discovering New Trends in Web Robot Traffic through Functional Classification

Derek Doran, Swapna S. Gokhale

Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

This paper proposes a novel functional classification scheme to understand and analyze web robot traffic. The scheme is rooted in the recognition that the crawling behavior of a robot on a site is primarily governed by its intended purpose or functionality. We apply the classification rules to analyze web server access logs from the University of Connecticut School of Engineering domain. The analysis results indicate how the classification scheme can provide insights into the robot traffic based on their functionality.

Original language	American English
Title of host publication	2008 Seventh IEEE International Symposium on Network Computing and Applications
Publisher	IEEE
Pages	275-278
Number of pages	4
ISBN (Print)	978-0-7695-3192-2
DOIs	https://doi.org/10.1109/NCA.2008.47
State	Published - 2008
Event	7th IEEE International Symposium on Networking Computing and Applications, NCA 2008 - Cambridge, MA, United States Duration: Jul 10 2008 → Jul 12 2008

Conference

Conference	7th IEEE International Symposium on Networking Computing and Applications, NCA 2008
Country/Territory	United States
City	Cambridge, MA
Period	7/10/08 → 7/12/08

ASJC Scopus Subject Areas

Computer Science Applications
Hardware and Architecture

Keywords

Robots
Servers
HTML
Websites
Bandwidth
Crawlers
Internet

Disciplines

Computer Sciences
Engineering

Access to Document

10.1109/NCA.2008.47License: Unspecified

Cite this

Doran, D & Gokhale, SS 2008, Discovering New Trends in Web Robot Traffic through Functional Classification. in 2008 Seventh IEEE International Symposium on Network Computing and Applications., 4579672, IEEE, pp. 275-278, 7th IEEE International Symposium on Networking Computing and Applications, NCA 2008, Cambridge, MA, United States, 7/10/08. https://doi.org/10.1109/NCA.2008.47

@inproceedings{158b51eb4cbd4146aa15ce84b6e1c0ee,

title = "Discovering New Trends in Web Robot Traffic through Functional Classification",

abstract = " This paper proposes a novel functional classification scheme to understand and analyze web robot traffic. The scheme is rooted in the recognition that the crawling behavior of a robot on a site is primarily governed by its intended purpose or functionality. We apply the classification rules to analyze web server access logs from the University of Connecticut School of Engineering domain. The analysis results indicate how the classification scheme can provide insights into the robot traffic based on their functionality.",

keywords = "Robots, Servers, HTML, Websites, Bandwidth, Crawlers, Internet",

author = "Derek Doran and Gokhale, {Swapna S.}",

year = "2008",

doi = "10.1109/NCA.2008.47",

language = "American English",

isbn = "978-0-7695-3192-2",

pages = "275--278",

booktitle = "2008 Seventh IEEE International Symposium on Network Computing and Applications",

publisher = "IEEE",

note = "7th IEEE International Symposium on Networking Computing and Applications, NCA 2008 ; Conference date: 10-07-2008 Through 12-07-2008",

}

TY - GEN

T1 - Discovering New Trends in Web Robot Traffic through Functional Classification

AU - Doran, Derek

AU - Gokhale, Swapna S.

PY - 2008

Y1 - 2008

N2 - This paper proposes a novel functional classification scheme to understand and analyze web robot traffic. The scheme is rooted in the recognition that the crawling behavior of a robot on a site is primarily governed by its intended purpose or functionality. We apply the classification rules to analyze web server access logs from the University of Connecticut School of Engineering domain. The analysis results indicate how the classification scheme can provide insights into the robot traffic based on their functionality.

AB - This paper proposes a novel functional classification scheme to understand and analyze web robot traffic. The scheme is rooted in the recognition that the crawling behavior of a robot on a site is primarily governed by its intended purpose or functionality. We apply the classification rules to analyze web server access logs from the University of Connecticut School of Engineering domain. The analysis results indicate how the classification scheme can provide insights into the robot traffic based on their functionality.

KW - Robots

KW - Servers

KW - HTML

KW - Websites

KW - Bandwidth

KW - Crawlers

KW - Internet

UR - http://www.scopus.com/inward/record.url?scp=51749114015&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=51749114015&partnerID=8YFLogxK

UR - https://corescholar.libraries.wright.edu/cse/257

U2 - 10.1109/NCA.2008.47

DO - 10.1109/NCA.2008.47

M3 - Conference contribution

SN - 978-0-7695-3192-2

SP - 275

EP - 278

BT - 2008 Seventh IEEE International Symposium on Network Computing and Applications

PB - IEEE

T2 - 7th IEEE International Symposium on Networking Computing and Applications, NCA 2008

Y2 - 10 July 2008 through 12 July 2008

ER -

Discovering New Trends in Web Robot Traffic through Functional Classification

Abstract

Conference

ASJC Scopus Subject Areas

Keywords

Disciplines

Access to Document

Other files and links

Cite this