Home | Browse | Search | Credits | About
Register | User Area | DL-Harvest | Help
DLIST

CI Spider: a tool for competitive intelligence on the Web

Chen, Hsinchun and Chau, Michael and Zeng, Daniel (2002) CI Spider: a tool for competitive intelligence on the Web. Decision Support Systems 34(1):pp. 1-17.

Full text available as:
PDF - Requires Adobe Acrobat Reader or other PDF viewer.

Abstract

Competitive Intelligence (CI) aims to monitor a firm’s external environment for information relevant to its decision-making process. As an excellent information source, the Internet provides significant opportunities for CI professionals as well as the problem of information overload. Internet search engines have been widely used to facilitate information search on the Internet. However, many problems hinder their effective use in CI research. In this paper, we introduce the Competitive Intelligence Spider, or CI Spider, designed to address some of the problems associated with using Internet search engines in the context of competitive intelligence. CI Spider performs real-time collection of Web pages from sites specified by the user and applies indexing and categorization analysis on the documents collected, thus providing the user with an up-to-date, comprehensive view of the Web sites of user interest. In this paper, we report on the design of the CI Spider system and on a user study of CI Spider, which compares CI Spider with two other alternative focused information gathering methods: Lycos search constrained by Internet domain, and manual within-site browsing and searching. Our study indicates that CI Spider has better precision and recall rate than Lycos. CI Spider also outperforms both Lycos and within-site browsing and searching with respect to ease of use. We conclude that there exists strong evidence in support of the potentially significant value of applying the CI Spider approach in CI applications.

EPrint Type:Journal Article (Paginated)
Keywords:National Science Digital Library, NSDL, Artificial Intelligence Lab, AI Lab, Spider, Competitive Intelligence
Subjects:Internet
Artificial Intelligence
World Wide Web
ID Code:421
Deposited On:16 August 2004
Alternative Locations:http://ai.bpa.arizona.edu/go/papers.html
Eprint Statistics:View statistics for this eprint
Tell A Colleague:Tell a colleague about it.

[1] R.D. Aaron, Giving away the store? Tell people about your

company on your Web site– but don’t overdo it, Competitive

Intelligence Review 8 (2) (1997) 80– 82.

[2] S. Chakrabarti, M. van der Berg, B. Dom, Focused crawling: a

new approach to topic-specific Web resource discovery, Proceedings

of the 8th International World Wide Web Conference

(Toronto, Canada, May 1999).

[3] H. Chen, Y. Chung, M. Ramsey, C.C. Yang, An intelligent

Personal Spider (agent) for dynamic Internet/Intranet searching,

Decision Support Systems 23 (1) (1998) 41–58.

[4] H. Chen, A. Houston, R. Sewell, B. Schatz, Internet browsing

and searching: user evaluations of category map and concept

space techniques, Journal of the American Society for Information

Science, Special Issue on ‘‘AI Techniques for Emerging

Information Systems Applications’’ 49 (7) (1998) 582–

603.

[5] A. Dutka, Competitive Intelligence for the Competitive Edge,

NTC Business Books, Chicago, IL, 1998.

[6] Futures Group, Ostriches & Eagles 1997, The Futures Group

Articles, 1998.

[7] B. Gilad, T. Gilad, The Business Intelligence System, AMACOM,

New York, 1988.

[8] Inktomi WebMap, available at http://www.inktomi.com/webmap/.

[9] R.J. Johnson, A Cognitive Approach to the Representation of

Managerial Competitive Intelligence Knowledge, Doctoral dissertation,

The University of Arizona, 1994.

[10] B.E. Keiser, Practical competitor intelligence, Planning Review

8 (1987) 14– 18.

[11] T. Kohonen, Self-Organizing Maps, Springer-Verlag, Berlin,

1995.

[12] S. Lawrence, C.L. Giles, Accessibility of information on the

Web, Nature 400 (1999) 107–109.

[13] C. Lin, H. Chen, J. Nunamaker, Verifying the proximity and

size hypothesis for self-organizing maps, Journal of Management

Information Systems 16 (3) (1999–2000) 61–73.

[14] P. Maes, Agents that reduce work and information overload,

Communications of the ACM 37 (7) (July 1994) 31– 40.

[15] J.J. McGonagle, C.M. Vella, Outsmarting the Competition,

Sourcebooks, Naperville, IL, 1990.

[16] J.J. McGonagle, C.M. Vella, The Internet Age of Competitive

Intelligence, Quorum Books, London, 1999.

[17] M. McQuaid, T. Ong, H. Chen, J. Nunamker, Multidimensional

scaling for group memory visualization, Decision Support

Systems 27 (1999) 163– 176.

[18] R. Orwig, H. Chen, J. Nunamaker, A graphical, self-organizing

approach to classifying electronic meeting output, Journal

of the American Society for Information Science 48 (2) (1997)

157– 170.

[19] J.E. Prescott, D.C. Smith, SCIP: who we are, what we do,

Competitive Intelligence Review 2 (1) (1991) 3 – 5.

[20] D. Roussinov, H. Chen, Document clustering for electronic

meetings: an experimental comparison of two techniques, Decision

Support Systems 27 (1999) 67– 69.

[21] G. Salton, Another look at automatic text-retrieval systems,

Communications of the ACM 29 (7) (1986) 648– 656.

[22] Society of Competitive Intelligence Professionals, http://

www.scip.org/.

[23] H. Sutton, Competitive Intelligence, The Conference Board,

New York, Report 913, 1988.

[24] K.M. Tolle, H. Chen, Comparing noun phrasing techniques

for use with medical digital library tools, Journal of the American

Society for Information Science 51 (4) (Apr. 2000) 352–

370.

[25] K.W. Tyson, Business Intelligence: Putting It All Together,

Leading Edge Publications, Lombard, IL, 1986.

[26] R.G. Vedder, M.T. Vanecek, C.S. Guynes, J.J. Cappel, CEO

and CIO perspectives on competitive intelligence, Communications

of the ACM 42 (8) (Aug. 1999) 109– 116.

[27] E. Voorhees, D. Harman, Overview of the sixth text retrieval

conference (TREC-6), in: E. Voorhees, D. Harman (Eds.),

NIST Special Publication 500-240: The Sixth Text Retrieval

Conference (TREC-6), National Institute of Standards and

Technology, Gaithersburg, MD, USA, 1997.

[28] C.C. Yang, J. Yen, H. Chen, Intelligent Internet searching

agent based on hybrid simulated annealing, Decision Support

Systems 28 (2000) 269– 277.

[29] O. Zamir, O. Etzioni, Grouper: a dynamic clustering interface

to Web search Results, Proceedings of the 8th International

World Wide Web Conference (Toronto, Canada, May 1999).

EPrints dLIST, an open access archive for the Information Sciences, is supported by the School of Information Resources and Library Science and Learning Technologies Center, University of Arizona. Established in 2002, dLIST has a global Advisory Board and is a part of the Information Technology & Society Research Lab. Open Archives
Contact: Admin | Donate