Improved Algorithms for Topic Distillation in a Hyperlinked Environment

Abstract This paper addresses the problem of topic distillation on the World Wide Web, namely, given a typ-ical user query to find quality documents related to the query topic. Connectivity analysis has been shown to be useful in identifying high quality pages within a topic specific graph of hyperlinked documents. The essence of our approach is to augment a previous connectivity anal-ysis based algorithm with content analysis. We identify three problems with the existing approach and devise al-gorithms to tackle them. The results of a user evaluation are reported that show an improvement of precision at 10 documents by at least 45 % over pure connectivity anal-ysis. 1