Learning to Select Text Databases with Neural Nets

Abstract

As the number and diversity of text databases on the Internet increases rapidly, users are faced with finding the text databases that are relevant to the user query. Identifying the relevant text databases out of many candidates for a given query is called the text database selection problem. In this paper, we propose a neural net based approach to the text database selection problem. First, we present a Database Selection Agent that learns about underlying text databases using neural net mechanism. For a given query, the Database Selection Agent, which is sufficiently trained on the basis of the backpropagation learning procedure, finds the text databases associated with the relevant documents and retrieves those documents effectively. In order to scale our approach with the large number of text databases, we also propose the hierarchical organization of Database Selection Agents which reduces the total training cost at the acceptable level. Finally, we evaluate the performance of our approach by comparing it to those of the conventional well-known approaches.