TwiSNER: Semi-supervised Method for Named Entity Recognition from Text Streams on Twitter

Abstract

The data on Social Network Services (SNSs) has recently become an interesting source for researchers conducting different Natural Language Processing (NLP) experiments, such as sentiment analysis, information extraction, Named Entity Recognition (NER), and so on. The characteristics of SNS data are usually described as short, noisy, with insufficient supplemental information. They often contain grammatical errors, misspellings, and unreliable capitalization. Thus, standard NLP tools (e.g., NER systems) have difficulty obtaining good results when they are applied on these data, even if they perform well on well-formatted texts. Most of the traditional NER methods are based on supervised learning techniques that often require a large amount of standard training data to train a classifier. In this paper, we propose a method called TwiSNER to classify named entities in Twitter data (called tweets) by using a semi-supervised learning approach combined with the conditional random field model, hand-made rules, and the co-occurrence coefficient of the featured words surrounding entities. In the experiments, TwiSNER is applied on a dataset collected from Twitter, which includes 11,425 tweets for training with 4,716 labeled tweets and 1,450 tweets for testing. TwiSNER produces promising results, where the best F-measure is better than the baselines.