A hybrid approach for extracting informative content from web pages


Uzun E., AGUN H. V., YERLİKAYA T.

Information Processing and Management, cilt.49, sa.4, ss.928-944, 2013 (SCI-Expanded, SSCI, Scopus) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 49 Sayı: 4
  • Basım Tarihi: 2013
  • Doi Numarası: 10.1016/j.ipm.2013.02.005
  • Dergi Adı: Information Processing and Management
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Social Sciences Citation Index (SSCI), Scopus
  • Sayfa Sayıları: ss.928-944
  • Anahtar Kelimeler: Template Detection, Web Cleaning, Web Content Extraction, Web Learning Modeling
  • Trakya Üniversitesi Adresli: Evet

Özet

Eliminating noisy information and extracting informative content have become important issues for web mining, search and accessibility. This extraction process can employ automatic techniques and hand-crafted rules. Automatic extraction techniques focus on various machine learning methods, but implementing these techniques increases time complexity of the extraction process. Conversely, extraction through hand-crafted rules is an efficient technique that uses string manipulation functions, but preparing these rules is difficult and cumbersome for users. In this paper, we present a hybrid approach that contains two steps that can invoke each other. The first step discovers informative content using Decision Tree Learning as an appropriate machine learning method and creates rules from the results of this learning method. The second step extracts informative content using rules obtained from the first step. However, if the second step does not return an extraction result, the first step gets invoked. In our experiments, the first step achieves high accuracy with 95.76% in extraction of the informative content. Moreover, 71.92% of the rules can be used in the extraction process, and it is approximately 240 times faster than the first step. © 2013 Elsevier Ltd. All rights reserved.