year = 2012\r
}\r
\r
-@book{Introduction_to_information_retrieval,\r
- abstract = {Class-tested and coherent, this textbook teaches classical and web information retrieval, including web search and the related areas of text classification and text clustering from basic concepts. It gives an up-to-date treatment of all aspects of the design and implementation of systems for gathering, indexing, and searching documents; methods for evaluating systems; and an introduction to the use of machine learning methods on text collections. All the important ideas are explained using examples and figures, making it perfect for introductory courses in information retrieval for advanced undergraduates and graduate students in computer science. Based on feedback from extensive classroom experience, the book has been carefully structured in order to make teaching more natural and effective.},\r
- added-at = {2012-05-30T10:50:27.000+0200},\r
- address = {Cambridge, UK},\r
- author = {Manning, Christopher D. and Raghavan, Prabhakar and Sch{\"u}tze, Hinrich},\r
- biburl = {\url{http://www.bibsonomy.org/bibtex/28516d94c1f7aa1e391ddd3ace4caa23b/flint63}},\r
- file = {Cambridge University Press Product Page:http\://www.cambridge.org/9780521865715:URL;Amazon Search inside:http\://www.amazon.de/gp/reader/0521865719/:URL;Google Books:http\://books.google.de/books?isbn=978-0-521-86571-5:URL},\r
- PAGES = {118-120},\r
- groups = {public},\r
- interhash = {b6954037b1d444f4afe4cad883b4d80c},\r
- intrahash = {8516d94c1f7aa1e391ddd3ace4caa23b},\r
- isbn = {978-0-521-86571-5},\r
- keywords = {v1205 book ai information retrieval language processing search xml web},\r
- publisher = {Cambridge University Press},\r
- timestamp = {2012-05-30T10:50:27.000+0200},\r
- title = {Introduction to Information Retrieval},\r
- username = {flint63},\r
- year = 2008\r
-}\r
-\r
@MISC{ententen,\r
key = "{Corpus}",\r
title = "{Sketch Engine EnTenTen Corpus}",\r
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{document}
-\title{Neco Simonovo and Feature Type Selection for Pairwise Document Comparison}
+\title{Diverse Queries and Feature Type Selection for Pairwise Document Comparison}
%%% Please do not remove the subtitle.
\subtitle{Notebook for PAN at CLEF 2013}
on uncovering plagiarism, authorship, and social software misuse.
We present modified three-way search methodology for Source Retrieval subtask and analyse snippet similarity performance.
The results show, that presented approach is adaptable in real-world plagiarism situations.
-For the detailed comparison task, we discuss feature type selection,
-global postprocessing. We have significantly improved the pairwise comparison
+For the Detailed Comparison task, we discuss feature type selection,
+global postprocessing. We significantly improved the pairwise comparison
results with even further optimizations possible.
\end{abstract}
types of queries. We present a new type of query based on text paragraphs.
The query execution were controled by its type and by preliminary similarities
discovered during the searches.
-In Section~\ref{text_alignment} we describe our approach for the text alignment
+Section~\ref{text_alignment} describes our approach for the text alignment
(pairwise comparison) subtask. We briefly introduce our system,
-and then we discuss the feature types, which are usable for pairwise comparison,including the evaluation of their feasibility for this purpose. We then describe
+and then we discuss the feature types, which are usable for pairwise comparison,
+including the evaluation of their feasibility for this purpose. We then describe
the global (corpus-wide) optimizations used, and finally we discuss
the results achieved and further development.
\section{Conclusions}
-
-Unfortunately the ChatNoir search engine does not support phrasal search, therefore it
+We introduces querying strategy with snippet similarity measure which approved to be
+competitive. In source retrieval subtask the strategy performed with the second best ratio
+of recall to the number of used queries.
+We focused our queries on selected parts of text
+and on parts with no discovered external similarities.
+Unfortunately the ChatNoir search engine currently does not support phrasal search, therefore it
is possible that evaluated results may be quite distorted in this manner.
In the text alignment subtask, we have achieved a significant improvement
Their purpose is to retrieve documents concerning the same theme.\r
%Two documents discussing the same theme usually share a set of overlapping keywords. Also the combination of keywords in query matters. \r
As a method for automated keywords extraction, we used a frequency based approach described in~\cite{suchomel_kas_12}.\r
-The method combines term frequency analysis with TF-IDF score~\cite{Introduction_to_information_retrieval}. As a reference\r
-corpus we used English web corpus~\cite{ententen} crawled by SpiderLink~\cite{SpiderLink} in 2012 which contains 4.65 billion tokens. \r
+The method combines term frequency analysis with TF-IDF score.\r
+As a reference corpus we used English web corpus~\cite{ententen} crawled by SpiderLink~\cite{SpiderLink} in 2012 which contains 4.65 billion tokens. \r
\r
-Each keywords based query was constructed from five top ranked keywords consecutively. Each keyword was\r
-used only in one query. Too long keywords based queries would be overspecific and it would have resulted\r
-in a low recall. On the other hand having constructed too short queries (one or two tokens) would have resulted\r
-in a low precision and also possibly low recall since they would be too general.\r
+Each keywords based query was constructed from five top ranked keywords consecutively.\r
+Each keyword was used only in one query. \r
+%Too long keywords based queries would be overspecific and it would have resulted in a low recall.\r
+%On the other hand having constructed too short queries (one or two tokens) would have resulted in a low precision and also possibly low recall since they would be too general.\r
In order to direct the search more at the highest ranked keywords we also extracted their \r
-most frequent two and three term long collocations. These were combined also into queries of 5 words.\r
+most frequent two and three term long collocations.\r
+These were combined also into queries of 5 words.\r
Resulting the 4 top ranked keywords alone can appear in two different queries, one from the keywords\r
alone and one from the collocations.\r
%Collocation describes its keyword better than the keyword alone. \r
\subsubsection{Intrinsic Plagiarism Based Queries.}\r
The second type of queries purpose to retrieve pages which contain text detected\r
as different, in a manner of writing style, from other parts of the suspicious document.\r
-Such a change may point out plagiarized passage which is intrinsically bound up with the text. \r
-We implemented vocabulary richness method which computes average word frequency class value for \r
-a given text part. The method is described in~\cite{awfc}.\r
+%Such a change may point out plagiarized passage which is intrinsically bound up with the text. \r
+%We implemented vocabulary richness method which computes average word frequency class value for \r
+%a given text part. The method is described in~\cite{awfc}.\r
+For this purpose we implemented vocabulary richness method~\cite{awfc} together with\r
+sliding windows concept for text chunking as described in~\cite{suchomel_kas_12}.\r
%The problem is that generally methods based on the vocabulary statistics work better for longer texts.\r
%According to authors this method scales well for shorter texts than other text style detection methods. \r
-The usage of this method is in our case limited by relatively short texts.\r
-It is also difficult to determine\r
-what parts of text to compare. Therefore we used sliding window concept for text chunking with the \r
-same settings as described in~\cite{suchomel_kas_12}.\r
+%The usage of this method is in our case limited by relatively short texts.\r
+%It is also difficult to determine\r
+%what parts of text to compare. Therefore we used sliding window concept for text chunking with the \r
+%same settings as described in~\cite{suchomel_kas_12}.\r
\r
A representative sentence longer than 6 words was randomly selected among those that apply from the suspicious part of the document.\r
The query was created from the representative sentence leaving out stop words.\r
-The intrinsic plagiarism based queries are positional. They carry the position of the representative sentence in the document.\r
+The intrinsic plagiarism based queries are positional. They carry the position of the representative sentence.% in the document.\r
They are phrasal, since they represent a search for a specific sentence. And they are\r
nondeterministic, because the representative sentence is selected randomly. \r
\r
\subsection{Search Control}\r
For each suspicious document we prepared all three types of queries during the first phase at once.\r
Queries were executed stepwise. \r
-After processing each query the results were evaluated %(see the following subsection~\ref{resSelection} for more details)\r
-and from \r
-all textual similarities between each result and the suspicious document the suspicious document intervals of those similarities\r
+After processing each query the results were evaluated. %(see the following subsection~\ref{resSelection} for more details)\r
+From all textual similarities between each result and the suspicious document, the document intervals of those similarities\r
were marked as {\it discovered}. \r
+\r
%At first there were processed the keywords based queries.\r
Firstly, there were always all of the keywords based queries executed. \r
%After having all the keywords based queries processed,\r
Secondly the intrinsic plagiarism based queries were executed according to \r
their creation sequence. \r
-Since they carry its position, not all of them were carried out.\r
+%Since they carry its position, not all of them were carried out.\r
During the execution, if any of the query position intersected with any of the {\it discovered} interval, the\r
query was dropped out. Analogically, the last there were paragraph based queries processed. \r
\r
This search control results in two major properties. Firstly, the source retrieval effort was increased \r
in parts of the suspicious document, where there has not yet been found any textual similarity.\r
-It was increased especially by the paragraph based queries. And secondly, after detection a similarity for a certain part of the text,\r
+%It was increased especially by the paragraph based queries.\r
+And secondly, after detection a similarity for a certain part of the text,\r
no more intentionally retrieval attempts for that part were effectuated. Meaning that all\r
discovered search engine results were evaluated, but there were executed no more queries regarding that passage.\r
\r
\r
\subsection{Result Selection}~\label{resSelection}\r
The second main decisive area about source retrieval task is to decide which from the search engine results to download.\r
-This process is represented in Figure~\ref{fig:source_retr_process} as 'Selecting'. \r
-Nowadays in the real-world a download is cheap and quick operation.\r
+This process is represented in Figure~\ref{fig:source_retr_process} as Selecting. \r
+%Nowadays in the real-world a download is cheap and quick operation.\r
%There can be some disk space considerations if there is a need to store original downloaded documents.\r
- The main cost probably represents a document post processing. \r
-Mainly on the Internet there is a wide range of file formats, which for text alignment must be\r
-converted into plaintext. This can be time and computational-consuming.\r
-For example from many pdf documents the plain text is hardly extractable, thus one need to use optical character recognition methods.\r
+% The main cost probably represents document post processing. \r
+%Mainly on the Internet there is a wide range of file formats, which for text alignment must be\r
+%converted into plaintext. This can be time and computational-consuming.\r
+%For example from many pdf documents the plain text is hardly extractable, thus one need to use optical character recognition methods.\r
\r
The ChatNoir offers snippets for discovered documents. The snippet generation is considered costless\r
operation. The snippet purpose is to have a quick glance at a small extract of resulting page.\r
https://www.fi.muni.cz/~kas/git / pan13-paper.git / commitdiff