-\section{Source Retrieval}\r
+\section{Source Retrieval}~\label{source_retr}\r
The source retrieval is a subtask in a plagiarism detection process during\r
which only a relatively small subset of documents are retrieved from the\r
large corpus. Those candidate documents are usually further compared in detail with the\r
\r
The test corpus contained 58 documents each discussing only one theme.\r
Those documents were created intentionally by\r
- semiprofessional writers, thus they feature nearly realistic plagiarism cases~\cite{plagCorpus}. \r
- Such conditions are similar to a realistic plagiarism detection scenario, such as for\r
-state of the art commercial plagiarism detection systems or the anti-plagiarism service developed on and\r
-utilized at the Masaryk University. The main difference between real-world corpus \r
+ semiprofessional writers, thus they featured nearly realistic plagiarism cases~\cite{plagCorpus}.\r
+Resources were looked up in the ClueWeb\footnote{\url{http://lemurproject.org/clueweb09.php/}} corpus.\r
+ Such conditions are similar to a realistic plagiarism detection scenario.%, such as for state of the art commercial plagiarism detection systems or the anti-plagiarism service developed on and utilized at the Masaryk University.\r
+The main difference between real-world corpus \r
of suspicious documents such as for example corpus created from theses stored in the Information System of Masaryk University\r
and the corpus of suspicious documents used during the PAN 2013 competition is that in the PAN\r
-corpus each document contains plagiarism passages. Therefore we can deepen the search during the process\r
-in certain parts of the document where no similar passage has yet been found. This is the main\r
-idea of improving recall of detected plagiarism in a suspicious document.\r
+corpus each document contains plagiarized passages. Therefore we can expected existence of\r
+a plagiarized passage and deepen the search during the process\r
+in certain parts of the document where no similar passage has yet been found.\r
+% This is the main idea of improving recall of detected plagiarism in a suspicious document.\r
\r
\r
\begin{figure}\r
we need to find original documents from which the plagiarized document was created.\r
During the process the plagiarist locates original documents with the use of a search engine.\r
The user decides what query the search engine to ask and which of the results from the result page to use.\r
-In the real-world scenario the corpus is the whole Web and the search engine can be a contemporary commercial search engine\r
-which scales to the size of the Web. This methodology is based on the fact that we do not\r
+\r
+%In the real-world scenario the corpus is the whole Web and the search engine can be a contemporary commercial search engine which scales to the size of the Web.\r
+ \r
+The same methodology -- utilizing a search engine is use for the source retrieval. \r
+This approach is based on the fact that we do not\r
possess enough resources to download and effectively process the whole corpus.\r
-In the case of PAN 2013 competition the corpus\r
-of source documents is the ClueWeb\footnote{\url{http://lemurproject.org/clueweb09.php/}} corpus.\r
+In the case of PAN 2013 competition we utilized the ChatNoir~\cite{chatnoir} \r
+search engine which indexes the English subset of the ClueWeb.\r
\r
-As a document retrieval tool for the competition we utilized the ChatNoir~\cite{chatnoir} search engine which indexes the English\r
-subset of the ClueWeb. \r
The reverse engineering decision process resides in creation of suitable queries on the basis of the suspicious document\r
and in decision what to actually download and what to report as a plagiarism case from the search results.\r
\r
These first two stages are shown in Figure~\ref{fig:source_retr_process} as Querying and Selecting. Selected results \r
from the search engine are forthwith textually aligned with the suspicious document (see section~\ref{text_alignment} for more details).\r
-This is the last decision phase -- what to report.\r
+%This is the last decision phase -- what to report.\r
If there is any continuous passage of reused text detected, the result document is reported\r
- and the continuous passages in the suspicious document are marked as 'discovered' and no further processing\r
+ and the continuous passages in the suspicious document are marked as {\it discovered} and no further processing\r
of those parts is done. \r
\r
\subsection{Querying}\r
Positional queries carry extra information about a textual interval in the suspicious document which the query represents.\r
A phrasal query aims for retrieval of documents containing the same small piece of a text. They are usually created from closely coupled words. \r
Deterministic queries for specific suspicious document are always the same no matter how many times we run the software. \r
-On the contrary the software can create in two runs potentially different nondeterministic queries.\r
+%On the contrary the software can create in two runs potentially different nondeterministic queries.\r
\r
\subsubsection{Keywords Based Queries.}\r
The keywords based queries are composed of automatically extracted keywords from the whole suspicious document.\r
\subsection{Search Control}\r
For each suspicious document we prepared all three types of queries during the first phase at once.\r
Queries were executed stepwise. \r
-After processing each query the results were evaluated (see the following subsection~\ref{resSelection} for more details) and from\r
+After processing each query the results were evaluated %(see the following subsection~\ref{resSelection} for more details)\r
+and from \r
all textual similarities between each result and the suspicious document the suspicious document intervals of those similarities\r
-were marked as 'discovered'. \r
-At first there were processed the keywords based queries. All of them were\r
-always executed. \r
-After having all the keywords based queries processed, the intrinsic plagiarism based queries were executed according to \r
+were marked as {\it discovered}. \r
+%At first there were processed the keywords based queries.\r
+Firstly, there were always all of the keywords based queries executed. \r
+%After having all the keywords based queries processed,\r
+Secondly the intrinsic plagiarism based queries were executed according to \r
their creation sequence. \r
-Since they carry its position, not all of the intrinsic plagiarism based queries were carried out.\r
-During the execution, if any of the query position intersected with any of the 'discovered' interval, the\r
-query was dropped out. In the same way, the last paragraph based queries were processed. \r
+Since they carry its position, not all of them were carried out.\r
+During the execution, if any of the query position intersected with any of the {\it discovered} interval, the\r
+query was dropped out. Analogically, the last there were paragraph based queries processed. \r
\r
This search control results in two major properties. Firstly, the source retrieval effort was increased \r
in parts of the suspicious document, where there has not yet been found any textual similarity.\r
\subsection{Result Selection}~\label{resSelection}\r
The second main decisive area about source retrieval task is to decide which from the search engine results to download.\r
This process is represented in Figure~\ref{fig:source_retr_process} as 'Selecting'. \r
-Nowadays in the real-world a download is very cheap and quick operation. There can be some disk space considerations\r
-if there is a need to store original downloaded documents. The main cost represents documents post processing. \r
+Nowadays in the real-world a download is cheap and quick operation.\r
+%There can be some disk space considerations if there is a need to store original downloaded documents.\r
+ The main cost probably represents a document post processing. \r
Mainly on the Internet there is a wide range of file formats, which for text alignment must be\r
-converted into plaintext. This can time and computational-consuming. For example from many\r
-pdf documents the plain text is hardly extractable, thus one need to use optical character recognition methods.\r
+converted into plaintext. This can be time and computational-consuming.\r
+For example from many pdf documents the plain text is hardly extractable, thus one need to use optical character recognition methods.\r
\r
The ChatNoir offers snippets for discovered documents. The snippet generation is considered costless\r
operation. The snippet purpose is to have a quick glance at a small extract of resulting page.\r
Since the snippet is relatively small and it can be discontinuous part of the text, the \r
text alignment methods described in section~\ref{text_alignment} were insufficient \r
in decision making over document download. Therefore we chose to compare existence of snippet word tuples\r
-in the suspicious document. For 1-tuples the measure means how many words from the snippet\r
-also exist in the suspicious document. If the snippet contains many common words they may\r
-also occur in many documents. In this case the 1-tuple measurement is little decisive. \r
+in the suspicious document. \r
+%For 1-tuples the measure means how many words from the snippet\r
+%also exist in the suspicious document. If the snippet contains many common words they may\r
+%also occur in many documents. In this case the 1-tuple measurement is little decisive. \r
\r
We used 2-tuples measurement, which indicates how many neighbouring word pairs coexist in the snippet and in the suspicious document.\r
We decided according to this value whether to download the source or not. For the deduction \r
of the threshold value we used 4413 search results from various queries according to documents \r
in the training corpus. Each resulting document was textually aligned to its corresponding suspicious document.\r
-One similarity represents continuous passage of text alignment similarity as is described in the following section~\ref{text_alignment}.\r
-In this way we obtained 248 similarities in total after downloading all of the 4431 documents.\r
+%One similarity represents continuous passage of text alignment similarity as is described in the following section~\ref{text_alignment}.\r
+In this way we calculated 248 similarities in total after downloading all of the 4431 documents.\r
\r
The 2-tuples similarity performance is depicted in Figure~\ref{fig:snippet_graph}.\r
Horizontal axis represents threshold of the 2-tuples similarity percentage between the snippet and the suspicious document.\r
-The graph curves represent obtain resource percentage according to the snippet similarity threshold.\r
+The graph curves represent obtained resource percentage according to the snippet similarity threshold.\r
A profitable threshold is the one with the largest distance between those two curves.\r
We chose threshold of the snippet similarity to 20\%, which in the graph corresponds to 20\% of all\r
downloads and simultaneously with 70\% discovered similarities.\r
performance measures. Possibly each approach has its pros and cons and many approaches\r
are usable in different situations. \r
\r
-We believe that in the realistic plagiarism detection the most important is keeping the number of\r
-queries low and simultaneously maximizing recall. \r
+We believe that in the realistic plagiarism detection the most important is to keep the number of\r
+queries low and simultaneously maximize recall. \r
% It is often some tradeoff between cost and efectivness.\r
-It is also advisable to keep the number of downloads down, but on the other hand,\r
+It is also advisable to keep the number of downloads low, but on the other hand,\r
it is relatively cheep and easily scalable operation.\r
\r
-Our approach had the second best ration of recall to the number of used queries, which\r
+Our approach had the second best ratio of recall to the number of used queries, which\r
tells about the query efficacy. The approach with the best ratio used few queries (4.9 queries per document which\r
was 0.4 of the amount we used), but also obtained the lowest recall (0.65 of our recall).\r
The approach with highest recall (and also lowest precision) achieved 2.8 times higher recall with 3.9 times more queries compared to ours.\r
https://www.fi.muni.cz/~kas/git / pan13-paper.git / commitdiff