Ctvrtecni upravy

diff --git a/pan13-paper/pan13-notebook.bib b/pan13-paper/pan13-notebook.bib
index 59c4aa082237448f51861c2319e8ccdf33a5cbcb..204d2fcea2023069b652900d6638d7f0e101a337 100755 (executable)
--- a/pan13-paper/pan13-notebook.bib
+++ b/pan13-paper/pan13-notebook.bib
@@ -12,7 +12,6 @@
         YEAR               = {2012}\r
 }\r
 \r
         YEAR               = {2012}\r
 }\r
 \r

-<<<<<<< HEAD

 @inproceedings{suchomel_kas_12,\r
   added-at = {2012-10-01T11:37:58.000+0200},\r
   author = {Suchomel, {\v S}imon and Kasprzak, Jan and Brandejs, Michal},\r
 @inproceedings{suchomel_kas_12,\r
   added-at = {2012-10-01T11:37:58.000+0200},\r
   author = {Suchomel, {\v S}imon and Kasprzak, Jan and Brandejs, Michal},\r


@@ -85,7 +84,8 @@
         TITLE              = {Crowdsourcing Interaction Logs to Understand Text Reuse from the Web},\r
         URL                = {},\r
         YEAR               = {2013}\r
         TITLE              = {Crowdsourcing Interaction Logs to Understand Text Reuse from the Web},\r
         URL                = {},\r
         YEAR               = {2013}\r

-=======
+}\r
+\r

 @INPROCEEDINGS{Kasprzak2009a,\r
   AUTHOR =       "Jan Kasprzak and Michal Brandejs and Miroslav K\v{r}ipa\v{c}",\r
   TITLE =        "Finding Plagiarism by Evaluating Document Similarities",\r
 @INPROCEEDINGS{Kasprzak2009a,\r
   AUTHOR =       "Jan Kasprzak and Michal Brandejs and Miroslav K\v{r}ipa\v{c}",\r
   TITLE =        "Finding Plagiarism by Evaluating Document Similarities",\r


@@ -124,5 +124,4 @@
   booktitle={CLEF (Online Working Notes/Labs/Workshop)},\r
   pages={1--8},\r
   year={2012}\r
   booktitle={CLEF (Online Working Notes/Labs/Workshop)},\r
   pages={1--8},\r
   year={2012}\r

->>>>>>> 2278ad058d0a6e0c2228741c76aece9ace432912

 }\r
 }\r

diff --git a/pan13-paper/simon-source_retrieval.tex b/pan13-paper/simon-source_retrieval.tex
index d5b338b948a6cc4a13fac8319d8bb51328861b7e..29e97818b4c29103fc7cc8a5c9f1c233515a0594 100755 (executable)
--- a/pan13-paper/simon-source_retrieval.tex
+++ b/pan13-paper/simon-source_retrieval.tex
@@ -73,7 +73,7 @@ The method combines term frequency analysis with TF-IDF score~\cite{Introduction
 corpus we used English web corpus~\cite{ententen} crawled by SpiderLink~\cite{SpiderLink} in 2012 which contains 4.65 billion tokens. \r
 \r
 Each keywords based query were constructed from five top ranked keywords consecutively. Each keyword were\r
 corpus we used English web corpus~\cite{ententen} crawled by SpiderLink~\cite{SpiderLink} in 2012 which contains 4.65 billion tokens. \r
 \r
 Each keywords based query were constructed from five top ranked keywords consecutively. Each keyword were\r

-used only in one query. Too long keywords based queries would be over-specific and it would have resulted\r
+used only in one query. Too long keywords based queries would be overspecific and it would have resulted\r

 in a low recall. On the other hand having constructed too short (one or two tokens) queries would have resulted\r
 in a low precision and also possibly low recall since they would be too general.\r
 \r
 in a low recall. On the other hand having constructed too short (one or two tokens) queries would have resulted\r
 in a low precision and also possibly low recall since they would be too general.\r
 \r


@@ -143,7 +143,7 @@ no more intentionally retrieval attempts for that part were effectuated. Meaning
 discovered search engine results were evaluated, but there were executed no more queries regarding that passage.\r
 \r
 \r
 discovered search engine results were evaluated, but there were executed no more queries regarding that passage.\r
 \r
 \r

-\subsection{Result Selection}\r
+\subsection{Result Selection}~\label{resSelection}\r

 The second main decisive area about source retrieval task is to decide which from the search engine results to download.\r
 This process is represented in figure~\ref{fig:source_retr_process} as 'Selecting'. \r
 Nowadays in real-world is download very cheap and quick operation. There can be some disk space considerations\r
 The second main decisive area about source retrieval task is to decide which from the search engine results to download.\r
 This process is represented in figure~\ref{fig:source_retr_process} as 'Selecting'. \r
 Nowadays in real-world is download very cheap and quick operation. There can be some disk space considerations\r


@@ -157,11 +157,6 @@ operation. The snippet purpose is to have a quick glance at a small extract of r
 The extract is maximally 500 characters long and it is a portion of the document around given keywords.\r
 On the basis of snippet, we needed to decide whether to actually download the result or not.\r
 \r
 The extract is maximally 500 characters long and it is a portion of the document around given keywords.\r
 On the basis of snippet, we needed to decide whether to actually download the result or not.\r
 \r

-Since the snippet is relatively small and it can be discontinuous part of the text, the \r
-text alignment methods described in section~\ref{text_alignment} were insufficient for \r
-\r
-\r
-\r

 \subsection{Snippet Control}\r
 \begin{figure}\r
   \centering\r
 \subsection{Snippet Control}\r
 \begin{figure}\r
   \centering\r


@@ -169,6 +164,46 @@ text alignment methods described in section~\ref{text_alignment} were insufficie
   \caption{Downloads and similarities performance.}\r
   \label{fig:snippet_graph}\r
 \end{figure}\r
   \caption{Downloads and similarities performance.}\r
   \label{fig:snippet_graph}\r
 \end{figure}\r

-\subsection{Source Retrieval Results}\r

 \r
 \r

+Since the snippet is relatively small and it can be discontinuous part of the text, the \r
+text alignment methods described in section~\ref{text_alignment} were insufficient \r
+in decision making over document download. Therefore we chose to compare existence of snippet word tuples\r
+in the suspicious document. For 1-tuples the measure means how many words from the snippet\r
+also exist in the suspicious document. If the snippet contains many common words they may\r
+also occur in many documents. In this case the 1-tuple measurement is little decisive. \r
+\r
+We used 2-tuples measurement, which indicates how many neighbouring word pairs coexist in the snippet and in the suspicious document.\r
+We decided according to this value whether to download the source or not. For the deduction \r
+ of the threshold value we used 4413 search results from various queries according to documents \r
+ in the training corpus. Each resulting document was textually aligned to its corresponding suspicious document.\r
+One similarity represents continuous passage of text alignment similarity as is described in the following section~\ref{text_alignment}.\r
+In this way we obtained 248 similarities in total after downloading all of the 4431 documents.\r
+\r
+The 2-tuples similarity performance is depicted in figure~\ref{fig:snippet_graph}.\r
+Horizontal axis represents threshold of the 2-tuples similarity percentage between the snippet and the suspicious document.\r
+The graph curves represent obtain resource percentage according to the snippet similarity threshold.\r
+A profitable threshold is the one with the largest distance between those two curves.\r
+We chose threshold of the snippet similarity to 20\%, which in the graph corresponds to 20\% of all\r
+downloads and simultaneously with 70\% discovered similarities.\r
+ \r
+\subsection{Source Retrieval Results}\r
+In PAN 2013 Source Retrieval subtask we competed with other 8 teams. \r
+There can not be selected the best approach because there were several independent\r
+performance measures. Possibly each approach has its pros and cons and many approaches\r
+are usable in different situations. \r
+\r
+We believe that in the realistic plagiarism detection the most important is keeping the number of\r
+queries low and simultaneously maximizing recall. \r
+% It is often some tradeoff between cost and efectivness.\r
+It is also advisable to keep the number of downloads down, but on the other hand,\r
+it is relatively cheep and easily scalable operation.\r
+\r
+Our approach had the second best ration of recall to the number of used queries, which\r
+tells about the query efficacy. The approach with the best ratio used few queries (4.9 queries per document which\r
+was 0.4 of the amount we used), but also obtained the lowest recall (0.65 of our recall).\r
+The approach with highest recall (and also lowest precision) achieved 2.8 times higher recall with 3.9 times more queries.\r
+\r
+Our approach achieved also low precision, which means we reported many more results and they\r
+were not considered as correct hits. On the other hand each reported result contained some\r
+textual similarity according to text alignment subtask score, which we believe is still worthwhile to report.\r

 \r
 \r