The detailed comparison task of PAN 2012 consisted in a comparison
of given document pairs, with the expected output being the annotation of
similarities found between these documents.
-The submitted program has been run in a controlled environment
+The submitted program was running in a controlled environment
separately for each document pair, without the possibility of keeping any
-data between runs.
+cached data between runs.
%In this section, we describe our approach in the detailed comparison
%task. The rest of this section is organized as follows: in the next
\subsection{Differences Against PAN 2010}
Our approach in this task
-is loosely based on the approach we have used in PAN 2010 \cite{Kasprzak2010}.
+is loosely based on the approach we used in PAN 2010 \cite{Kasprzak2010}.
The main difference is that instead of looking for similarities of
one type (for PAN 2010, we have used word 5-grams),
-we have developed a method of evaluating multiple types of similarities
+we developed a method of evaluating multiple types of similarities
(we call them {\it common features}) of different properties, such as
density and length.
-As a proof of concept, we have used two types of common features: word
-5-grams and stop-word 8-grams, the later being based on the method described in
+As a proof of concept, we used two types of common features: word
+5-grams and stop word 8-grams, the later being based on the method described in
\cite{stamatatos2011plagiarism}.
-In addition to the above, we have made several minor improvements to the
-algorithm, such as parameter tuning and improving the detections
+In addition to the above, we made several minor improvements to the
+algorithm such as parameter tuning and improving the detections
merging in the post-processing stage.
\subsection{Algorithm Overview}
We tokenize the document into words, where word is a sequence of one
or more characters of the {\it Letter} Unicode class.
-With each word, two additional attributes, needed for further processing,
+With each word, two additional attributes needed for further processing,
are associated: the offset where the word begins, and the word length.
The offset where the word begins is not necessarily the first letter character
-of the word itself. We have discovered that in the training corpus,
+of the word itself. We discovered that in the training corpus
some plagiarized passages were annotated including the preceding
-non-letter characters. We have used the following heuristics to add
-parts of the inter-word gap to the previous, or the next adjacent word:
+non-letter characters. We used the following heuristics to add
+parts of the inter-word gap to the previous or the next adjacent word:
\begin{itemize}
-\item when the inter-word gap contains interpunction (any of the dot,
-semicolon, colon, comma, exclamation mark, question mark, or quotes),
-add the characters up to, and including the interpunction, to the previous
-word, ignore the space character(s) after the interpunction, and add
-the rest to tne next word.
-\item otherwise, when the inter-word gap contains newline, add the character
-before the first newline to the previous word, ignore the first newline
-character, and add the rest to the next word.
-\item otherwise, ignore the inter-word gap characters altogether.
+\item When the inter-word gap contains interpunction (any of the dot,
+semicolon, colon, comma, exclamation mark, question mark, or quotes):
+\begin{itemize}
+\item add the characters up to and including the interpunction character
+to the previous word,
+\item ignore the space character(s) after the interpunction
+character,
+\item add the rest to the next word.
+\end{itemize}
+\item Otherwise, when the inter-word gap contains newline:
+\begin{itemize}
+\item add the character before the first newline to the previous word,
+\item ignore the first newline character,
+\item add the rest to the next word.
+\end{itemize}
+\item Otherwise: ignore the inter-word gap characters altogether.
\end{itemize}
-When the detection program has been presented three different
+When the detection program was given three different
files instead of two (meaning the third one is machine-translated
-version of the second one), we have tokenized the translated document instead
-of the source one. We have make use of the line-by-line alignment of the
+version of the second one), we tokenized the translated document instead
+of the source one. We used the line-by-line alignment of the
source and machine-translated documents to transform the word offsets
and lengths in the translated document to the terms of the source document.
\begin{itemize}
\item Lexicographically sorted word 5-grams, formed of words at least
-three characters long, and
-\item unsorted stop-word 8-grams, formed from 50 most frequent words in English,
+three characters long.
+\item Unsorted stop word 8-grams, formed from 50 most frequent words in English,
as described in \cite{stamatatos2011plagiarism}. We have further ignored
the 8-grams, formed solely from the six most frequent English words
-(the, of, and, a, in, to), or the possessive {\it'{}s}.
+({\it the}, {\it of}, {\it and}, {\it a}, {\it in}, {\it to}), or the possessive {\it'{}s}.
\end{itemize}
-We have represented each feature with the 32 highest-order bits of its
-MD5 digest. This is only a performance optimization, targeted for
+We represented each feature with the 32 highest-order bits of its
+MD5 digest. This is only a performance optimization targeted for
larger systems. The number of features in a document pair is several orders
-of magnitude lower than $2^{32}$, so the probability of hash function
+of magnitude lower than $2^{32}$, thus the probability of hash function
collision is low. For pair-wise comparison, it would be feasible to compare
the features directly instead of their MD5 sums.
-Each feature has also the offset and length attributes.
+Each feature has also two attributes: offset and length.
Offset is taken as the offset of the first word in a given feature,
and length is the offset of the last character in a given feature
minus the offset of the feature itself.
\subsection{Common Features}
-For further processing, we have taken into account only the features
+For further processing, we took into account only the features
present both in source and suspicious document. For each such
-{\it common feature}, we have created the list of
+{\it common feature}, we created the list of
$(\makebox{offset}, \makebox{length})$ pairs for the source document,
and a similar list for the suspicious document. Note that a given feature
can occur multiple times both in source and suspicious document.
To detect a plagiarized passage, we need to find a set of common features,
which map to a dense-enough interval both in the source and suspicious
-document. In our previous work, we have presented the algorithm
+document. In our previous work, we described the algorithm
for discovering these {\it valid intervals} \cite{Kasprzak2009a}.
A similar approach is used also in \cite{stamatatos2011plagiarism}.
-Both of these algorithms use the features of a single type, which
+Both of these algorithms use features of a single type, which
allows to use the ordering of features as a measure of distance.
When we use features of different types, there is no natural ordering
-of them: for example, a stop-word 8-gram can span multiple sentences,
+of them: for example a stop word 8-gram can span multiple sentences,
which can contain several word 5-grams. The assumption of both of the
above algorithms, that the last character of the previous feature
is before the last character of the current feature, is broken.
-We have modified the algorithm for computing valid intervals with
+We modified the algorithm for computing valid intervals with
multi-feature detection to use character offsets
-only instead of feature order numbers. We have used valid intervals
+only instead of feature order numbers. We used valid intervals
consisting of at least 4 common features, with the maximum allowed gap
inside the interval (characters not belonging to any common feature
of a given valid interval) set to 4000 characters.
\subsection{Postprocessing}
\label{postprocessing}
-In the postprocessing phase, we took the resulting valid intervals,
-and made attempt to further improve the results. We have firstly
+In the postprocessing phase we took the resulting valid intervals
+and made attempt to further improve the results. We firstly
removed overlaps: if both overlapping intervals were
shorter than 300 characters, we have removed both of them. Otherwise, we
kept the longer detection (longer in terms of length in the suspicious document).
-We have then joined the adjacent valid intervals into one detection,
-if at least one of the following criteria has been met:
+We then joined the adjacent valid intervals into one detection,
+if at least one of the following criteria were met:
\begin{itemize}
\item the gap between the intervals contained at least 4 common features,
and it contained at least one feature per 10,000
characters\footnote{we have computed the length of the gap as the number
of characters between the detections in the source document, plus the
-number of charaters between the detections in the suspicious document.}, or
+number of charaters between the detections in the suspicious document.}
\item the gap was smaller than 30,000 characters and the size of the adjacent
-valid intervals was at least twice as big as the gap between them, or
+valid intervals was at least twice as big as the gap between them
\item the gap was smaller than 30,000 characters and the number of common
features per character in the adjacent interval was not more than three times
bigger than number of features per character in the possible joined interval.
\subsection{Other Approaches Explored}
-There are several other approaches we have evaluated, but which were
+There are several other approaches we evaluated, but which were
omitted from our final submission for various reasons. We think mentioning
-them here is worthwhile nevertheless.
+them here is worthwhile nevertheless:
\subsubsection{Intrinsic Plagiarism Detection}
intervals, if they both belong to the same passage, detected by
the intrinsic detector. This approach did not provide improvement
when compared to the static gap limits, as described in Section
-\ref{postprocessing}, so we have omitted it from our final submission.
+\ref{postprocessing}, therefore we have omitted it from our final submission.
%\subsubsection{Language Detection}
%
\subsubsection{Cross-lingual Plagiarism Detection}
For cross-lingual plagiarism detection, our aim was to use the public
-interface to Google translate if possible, and use the resulting document
+interface to Google Translate\footnote{\url{http://translate.google.com/}} if possible, and use the resulting document
as the source for standard intra-lingual detector.
Should the translation service not be available, we wanted
to use the fall-back strategy of translating isolated words only,
\subsubsection{Plagdet score}
-Our results from tuning the parameters show that the plagdet score\cite{potthastfamework}
+Our results from tuning the parameters show that the plagdet score\cite{potthastframework}
is not a good measure for comparing the plagiarism detection systems:
for example, the gap of 30,000 characters, described in Section \ref{postprocessing},
can easily mean several pages of text. And still the system with this
plagdet score using the official script in Python took between 120 and
180 seconds, as there is no parallelism in this script.
-When we have tried to use intrinsic plagiarism detection and language
+When we tried to use intrinsic plagiarism detection and language
detection, the first phase took about 12500 seconds. Thus omitting these
featurs clearly provided huge performance improvement.
https://www.fi.muni.cz/~kas/git / pan12-paper.git / blobdiff