Measure time (ms) of selected operation running on card (e.g., AESKey.setKey()). Problem: Selected operation cannot be measured directly (e.g., elapsed = timeEnd - timeStart) because of missing on-card timer and protected environment of smart card (no possibility to time start and end of operation inside surrounding code). As a result, one can only measure overall time between send of input data that will trigger operation (PC/SC SCardTransmit()) on host system.
Time necessary to perform whole logical cryptographic operation on-card (e.g., transmit X bytes of data to card, encrypt it by AES and transmit back result) [FULL_OP].
Some operations can be measured without additional preparation of target object (e.g., AESKey.setKey()). Target method call is then iterated on-card according to TEST_SETTINGS.numRepeatWholeOperation to suppress of significance of overhead (apdu transmission etc.) on measurement
Additionally, target operation usually takes only small fraction of measurement with majority taken by operations we like to exclude from measurement (~one ms vs. ~tens of ms). Situation is additionally worsen by possible non-determistic time fluctuations on a host side.
Note: Measurement fluctuations on host side can be mitigated if simpler host architecture is available (e.g., microcontroller-based card reader).
Note: Very precise measurement of elapsed time can be obtained from a power trace, if start and end of selected operation can be identified. Such a measurement requires access to setup with osciloscope and significant time for identification of target operation. We verified selected operation using this method.
Note: Some operations cannot be meaningfully measured without additional operation executed together (e.g., setKey before target operation Cipher.init). We measure both operations together and then substract time for additional operation.
Note: We intentionally did not exclude outlayer measurements as it may contain interesting information regarding non-deterministic behaviour of a card
Listed segments show all parts of performance test execution. Part 5 executes target operation we want to measure (green). Since we are not able to perform this execution directly, by subtracting the time of execution of all operations in the left column from all operations in the right column, we gain quite an accurate operation run time. The subtracted time consists of host PC processing (yellow), data transmission between host PC and a card (yellow-blue) and smart card processing (blue).