Dalibor Klusáček - personal webpage

CERIT-SC (a part of MetaCentrum) workload log (January - April 2015)

Here you can find the data sets generated from TORQUE traces during the first 4 months of the year 2015. The data sets contains:
  • job descriptions (102,657 jobs, divided per-user, with specified batches and their mutual dependencies)
  • machine description (8 clusters, ~4,000 CPUs)

Usage rules

The CERIT-SC workload log was graciously provided by the CERIT Scientific Cloud. If you use this log in your work, please use a similar acknowledgment.

File format

Job description
The job log is provided in a (per-user) format suitable for dynamic workload simulations using Zakay and Feitelson's dynamic model. Otherwise, it is in (extended) Standard Workload Format (SWF).
The log can be obtained here [2.3 MB, zip file]
The log format is compatible with the Alea jobs scheduling simulator.

CERIT-SC (a part of MetaCentrum) workload log (year 2013)

Here you can find the data sets generated from TORQUE traces during the first 3 months of the year 2013. The data sets contains:
  • job descriptions (17,900 jobs)

Usage rules

The CERIT-SC workload log was graciously provided by the CERIT-SC and the Czech National Grid Infrastructure MetaCentrum. If you use this log in your work, please use a similar acknowledgment.

File format

Job description
The job log is in Standard Workload Format (SWF). The log can be obtained at: job descriptions [2.4 MB, SWF file]

MetaCentrum workload log (January-June-2013)

Here you can find the data sets generated from TORQUE traces during the first 6 months of the year 2013. The data sets contains:
  • job descriptions (495,299 jobs)
  • machine descriptions
  • queue descriptions

Usage rules

The workload log was graciously provided by the Czech National Grid Infrastructure MetaCentrum. If you use this log in your work, please use a similar acknowledgment.

File format

Job description
The job log is in Standard Workload Format (SWF). The log can be obtained at: workload archive [6.9 MB, ZIP archive]

Zewura workload log (year 2012)

Here you can find the data sets generated from TORQUE traces during the first five months of the year 2012.
Zewura log coveres five months of job execution (January - May, 2012). Zewura cluster consists of 20 shared memory machines. Each machine has 80 CPUs and 512 GB of RAM.
The data sets contains:
  • job descriptions (17,256 jobs)

Usage rules

The Zewura workload log was graciously provided by the Czech National Grid Infrastructure MetaCentrum. If you use this log in your work, please use a similar acknowledgment.

File format

Job description
The job log is in Standard Workload Format (SWF). The log can be obtained at: job descriptions [1.2 MB, SWF file]

MetaCentrum data sets (year 2009)

Here you can find the data sets generated from PBSpro traces during the first five months of the year 2009. The data sets contains:
  • job descriptions (103656 jobs)
  • node descriptions (14 clusters, 806 CPUs)
  • list of queues
  • descriptions of machines in maintenance (failures & upgrades)
  • descriptions of dedicated and reserved machines
  • SPEC CPU2006 benchmark results of MetaCentrum cluster's

Usage rules

The MetaCentrum workload log was graciously provided by the Czech National Grid Infrastructure MetaCentrum. If you use this log in your work, please use a similar acknowledgment.

File format

Job description
valuetypemeaningexample
job_id(int)job id123
user(String)job owneruser_43
queue_name(String)queue where the job was originaly submittedq2
used_CPUs(int)number of used CPUs16
used_nodes(int)number of used nodes2
required_properties[String;String;...]required properties that must be available on the target machine[p1;p2;p14]
used_main_memory(kB)(int)used memory2989520
arrival_time(int)in epoch format1230675195
start_time(int)in epoch format1230675200
end_time(int)in epoch format1230768010
duration(int)in seconds92810
exit_status(int)usually the exit status of the shell executing the job script1
id(s)_of_assigned_CPUslist of ints, separated by spacesCPUs where the job was executed (see Node description)16 17 18 19 20 21

:: View example

Node descriptions
valuetypemeaningexample
node_id(int)id of the node 11
node_name(String)node namecluster_9
cpu_speed(MHz)(int)CPU speed in MHz2400
main_memory_size(kB)(int) RAM size in kB of one node's machine4000000
CPU_type(String) CPU typeXeon
operating_system(String) OSlinux
list_of_supported_properties(list of Strings separated by commas) list of supported propertiesp2,p12,p25,p10,p22,p9
number_of_machines(int)the number of machines within this node3
total_number_of_CPUs(int)the number of CPUs within this node8
list_of_number_of_CPUs_on_each_machine(list of ints separated by commas) each int represents the number of CPUs of some node's machine2,2,4
list_of_CPUs_ids(list of ints)CPU ids corresponding to the ids stored in the job description file16,17,18,19,20,21,22,23

:: View example

List of queues
valuetypemeaningexample
queue_name(int)queue name as appears in job descriptionq3
queue_priority(int)priority of this queue - used by the PBSpro scheduler to define the order of queue selection during scheduling process (higher number = queue is chosen sooner)50
time limit(int)job time limit in hours. If the execution time of a job being submitted into this queue exceeds this limit then the job is killed.24

:: View example

Maintenance & reservations & dedicated machines
valuetypemeaningexample
start_time(int)when machine became failed / reserved / dedicated / unavailable (in epoch format)1231771012
node_name(String)name of the nodecluster_8
duration(int)the duration of the failure / reservation /... (in seconds)11231
affected_machineslist of machine's ids (int) separated by spacesdefines which machines from node were failed / reserved / ...2 3 4

:: View example

Distribution of jobs into queues

queue name# of jobs
q378183
q213059
q45845
q14629
q5720
q6696
q9282
q10123
q7113
q116

Job paralelism = 14,06%

used CPUs# of jobs
189078
46330
24234
81689
3963
16671
12187
6124
3298
2086
2441
540
1038
2828
1512
1410
78
134
213
183
602
112
302
311
221
91

SPEC CPU2006 benchmarks of MetaCentrum cluster's

Coming soon.

Download

MetaCentrum workload files (2009)
job descriptions [2.05 MB, zip archive]
node descriptions [txt file]
queue descriptions [txt file]
maintenance descriptions [txt file]
dedicated machines descriptions [txt file]
reserved machines descriptions [txt file]

Zewura workload (2012)
The log can be obtained at: job descriptions [1.2 MB, SWF file]

CERIT-SC (part of MetaCentrum) workload (2013)
The log can be obtained at: job descriptions [2.4 MB, SWF file]

If you have any questions, please contact me via the e-mail bellow.

© 2007-2016 Dalibor Klusáček | xklusac(at)fi.muni.cz