Computing Reviews

A quantitative study on the re-executability of publicly shared scientific workflows
Mayer R., Rauber A.  e-Science 2015 (Proceedings of the 2015 IEEE 11th International Conference on e-Science, Munich, Aug 31-Sep 4, 2015)IEEE,312-321,2015.Type:Proceedings
Date Reviewed: 07/20/16

Workflow languages and engines, such as the Taverna workflow suite, support the repeatable execution of computational experiments. A workflow can make use of resources and processes on the local host and on remote hosts. Unfortunately, workflows can break. The various reasons for breakage are enumerated. To determine the prevalence of each cause of breakage, an attempt was made to automatically re-execute 1,443 Taverna workflows held in the public workflow repository myExperiment.

Only 341 workflows were successfully automatically re-executed. Many (526) workflows were found to be missing some or all of the values required for their workflow input ports. Because an initial, basic check on the reachability of required web services failed, 180 workflows could not be re-executed. Numerous (364) workflows produced errors during execution. Included among these errors were 40 failures due to missing authentication data and 14 failures due to a resource, such as a file, not being available. Six workflows did not terminate after more than 48 hours of runtime.

To avoid the need for external calls, one recommendation is that the Taverna workflow suite be extended to allow other languages, such as Python or Perl, to be used to express process logic. Another recommendation is to provide support for the specification of matching input and output data.

While table 11 should have had a more detailed breakdown of the reasons for failure during execution, the several recommendations made are all sound. This paper is strongly recommended to those with an interest in workflow suites and the repeatability of computational experiments.

Reviewer:  Andy Brooks Review #: CR144603 (1611-0820)

Reproduction in whole or in part without permission is prohibited.   Copyright 2024 ComputingReviews.com™
Terms of Use
| Privacy Policy