Slim_Migrate_And_QA_nfs_output_path00
/home/bolette/TestOutput/
2014-02-16 18:54:47.634 UTC
Output directory fora the migrated wav files on nfs.
2014-02-16 18:54:23.579 UTC
mp3_list_on_hdfs_input_path00
path to input file on hdfs containing list of paths to mp3 files on nfs to be migrated
2014-02-16 18:52:53.590 UTC
input/mp3/filelist.txt
2014-02-16 18:53:38.293 UTC
hdfs_output_path_200
Output directory for preservation event files and other log files.
2014-02-13 13:13:40.529 UTC
output/test-output/MigrateMp3ToWav/
2014-02-13 13:14:10.587 UTC
mapreduce_output_path00
output/test2014-009
2014-02-16 18:51:54.334 UTC
output directory for Hadoop output
2014-02-16 18:51:32.272 UTC
jar_input_path00
/scape/shared/jars/
/home/bolette/Projects/scape-audio-qa/migrate_mp3_to_wav_hadoop/target
2014-02-26 10:03:13.601 UTC
The directory where the jar file with the hadoop jobs is.
2014-02-26 09:59:22.177 UTC
max_split_size00
max-split-size is the max input size to a Hadoop map task. The input to these Hadoop jobs are file lists, and we actually want a very small max-split-size, so each map task only gets few files to process.
2014-02-26 14:16:10.840 UTC
256
2014-02-26 14:17:05.227 UTC
remove_wav_files_really_remove00xcorrSound_waveform__GetResultsFromHadoopJob_STDERRxcorrSound_waveform__GetResultsFromHadoopJob_STDOUTxcorrSound_waveform__HadoopJob_STDERRxcorrSound_waveform__HadoopJob_STDOUTWriteFilePairListToHDFS_STDERRWriteFilePairListToHDFS_STDOUTremove_wav_files_successremove_wav_files_2_successFfmpegMigrate_Tavernnfs_output_path0mp3_list_on_hdfs_input_path0hdfs_output_path_20mapreduce_output_path0jar_input_path0max_split_size0GetResultsFromHadoopJob_STDOUT00net.sf.taverna.t2.activitiesdataflow-activity1.4net.sf.taverna.t2.activities.dataflow.DataflowActivitynet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeMpg321Convert_Tavernhdfs_output_path_20nfs_output_path0mp3_list_on_hdfs_input_path0mapreduce_output_path0jar_input_path0max_split_size0GetResultsFromHadoopJob_STDOUT00net.sf.taverna.t2.activitiesdataflow-activity1.4net.sf.taverna.t2.activities.dataflow.DataflowActivitynet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeMakeWavFilePairsListffmpegMigratedWavPaths1mpg321ConvertedWavPaths1wavFilePathPairs11net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity
ffmpegMigratedWavPaths
1
text/plain
java.lang.String
true
mpg321ConvertedWavPaths
1
text/plain
java.lang.String
true
wavFilePathPairs
1
1
workflow
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeOutputDirFfmpegJoboutputdir0ffmpegHadoopJobOutputDir00net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity
outputdir
0
text/plain
java.lang.String
true
ffmpegHadoopJobOutputDir
0
0
workflow
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeOutputDirMpg321Joboutputdir0mpg321HadoopJobOutputDir00net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity
outputdir
0
text/plain
java.lang.String
true
mpg321HadoopJobOutputDir
0
0
workflow
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeSplit_string_into_string_list_by_regular_expressionregex0string0split11net.sf.taverna.t2.activitieslocalworker-activity1.4net.sf.taverna.t2.activities.localworker.LocalworkerActivity
string
0
'text/plain'
java.lang.String
true
regex
0
'text/plain'
java.lang.String
true
split
1
l('text/plain')
1
workflow
org.embl.ebi.escience.scuflworkers.java.SplitByRegex
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeFormatterlines1formatted11net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity
lines
1
text/plain
java.lang.String
true
formatted
1
1
workflow
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeFormatter_2lines1formatted11net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity
lines
1
text/plain
java.lang.String
true
formatted
1
1
workflow
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Invokeregex_valuevalue00net.sf.taverna.t2.activitiesstringconstant-activity1.4net.sf.taverna.t2.activities.stringconstant.StringConstantActivity
\n
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeSplit_string_into_string_list_by_regular_expression_2string0regex0split11net.sf.taverna.t2.activitieslocalworker-activity1.4net.sf.taverna.t2.activities.localworker.LocalworkerActivity
string
0
'text/plain'
java.lang.String
true
regex
0
'text/plain'
java.lang.String
true
split
1
l('text/plain')
1
workflow
org.embl.ebi.escience.scuflworkers.java.SplitByRegex
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeMerge_String_List_to_a_Stringstringlist1concatenated00net.sf.taverna.t2.activitieslocalworker-activity1.4net.sf.taverna.t2.activities.localworker.LocalworkerActivity
stringlist
1
l('text/plain')
java.lang.String
true
seperator
0
'text/plain'
java.lang.String
true
concatenated
0
'text/plain'
0
workflow
org.embl.ebi.escience.scuflworkers.java.StringListMerge
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeWriteFilePairListToHDFSmapreduce_output_path0file_pair_list_on_nfs0STDERR00STDOUT00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity
789663B8-DA91-428A-9F7D-B3F3DA185FD4
default local
<?xml version="1.0" encoding="UTF-8"?>
<localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation>
9523bce2-86c7-4caf-a720-709f6ccd877d
# Write FilePairList to HDFS
hadoop fs -mkdir %%mapreduce_output_path%%;
hadoop fs -put %%file_pair_list_on_nfs%% %%mapreduce_output_path%%/
1200
1800
file_pair_list_on_nfs
mapreduce_output_path
mapreduce_output_path
mapreduce_output_path
false
false
false
UTF-8
false
false
false
file_pair_list_on_nfs
file_pair_list_on_nfs
false
false
false
UTF-8
false
false
false
false
true
true
0
false
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeWrite_Text_Filefilecontents0encoding0outputFile0outputFile00net.sf.taverna.t2.activitieslocalworker-activity1.4net.sf.taverna.t2.activities.localworker.LocalworkerActivity
outputFile
0
'text/plain'
java.lang.String
true
filecontents
0
'text/plain'
java.lang.String
true
encoding
0
'text/plain'
java.lang.String
true
outputFile
0
'text/plain'
0
workflow
net.sourceforge.taverna.scuflworkers.io.TextFileWriter
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Invokeutf8value00net.sf.taverna.t2.activitiesstringconstant-activity1.4net.sf.taverna.t2.activities.stringconstant.StringConstantActivity
utf8
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeWavFilePairListFullPathOnNFSstring20string10output00net.sf.taverna.t2.activitieslocalworker-activity1.4net.sf.taverna.t2.activities.localworker.LocalworkerActivity
string1
0
'text/plain'
java.lang.String
true
string2
0
'text/plain'
java.lang.String
true
output
0
0
workflow
org.embl.ebi.escience.scuflworkers.java.StringConcat
UserNameHere
2014-02-17 10:06:21.470 UTC
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokewavFilePairsList.txtvalue00net.sf.taverna.t2.activitiesstringconstant-activity1.4net.sf.taverna.t2.activities.stringconstant.StringConstantActivity
wavFilePairsList.txt
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeOutputDirWaveformCompareJoboutputdir0waveformcompareHadoopJobOutputDir00net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity
outputdir
0
text/plain
java.lang.String
true
waveformcompareHadoopJobOutputDir
0
0
workflow
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokexcorrSound_waveform_wav_file_pairs_list_on_hdfs_input_path0nfs_output_path0mapreduce_output_path0hdfs_output_path_20jar_input_path0max_split_size0GetResultsFromHadoopJob_STDERR00GetResultsFromHadoopJob_STDOUT00HadoopJob_STDERR00HadoopJob_STDOUT00net.sf.taverna.t2.activitiesdataflow-activity1.4net.sf.taverna.t2.activities.dataflow.DataflowActivitynet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeWavFilePairListFullPathOnHDFSstring20string10output00net.sf.taverna.t2.activitieslocalworker-activity1.4net.sf.taverna.t2.activities.localworker.LocalworkerActivity
string1
0
'text/plain'
java.lang.String
true
string2
0
'text/plain'
java.lang.String
true
output
0
0
workflow
org.embl.ebi.escience.scuflworkers.java.StringConcat
UserNameHere
2014-02-17 10:06:21.470 UTC
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Invokeremove_wav_filesfile_list1really_remove0success00net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity
file_list
1
text/plain
java.lang.String
true
really_remove
0
text/plain
java.lang.String
true
success
0
0
workflow
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Invokeremove_wav_files_2file_list1really_remove0success00net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity
file_list
1
text/plain
java.lang.String
true
really_remove
0
text/plain
java.lang.String
true
success
0
0
workflow
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeReally_really_removereally_remove0really_really_remove00net.sf.taverna.t2.activitiesbeanshell-activity1.4net.sf.taverna.t2.activities.beanshell.BeanshellActivity
really_remove
0
text/plain
java.lang.String
true
really_really_remove
0
0
workflow
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeFfmpegMigrate_Tavernnfs_output_pathFfmpegMigrate_Tavernmp3_list_on_hdfs_input_pathFfmpegMigrate_Tavernhdfs_output_path_2FfmpegMigrate_Tavernmapreduce_output_pathFfmpegMigrate_Tavernjar_input_pathFfmpegMigrate_Tavernmax_split_sizeMpg321Convert_Tavernhdfs_output_path_2Mpg321Convert_Tavernnfs_output_pathMpg321Convert_Tavernmp3_list_on_hdfs_input_pathMpg321Convert_Tavernmapreduce_output_pathMpg321Convert_Tavernjar_input_pathMpg321Convert_Tavernmax_split_sizeMakeWavFilePairsListffmpegMigratedWavPathsMakeWavFilePairsListmpg321ConvertedWavPathsOutputDirFfmpegJoboutputdirOutputDirMpg321JoboutputdirSplit_string_into_string_list_by_regular_expressionregexSplit_string_into_string_list_by_regular_expressionstringFormatterlinesFormatter_2linesSplit_string_into_string_list_by_regular_expression_2stringSplit_string_into_string_list_by_regular_expression_2regexMerge_String_List_to_a_StringstringlistWriteFilePairListToHDFSmapreduce_output_pathWriteFilePairListToHDFSfile_pair_list_on_nfsWrite_Text_FilefilecontentsWrite_Text_FileencodingWrite_Text_FileoutputFileWavFilePairListFullPathOnNFSstring2WavFilePairListFullPathOnNFSstring1OutputDirWaveformCompareJoboutputdirxcorrSound_waveform_wav_file_pairs_list_on_hdfs_input_pathxcorrSound_waveform_nfs_output_pathxcorrSound_waveform_mapreduce_output_pathxcorrSound_waveform_hdfs_output_path_2xcorrSound_waveform_jar_input_pathxcorrSound_waveform_max_split_sizeWavFilePairListFullPathOnHDFSstring2WavFilePairListFullPathOnHDFSstring1remove_wav_filesfile_listremove_wav_filesreally_removeremove_wav_files_2file_listremove_wav_files_2really_removeReally_really_removereally_removexcorrSound_waveform__GetResultsFromHadoopJob_STDERRxcorrSound_waveform__GetResultsFromHadoopJob_STDOUTxcorrSound_waveform__HadoopJob_STDERRxcorrSound_waveform__HadoopJob_STDOUTWriteFilePairListToHDFS_STDERRWriteFilePairListToHDFS_STDOUTremove_wav_files_successremove_wav_files_2_success
0caeed66-b873-464d-9ce2-3213969627e9
2014-02-16 18:54:51.566 UTC
9a045cde-7ab7-4fd5-8d73-4437e592630f
2014-02-26 14:17:16.314 UTC
3365051c-85dc-407f-ab95-46b8537104dc
2014-02-26 10:07:10.538 UTC
This workflow migrates an input list (available on HDFS) of mp3 files (available on NFS) to wav files (in output directory on NFS) using an ffmpeg Hadoop job. The workflow then compares content of the original mp3 and the migrated wav by first converting the two files to wav using an mpg123 Hadoop job and the identity function respectively, and then using an xcorrSound waveform-compare Hadoop job.
The needed Hadoop jobs are available from https://github.com/statsbiblioteket/scape-audio-qa-experiments
2014-06-30 09:13:58.378 UTC
27e4ff4b-82a0-404f-b753-593b5308f0c9
2014-06-30 09:13:35.489 UTC
c96e41f1-f42d-468d-b52b-9e4a0985ba64
2014-05-02 08:36:35.55 UTC
9a413a7f-132b-4c3c-907a-6b03ba62e307
2014-02-17 10:57:56.887 UTC
252983cc-784d-49a9-b01f-cce3fd6a9c78
2014-05-05 07:47:35.458 UTC
d4921c6e-ce9f-40b2-bb73-920796fd520d
2014-02-13 13:09:59.190 UTC
cd485079-dc72-4ff6-b83d-10c4ed874268
2014-02-26 10:03:17.305 UTC
571b5955-f50b-4023-a631-a9ae30fd8e37
2014-02-26 10:03:11.355 UTC
371f1f72-0284-4172-a890-84a48953a9a6
2014-02-16 18:53:05.825 UTC
103b3005-b5b3-4be9-a16b-8c0dc33a44a0
2014-02-16 19:27:42.513 UTC
64b48150-91d7-4bc1-a4f8-57629a2d104c
2014-02-17 11:07:16.308 UTC
2cbe6947-ee2e-49ad-90b4-ec73cdf97100
2014-02-16 19:30:20.245 UTC
0c9b11e3-e4ca-4a7d-b14d-d4e5cbbbf6da
2014-02-17 10:19:56.227 UTC
fba56f43-ae59-4ab0-bb53-96d440656aa3
2014-02-13 13:08:59.949 UTC
73fe964f-b4eb-4754-b0f0-5162604773d8
2014-03-18 07:54:36.272 UTC
45f29765-faca-4f3b-bc20-06fe4bb28d6e
2014-02-14 18:57:27.188 UTC
e8cc6437-a0a1-444c-94e6-1c3db9ad4847
2014-02-17 09:34:23.577 UTC
98f6eb8c-384e-4f13-9560-822077e714fb
2014-02-13 20:48:34.593 UTC
4e72ff15-62e8-4288-9890-385fc226fed1
2014-02-26 09:59:43.201 UTC
5725cda0-45dd-4d55-90b8-730d1a5017f4
2014-05-02 09:00:39.755 UTC
d61b63e1-ffb1-4626-a236-81f8f032119a
2014-02-26 14:17:01.797 UTC
921fd4bd-d9b4-4c0f-9eb5-2a2b1ad53a1b
2014-02-16 19:12:41.848 UTC
73836ffb-1126-4cac-b0a6-e2bc2852c7b6
2014-02-15 07:53:51.249 UTC
0d5751ff-9d71-4b98-9d49-d64175bef90a
2014-02-13 13:14:10.852 UTC
a2cd843f-4333-4bd4-b8f2-3f9779bf5720
2014-02-13 13:15:59.352 UTC
e48412d7-902a-44a8-bebd-09b9d0a2d589
2014-02-13 13:26:11.975 UTC
7d3220af-b97f-464e-947d-07c78e254e35
2014-06-30 09:21:00.413 UTC
c323dc39-e523-4f39-95af-e2c83a160347
2014-02-15 07:50:03.286 UTC
06cb246f-ba63-4250-af6b-9fc73841438f
2014-02-16 19:09:24.815 UTC
1e8d7bab-8999-4a84-ae6f-358edc0e8dac
2014-02-14 12:25:21.932 UTC
cb085352-ad80-4726-a4bc-86c8850af4ab
2014-06-23 08:17:58.553 UTC
51fbbf97-f74c-4631-8457-6e67d85de442
2014-02-14 18:58:03.44 UTC
9ebbac67-c787-43f3-a529-5b321dec0cb6
2014-02-14 11:42:19.49 UTC
ec222728-c58a-45d9-93a3-170443dea987
2014-02-17 10:47:02.594 UTC
Slim Migrate And QA mp3 to Wav Using Hadoop Jobs.
2014-02-13 13:09:30.476 UTC
c7d4b459-77e7-42c5-8c9d-ff83e2d22a46
2014-02-17 10:08:54.599 UTC
f2a09fac-2aa2-477a-81d2-789674641b3c
2014-02-17 10:28:42.783 UTC
de11c668-240b-4244-a35a-be596b09804d
2014-02-17 10:41:25.125 UTC
cd4670f1-aaf3-4311-a539-41683050f181
2014-02-16 18:53:19.461 UTC
70f547a3-5fd4-4b3b-ac10-7c3445475b76
2014-04-08 07:47:05.137 UTC
a84423d2-144a-4c61-8639-f473826be3da
2014-03-17 08:53:10.897 UTC
62df025e-55c3-4657-be4b-867e67c463bc
2014-03-17 11:43:29.195 UTC
92c9a0e2-b0b8-4de5-82f8-f05e142f7ac9
2014-03-19 09:38:19.864 UTC
f7cdbaad-60f9-4c4a-a71d-0fc8344e7f23
2014-02-26 14:20:04.140 UTC
83b6603a-0f4d-4b33-99c8-99036af8336f
2014-02-14 19:25:25.29 UTC
dc25ff57-0f95-4db2-a81e-d22df7f9e3cc
2014-03-17 11:44:26.604 UTC
8a79036c-0688-48a6-8772-74aa1058b08e
2014-02-16 19:20:19.21 UTC
cfc2d52f-4395-4a8d-9d67-13f7cc26f6bc
2014-02-17 10:34:13.568 UTC
9a937735-2940-4ca6-983c-55cee03f8e7c
2014-02-17 10:37:51.543 UTC
Bolette A. Jurik, Statsbiblioteket & SCAPE
2014-02-13 13:10:00.54 UTC
43c70b3f-ead8-48ef-a9c2-c89d12b89d31
2014-02-14 18:49:24.888 UTC
4d52b623-e925-424a-9522-7b9dd44ddce9
2014-02-17 11:10:42.102 UTC
e592ba05-3c7a-49d4-8590-3b7fd44ea489
2014-02-26 14:22:43.551 UTC
e3d47c73-e233-4f12-a3db-70d89f40fcb7
2014-02-15 10:29:25.957 UTC
bcfe7d75-096b-4759-beca-fd2d9bb2e5d3
2014-02-13 13:11:49.886 UTC
FfmpegMigrate_Tavernmp3_list_on_hdfs_input_path00
path to input file on hdfs containing list of paths to mp3 files on nfs to be migrated
2014-02-16 18:53:19.779 UTC
input/mp3/filelist.txt
2014-01-14 10:45:01.105 UTC
mapreduce_output_path00
output/test2014-009
2014-01-14 10:44:42.669 UTC
output directory for Taverna output
2014-01-14 10:44:26.511 UTC
hdfs_output_path_200
Output directory for preservation event files and other log files.
2014-01-30 15:13:40.77 UTC
output/test-output/MigrateMp3ToWav/
2014-01-30 15:14:15.437 UTC
nfs_output_path00
Output directory fora the migrated wav files on nfs.
2014-02-16 18:54:13.255 UTC
/home/bolette/TestOutput/
2014-01-30 15:15:12.216 UTC
jar_input_path00max_split_size00HadoopJob_STDOUTHadoopJob_STDERRGetResultsFromHadoopJob_STDERRGetResultsFromHadoopJob_STDOUTFfmpegMigrateHadoopJobhdfs_input_path0mapreduce_output_path0hdfs_output_path_20nfs_output_path0jar_input_path0max_split_size0STDOUT00STDERR00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity
789663B8-DA91-428A-9F7D-B3F3DA185FD4
default local
<?xml version="1.0" encoding="UTF-8"?>
<localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation>
9523bce2-86c7-4caf-a720-709f6ccd877d
# Configure
migrate_mp3_to_wav_hadoop_JAR_PATH=%%jar_input_path%%/migrate_mp3_to_wav_hadoop-0.1-SNAPSHOT-jar-with-dependencies.jar
# Hadoop job
hadoop jar ${migrate_mp3_to_wav_hadoop_JAR_PATH} eu.scape_project.audio_qa.ffmpeg_migrate.FfmpegMigrate -Dmapred.max.split.size=%%max_split_size%% %%hdfs_input_path%% %%mapreduce_output_path%% %%hdfs_output_path_2%% %%nfs_output_path%%
1200
1800
hdfs_input_path
hdfs_output_path_2
jar_input_path
mapreduce_output_path
max_split_size
nfs_output_path
max_split_size
max_split_size
false
false
false
UTF-8
false
false
false
hdfs_input_path
hdfs_input_path
false
false
false
UTF-8
false
false
false
nfs_output_path
nfs_output_path
false
false
false
UTF-8
false
false
false
mapreduce_output_path
mapreduce_output_path
false
false
false
UTF-8
false
false
false
hdfs_output_path_2
hdfs_output_path_2
false
false
false
UTF-8
false
false
false
jar_input_path
jar_input_path
false
false
false
UTF-8
false
false
false
false
true
true
0
false
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeGetResultsFromHadoopJobmapreduce_output_path0STDERR00STDOUT00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity
789663B8-DA91-428A-9F7D-B3F3DA185FD4
default local
<?xml version="1.0" encoding="UTF-8"?>
<localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation>
9523bce2-86c7-4caf-a720-709f6ccd877d
# Read HDFS Hadoop job output
hadoop fs -cat %%mapreduce_output_path%%/part-r-00000
1200
1800
mapreduce_output_path
mapreduce_output_path
mapreduce_output_path
false
false
false
UTF-8
false
false
false
false
true
true
0
false
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeFfmpegMigrateHadoopJobhdfs_input_pathFfmpegMigrateHadoopJobmapreduce_output_pathFfmpegMigrateHadoopJobhdfs_output_path_2FfmpegMigrateHadoopJobnfs_output_pathFfmpegMigrateHadoopJobjar_input_pathFfmpegMigrateHadoopJobmax_split_sizeGetResultsFromHadoopJobmapreduce_output_pathHadoopJob_STDOUTHadoopJob_STDERRGetResultsFromHadoopJob_STDERRGetResultsFromHadoopJob_STDOUT
Bolette A. Jurik, Statsbiblioteket & SCAPE
2014-01-14 10:46:02.398 UTC
31153f8c-ce5e-44ed-bb7f-1ea6c1f59cb1
2014-02-13 13:07:40.427 UTC
014fa84d-e620-47de-9f8b-fc5ec16259a5
2014-02-26 09:58:04.134 UTC
b3b071bb-6a05-4e22-804d-3ee6b160cfa6
2014-01-30 15:15:18.104 UTC
1caf69fe-5742-40a7-b776-e053f59e29e1
2014-02-16 18:53:05.626 UTC
92bb3da2-f8f4-446d-9a52-5b33c113340e
2014-02-13 10:47:40.857 UTC
04aab78f-3312-48aa-b5b3-5ade853dcbc2
2014-02-16 18:54:51.313 UTC
18331161-9134-4992-826d-0c61a145fa2a
2014-02-13 10:36:45.35 UTC
a6a4362d-ffef-429c-b1f4-517c937cbc3a
2014-01-30 15:15:10.184 UTC
1ef352ba-0251-4ab0-8d2e-901142ac08f3
2014-01-14 10:21:04.29 UTC
33b9466a-2cdb-42b5-b9a8-8a9e4b49ed1c
2014-01-31 07:56:06.362 UTC
0b665efc-5f67-47c9-b9de-42f47fe204f2
2014-02-26 14:12:44.398 UTC
dcb03855-b944-46cb-9991-5445b4414d7f
2014-02-13 13:07:08.919 UTC
3629a1cb-67ca-4ccf-b3ae-960c6f29543d
2014-01-30 15:19:13.424 UTC
20ff7e22-21a2-4b68-8cb0-aee205216e91
2014-02-16 18:53:19.287 UTC
75d877e4-d973-43fd-aee8-013f9724f918
2014-01-14 10:47:27.703 UTC
3b0c8bd3-a022-4e1e-9f96-d1456a9c9921
2014-02-11 13:42:20.515 UTC
FfmpegMigrate Taverna Workflow using FfmpegMigrate Hadoob Job to migrate a list of mp3 files to wav files.
2014-02-13 13:07:11.522 UTC
xcorrSound_waveform_wav_file_pairs_list_on_hdfs_input_path00
input/wav_file_pairs.txt
2014-02-13 12:51:22.428 UTC
path to input file on hdfs containing list of pairs of paths to wav files on nfs to be compared
2014-02-13 12:49:29.669 UTC
mapreduce_output_path00
output/test2014-009
2014-01-14 10:44:42.669 UTC
output directory for Taverna output
2014-01-14 10:44:26.511 UTC
hdfs_output_path_200
Output directory for preservation event files and other log files.
2014-01-30 15:13:40.77 UTC
output/test-output/MigrateMp3ToWav/
2014-01-30 15:14:15.437 UTC
nfs_output_path00
/home/bolette/TestOutput/
2014-01-30 15:15:12.216 UTC
Output directory for the migrated wav files on nfs.
2014-01-30 15:14:46.670 UTC
jar_input_path00max_split_size00HadoopJob_STDOUTHadoopJob_STDERRGetResultsFromHadoopJob_STDERRGetResultsFromHadoopJob_STDOUTWavefileCompareHadoopJobhdfs_input_path0mapreduce_output_path0hdfs_output_path_20nfs_output_path0jar_input_path0max_split_size0STDOUT00STDERR00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity
789663B8-DA91-428A-9F7D-B3F3DA185FD4
default local
<?xml version="1.0" encoding="UTF-8"?>
<localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation>
9523bce2-86c7-4caf-a720-709f6ccd877d
# Configure
migrate_mp3_to_wav_hadoop_JAR_PATH=%%jar_input_path%%/migrate_mp3_to_wav_hadoop-0.1-SNAPSHOT-jar-with-dependencies.jar
# Hadoop job
hadoop jar ${migrate_mp3_to_wav_hadoop_JAR_PATH} eu.scape_project.audio_qa.waveform_compare.WaveformCompare -Dmapred.max.split.size=%%max_split_size%% %%hdfs_input_path%% %%mapreduce_output_path%% %%hdfs_output_path_2%% %%nfs_output_path%%
1200
1800
hdfs_input_path
hdfs_output_path_2
jar_input_path
mapreduce_output_path
max_split_size
nfs_output_path
max_split_size
max_split_size
false
false
false
UTF-8
false
false
false
hdfs_input_path
hdfs_input_path
false
false
false
UTF-8
false
false
false
nfs_output_path
nfs_output_path
false
false
false
UTF-8
false
false
false
mapreduce_output_path
mapreduce_output_path
false
false
false
UTF-8
false
false
false
hdfs_output_path_2
hdfs_output_path_2
false
false
false
UTF-8
false
false
false
jar_input_path
jar_input_path
false
false
false
UTF-8
false
false
false
false
true
true
0
false
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeGetResultsFromHadoopJobmapreduce_output_path0STDERR00STDOUT00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity
789663B8-DA91-428A-9F7D-B3F3DA185FD4
default local
<?xml version="1.0" encoding="UTF-8"?>
<localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation>
9523bce2-86c7-4caf-a720-709f6ccd877d
# Read HDFS Hadoop job output
hadoop fs -cat %%mapreduce_output_path%%/part-r-00000
1200
1800
mapreduce_output_path
mapreduce_output_path
mapreduce_output_path
false
false
false
UTF-8
false
false
false
false
true
true
0
false
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeWavefileCompareHadoopJobhdfs_input_pathWavefileCompareHadoopJobmapreduce_output_pathWavefileCompareHadoopJobhdfs_output_path_2WavefileCompareHadoopJobnfs_output_pathWavefileCompareHadoopJobjar_input_pathWavefileCompareHadoopJobmax_split_sizeGetResultsFromHadoopJobmapreduce_output_pathHadoopJob_STDOUTHadoopJob_STDERRGetResultsFromHadoopJob_STDERRGetResultsFromHadoopJob_STDOUT
33b9466a-2cdb-42b5-b9a8-8a9e4b49ed1c
2014-01-31 07:56:06.362 UTC
6755a2ed-2aa1-485a-9e11-aa6170587ee5
2014-02-13 12:42:07.29 UTC
3629a1cb-67ca-4ccf-b3ae-960c6f29543d
2014-01-30 15:19:13.424 UTC
75d877e4-d973-43fd-aee8-013f9724f918
2014-01-14 10:47:27.703 UTC
b3b071bb-6a05-4e22-804d-3ee6b160cfa6
2014-01-30 15:15:18.104 UTC
Bolette A. Jurik, Statsbiblioteket & SCAPE
2014-01-14 10:46:02.398 UTC
58181448-3893-4460-934b-7c04b5557eaf
2014-02-13 12:51:16.574 UTC
a6a4362d-ffef-429c-b1f4-517c937cbc3a
2014-01-30 15:15:10.184 UTC
064f3c60-536e-4d12-b4bd-e234d7ef2c75
2014-02-26 10:06:44.556 UTC
92bb3da2-f8f4-446d-9a52-5b33c113340e
2014-02-13 10:47:40.857 UTC
3b0c8bd3-a022-4e1e-9f96-d1456a9c9921
2014-02-11 13:42:20.515 UTC
18331161-9134-4992-826d-0c61a145fa2a
2014-02-13 10:36:45.35 UTC
1ef352ba-0251-4ab0-8d2e-901142ac08f3
2014-01-14 10:21:04.29 UTC
ebd85210-f89f-4efd-99c9-a6213e1a8545
2014-02-26 14:22:06.102 UTC
xcorrSound waveform-compare Taverna Workflow using WaveformCompare Hadoob Job to compare a list of pairs of wav files.
2014-02-13 12:48:13.480 UTC
b5bc31e6-dfce-4d16-8092-0f10cc66720f
2014-02-13 12:53:09.539 UTC
Mpg321Convert_Tavernmp3_list_on_hdfs_input_path00
input/mp3/filelist.txt
2014-01-14 10:45:01.105 UTC
path to input file on hdfs containing list of paths to mp3 files on nfs to be migrated
2014-01-30 15:12:35.851 UTC
mapreduce_output_path00
output/test2014-009
2014-01-14 10:44:42.669 UTC
output directory for Taverna output
2014-01-14 10:44:26.511 UTC
hdfs_output_path_200
output/test-output/MigrateMp3ToWav/
2014-01-30 15:14:15.437 UTC
Output directory for preservation event files and other log files.
2014-01-30 15:13:40.77 UTC
nfs_output_path00
/home/bolette/TestOutput/
2014-01-30 15:15:12.216 UTC
Output directory for the migrated wav files on nfs.
2014-01-30 15:14:46.670 UTC
jar_input_path00max_split_size00HadoopJob_STDOUTHadoopJob_STDERRGetResultsFromHadoopJob_STDERRGetResultsFromHadoopJob_STDOUTMpg321ConvertHadoopJobhdfs_input_path0mapreduce_output_path0hdfs_output_path_20nfs_output_path0jar_input_path0max_split_size0STDOUT00STDERR00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity
789663B8-DA91-428A-9F7D-B3F3DA185FD4
default local
<?xml version="1.0" encoding="UTF-8"?>
<localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation>
9523bce2-86c7-4caf-a720-709f6ccd877d
# Configure
migrate_mp3_to_wav_hadoop_JAR_PATH=%%jar_input_path%%/migrate_mp3_to_wav_hadoop-0.1-SNAPSHOT-jar-with-dependencies.jar
# Hadoop job
hadoop jar ${migrate_mp3_to_wav_hadoop_JAR_PATH} eu.scape_project.audio_qa.mpg321_convert.Mpg321Convert -Dmapred.max.split.size=%%max_split_size%% %%hdfs_input_path%% %%mapreduce_output_path%% %%hdfs_output_path_2%% %%nfs_output_path%%
1200
1800
hdfs_input_path
hdfs_output_path_2
jar_input_path
mapreduce_output_path
max_split_size
nfs_output_path
max_split_size
max_split_size
false
false
false
UTF-8
false
false
false
hdfs_input_path
hdfs_input_path
false
false
false
UTF-8
false
false
false
nfs_output_path
nfs_output_path
false
false
false
UTF-8
false
false
false
mapreduce_output_path
mapreduce_output_path
false
false
false
UTF-8
false
false
false
hdfs_output_path_2
hdfs_output_path_2
false
false
false
UTF-8
false
false
false
jar_input_path
jar_input_path
false
false
false
UTF-8
false
false
false
false
true
true
0
false
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeGetResultsFromHadoopJobmapreduce_output_path0STDERR00STDOUT00net.sf.taverna.t2.activitiesexternal-tool-activity1.4net.sf.taverna.t2.activities.externaltool.ExternalToolActivity
789663B8-DA91-428A-9F7D-B3F3DA185FD4
default local
<?xml version="1.0" encoding="UTF-8"?>
<localInvocation><shellPrefix>/bin/sh -c</shellPrefix><linkCommand>/bin/ln -s %%PATH_TO_ORIGINAL%% %%TARGET_NAME%%</linkCommand></localInvocation>
9523bce2-86c7-4caf-a720-709f6ccd877d
# Read HDFS Hadoop job output
hadoop fs -cat %%mapreduce_output_path%%/part-r-00000
1200
1800
mapreduce_output_path
mapreduce_output_path
mapreduce_output_path
false
false
false
UTF-8
false
false
false
false
true
true
0
false
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Parallelize
1
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.ErrorBouncenet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Failovernet.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.Retry
1.0
1000
5000
0
net.sf.taverna.t2.coreworkflowmodel-impl1.4net.sf.taverna.t2.workflowmodel.processor.dispatch.layers.InvokeMpg321ConvertHadoopJobhdfs_input_pathMpg321ConvertHadoopJobmapreduce_output_pathMpg321ConvertHadoopJobhdfs_output_path_2Mpg321ConvertHadoopJobnfs_output_pathMpg321ConvertHadoopJobjar_input_pathMpg321ConvertHadoopJobmax_split_sizeGetResultsFromHadoopJobmapreduce_output_pathHadoopJob_STDOUTHadoopJob_STDERRGetResultsFromHadoopJob_STDERRGetResultsFromHadoopJob_STDOUT
75d877e4-d973-43fd-aee8-013f9724f918
2014-01-14 10:47:27.703 UTC
3b0c8bd3-a022-4e1e-9f96-d1456a9c9921
2014-02-11 13:42:20.515 UTC
18331161-9134-4992-826d-0c61a145fa2a
2014-02-13 10:36:45.35 UTC
3629a1cb-67ca-4ccf-b3ae-960c6f29543d
2014-01-30 15:19:13.424 UTC
6c962801-9103-47c0-b495-bbb0cbfbfded
2014-02-26 10:02:21.322 UTC
b3b071bb-6a05-4e22-804d-3ee6b160cfa6
2014-01-30 15:15:18.104 UTC
a6a4362d-ffef-429c-b1f4-517c937cbc3a
2014-01-30 15:15:10.184 UTC
Mpg321Convert Taverna Workflow using Mpg321Convert Hadoob Job to convert a list of mp3 files to wav files.
2014-02-13 12:38:02.24 UTC
85f11a4f-95f6-47db-ac89-118919873378
2014-02-26 14:19:18.787 UTC
33b9466a-2cdb-42b5-b9a8-8a9e4b49ed1c
2014-01-31 07:56:06.362 UTC
92bb3da2-f8f4-446d-9a52-5b33c113340e
2014-02-13 10:47:40.857 UTC
6755a2ed-2aa1-485a-9e11-aa6170587ee5
2014-02-13 12:42:07.29 UTC
1ef352ba-0251-4ab0-8d2e-901142ac08f3
2014-01-14 10:21:04.29 UTC
Bolette A. Jurik, Statsbiblioteket & SCAPE
2014-01-14 10:46:02.398 UTC