ToMaR HDFS Input Directory Processing

Created: 2014-03-04 12:47:29      Last updated: 2014-03-11 09:45:37

This workflow allows processing an HDFS input directory using ToMaR.

The "hdfs_working_dir" input port is the HDFS input directory which containes the data to be processed by ToMaR.

The "toolspec" input port contains the toolspec XML describing operations that can be used (see "operation" input port).

The "operation" input port defines the operation to be used in the current ToMaR job execution (see "toolspec" input port, an operation port used here must be defined in the tool specification).

The "hdfs_working_dir" input port defines the directory where the outputs will be stored in a date/time-subdirectory.

For example:

tomarworkingdir/20140304130007/dataout tomarworkingdir/20140304130007/joboutput tomarworkingdir/20140304130007/tomar-controlfile.txt tomarworkingdir/20140304130007/toolspec

The "dataout" directory contains the output data of the ToMaR process. Depending on the operation used, this can be the result of a file format identification or a data migration process. The "joboutput" directory contains the Hadoop job output of the ToMaR Hadoop job. The "tomar-controlfile.txt" file is the input file for the ToMaR Hadoop job execution. The "toolspec" directory contains the tool specification file given by the "toolspec" input port.

Information Preview

Information Run

Run this Workflow in the Taverna Workbench...

Option 1:

Copy and paste this link into File > 'Open workflow location...'
http://www.myexperiment.org/workflows/4144/download?version=1
[ More InfoExpand ]


Information Workflow Components

Information Authors (1)
Information Titles (1)
Information Descriptions (1)
Information Dependencies (0)
Inputs (4)
Processors (3)
Beanshells (0)
Outputs (1)
Datalinks (8)
Coordinations (1)

Information Workflow Type

Taverna 2

Information Uploader

Information License

All versions of this Workflow are licensed under:

Information Version 1 (earliest) (of 2)

View version:

Information Credits (1)

(People/Groups)

Information Attributions (0)

(Workflows/Files)

None

Information Tags (4)

Log in to add Tags

Information Shared with Groups (1)

Information Featured In Packs (0)

None

Log in to add to one of your Packs

Information Attributed By (0)

(Workflows/Files)

None

Information Favourited By (0)

No one

Information Statistics

 

Citations (0)

None


Version History

In chronological order:



Reviews Reviews (0)

No reviews yet

Be the first to review!



Comments Comments (0)

No comments yet

Log in to make a comment




Workflow Other workflows that use similar services (0)

There are no workflows in myExperiment that use similar services to this Workflow.