Staged iteration strategy

Created: 2008-05-07 12:10:27      Last updated: 2008-05-07 12:32:36

Consider two lists A and B, of equal size 3. A[1] corresponds to B[1], A[2] to B[2], etc, for instance A are image scans from 2007 and B from 2008, and the index indicates the patient number.

A = [a0, a1, a2]
B = [b0, b1, b2]

There's then two lists of possible parameters P and Q, of different lengths, P has 2 and Q has 4 items.

P = [p0, p1]
Q = [q0, q1, q2, q3]

Each of the A items should be processed in ap using each of the P parameters, and each of the B items processed in bq using each of the Q parameters.

The problem then is how to compare AnPp against the all BnQq - but notice that An and Bn have to match. The normal crossproduct would compare all AaPp against all BbQq - but we want to restrict the iteration strategy. We can't use the dot product directly because for a given patient n we want to compare all Ps against all Qs using the cross product.

This is solved in t2 using staged iteration, but here is a hack showing how this can be achieved in t1 using a nested workflow. The nested workflow basically "stops" the iteration at list level, due to the echo_lists inside that makes the nested workflow excpect lists.

There's explicit crossproduct(p,a) strategy set on ap and crossproduct(q,b) on bq to make sure they output with the a/b iteration at the highest level lists, ie. ap outputs:

[  [ a0p0, a0p1], 
   [ a1p0, a1p1],
   [ a2p0, a2p1]

So the top level lists from ap corresponds to each item of A. The same trick applies to bq - if we don't specify this the implicit iteration might output the opposite with the highest list corresponding to the Ps and Qs. (Hint: Simply drag p or a within the iteration strategy editor to change the order)

The second part is to use a nested workflow that takes the outputs from ap and bq, but through the Echo list local worker. This worker doesn't do anything except it forces the nested workflow to take a list of items as inputs instead of single inputs.

Hence we can set a dotproduct(pq, ap) on the processor for the nested workflow apbq_iter - since this nested workflow consumes lists at both ports this means it will be iterated over with these inputs:

ap = [ a0p0, a0p1]    bq = [b0q0, b0q1, b0q2, b0q3]
ap = [ a1p0, a1p1]    bq = [b1q0, b1q1, b1q2, b1q3]
ap = [ a2p0, a2p1]    bq = [b2q0, b2q1, b2q2, b2q3]

Inside the nested workflow there's the normal crossproduct(pq, ap) so that it can do an all-to-all comparison.

(The beanshell inside apbq here actually only returns the string ap+bq, ap returns a+p and bq returns p+q, but assume that instead there was real services invoked for each of these processors doing some kind of filtering/comparison operation using the given parameters, and that a0p0 etc. are the outputs of those operations.)

With this hack we can run a dotproduct for the outer list, and a crossproduct for the inner list.

The output of running this workflow should be:

  [ (a0p0b0q0, a0p0b0q1, a0p0b0q2, a0p0b0q3),
    (a0p1b0q0, a0p1b0q1, a0p1b0q2, a0p1b0q3)
  [ (a1p0b1q0, a1p0b1q1, a1p0b1q2, a1p0b1q3),
    (a1p1b1q0, a1p1b1q1, a1p1b1q2, a1p1b1q3)
  [ (a2p0b2q0, a2p0b2q1, a2p0b2q2, a2p0b2q3),
    (a2p1b2q0, a2p1b2q1, a2p1b2q2, a2p1b2q3)

So in the final output {} (depth=3) there's three big [] lists of depth 2, corresponding to a0/b0, a1/b1 and b2/b2. Within each of these are two () lists of depth 1 corresponding to p0 and p1. The content of these lists (depth 0) are the actual items returned from apbq - one for each of q0,q1,q2,q3.

Information Preview

Information Run

Run this Workflow in the Taverna Workbench...

Option 1:

Copy and paste this link into File > 'Open workflow location...'
[ More InfoExpand ]

Information Workflow Components

Inputs (0)
Processors (9)
Beanshells (7)
Outputs (2)
Links (10)
Coordinations (0)

Information Workflow Type

Taverna 1

Information License

All versions of this Workflow are licensed under:

Information Version 1 (of 1)

Information Credits (1)


Information Attributions (0)



Information Tags (7)

Log in to add Tags

Information Shared with Groups (0)


Information Featured In Packs (0)


Log in to add to one of your Packs

Information Attributed By (0)



Information Favourited By (0)

No one

Information Statistics


Citations (0)


Version History

In chronological order:

Reviews Reviews (0)

No reviews yet

Be the first to review!

Comments Comments (0)

No comments yet

Log in to make a comment

Workflow Other workflows that use similar services (0)

There are no workflows in myExperiment that use similar services to this Workflow.