# Staged iteration strategy

Created: 2008-05-07 12:10:27      Last updated: 2008-05-07 12:32:36

Consider two lists A and B, of equal size 3. `A[1]` corresponds to `B[1]`, `A[2]` to `B[2]`, etc, for instance A are image scans from 2007 and B from 2008, and the index indicates the patient number.

```A = [a0, a1, a2]
B = [b0, b1, b2]
```

There's then two lists of possible parameters P and Q, of different lengths, P has 2 and Q has 4 items.

```P = [p0, p1]
Q = [q0, q1, q2, q3]
```

Each of the A items should be processed in ap using each of the P parameters, and each of the B items processed in bq using each of the Q parameters.

The problem then is how to compare AnPp against the all BnQq - but notice that An and Bn have to match. The normal crossproduct would compare all AaPp against all BbQq - but we want to restrict the iteration strategy. We can't use the dot product directly because for a given patient n we want to compare all Ps against all Qs using the cross product.

This is solved in t2 using staged iteration, but here is a hack showing how this can be achieved in t1 using a nested workflow. The nested workflow basically "stops" the iteration at list level, due to the echo_lists inside that makes the nested workflow excpect lists.

There's explicit `crossproduct(p,a)` strategy set on ap and `crossproduct(q,b)` on bq to make sure they output with the a/b iteration at the highest level lists, ie. ap outputs:

```[  [ a0p0, a0p1],
[ a1p0, a1p1],
[ a2p0, a2p1]
]
```

So the top level lists from ap corresponds to each item of A. The same trick applies to bq - if we don't specify this the implicit iteration might output the opposite with the highest list corresponding to the Ps and Qs. (Hint: Simply drag p or a within the iteration strategy editor to change the order)

The second part is to use a nested workflow that takes the outputs from ap and bq, but through the Echo list local worker. This worker doesn't do anything except it forces the nested workflow to take a list of items as inputs instead of single inputs.

Hence we can set a `dotproduct(pq, ap)` on the processor for the nested workflow apbq_iter - since this nested workflow consumes lists at both ports this means it will be iterated over with these inputs:

```ap = [ a0p0, a0p1]    bq = [b0q0, b0q1, b0q2, b0q3]
ap = [ a1p0, a1p1]    bq = [b1q0, b1q1, b1q2, b1q3]
ap = [ a2p0, a2p1]    bq = [b2q0, b2q1, b2q2, b2q3]
```

Inside the nested workflow there's the normal ```crossproduct(pq, ap)``` so that it can do an all-to-all comparison.

(The beanshell inside apbq here actually only returns the string `ap+bq`, ap returns `a+p` and bq returns `p+q`, but assume that instead there was real services invoked for each of these processors doing some kind of filtering/comparison operation using the given parameters, and that `a0p0` etc. are the outputs of those operations.)

With this hack we can run a dotproduct for the outer list, and a crossproduct for the inner list.

The output of running this workflow should be:

```{
[ (a0p0b0q0, a0p0b0q1, a0p0b0q2, a0p0b0q3),
(a0p1b0q0, a0p1b0q1, a0p1b0q2, a0p1b0q3)
],
[ (a1p0b1q0, a1p0b1q1, a1p0b1q2, a1p0b1q3),
(a1p1b1q0, a1p1b1q1, a1p1b1q2, a1p1b1q3)
],
[ (a2p0b2q0, a2p0b2q1, a2p0b2q2, a2p0b2q3),
(a2p1b2q0, a2p1b2q1, a2p1b2q2, a2p1b2q3)
]
}

```

So in the final output `{}` (depth=3) there's three big `[]` lists of depth 2, corresponding to `a0/``b0`, `a1`/`b1` and `b2`/`b2`. Within each of these are two ` ()` lists of depth 1 corresponding to `p0` and `p1`. The content of these lists (depth 0) are the actual items returned from apbq - one for each of `q0,q1,q2,q3`.

### Run

#### Run this Workflow in the Taverna Workbench...

Option 1:

Copy and paste this link into File > 'Open workflow location...'

### Workflow Components

Inputs (0)
Processors (9)
Beanshells (7)
Outputs (2)
Coordinations (0)

Workflow Type

Taverna 1

All versions of this Workflow are licensed under:

Version 1 (of 1)

Credits (1)

(People/Groups)

(Workflows/Files)

None

Tags (7)

Shared with Groups (0)

None

Featured In Packs (0)

None

Attributed By (0)

(Workflows/Files)

None

Favourited By (0)

No one

Statistics

2484 viewings

[ see breakdown ]

More

Citations (0)

None

Version History

In chronological order:

No reviews yet

## Other workflows that use similar services(0)

There are no workflows in myExperiment that use similar services to this Workflow.

Funded by: BioVeL, EPSRC, JISC, Microsoft

Affiliates: myGrid, Taverna, BioCatalogue