You can do pipelining type parallelism quite easily in Erlang. Below is a shameless copy/paste from my blogpost of Jan 2008.
Also, Glasgow Parallel Haskell allows for parallel function composition, which amounts to the same thing, giving you implicit parallelisation.
You already think in terms of pipelines - how about "gzcat foo.tar.gz | tar xf -"? You may not have known it, but the shell is running the unzip and untar in parallel - the stdin read in tar just blocks until data is sent to stdout by gzcat.
Well a lot of tasks can be expressed in terms of pipelines, and if you can do that then getting some level of parallelisation is simple with David King's helper code (even across erlang nodes, ie. machines):
pipeline:run([pipeline:generator(BigList), {filter,fun some_filter/1}, {map,fun_some_map/1}, {generic,fun some_complex_function/2}, fun some_more_complicated_function/1, fun pipeline:collect/1]).So basically what he's doing here is making a list of the steps - each step being implemented in a fun that accepts as input whatever the previous step outputs (the funs can even be defined inline of course). Go check out David's blog entry for the code and more detailed explanation.
This is very much along the lines of what I am thinking about. I need to study it a bit more tomorrow after some coffee!
I accepted this answer, because it is the most elegant solution I saw and Mark pointed me to Hartmann pipelines in the above comment, which lead me to Flow-based programming.