paralab.fj-tasks
Module contains simple Clojure abstraction around ForkJoin library.
make-fj-task
(make-fj-task & {:keys [size-threshold size-f split-f process-f data], :as fj-task})Creates fj-task on the basis of keyword arguments: :data - data to be processed, :size-f - function returning size of data chunk, :split-f - function splitting data chunk :process-f - function processing chunk of data :size-threshold - maximum size of chunk processed by process-f :merge-f - function merging results of process-f ran on two chunks
:merge-f is optional. If you wish only to run for side effects using run-fj-task! it should not be provided.
make-fj-task-map-reduce-vec
(make-fj-task-map-reduce-vec & {:keys [map-f reduce-f data size-threshold]})Creates fj-task executing in parallel map-reduce style computation on data vector, which is equivalent to:
(reduce reduce-f (map map-f data)) Assumes that reduce-f is associative.
Uses the following keyword arguments: :data - data to be processed of type PersistentVector, :map-f - function processing chunk of data, :reduce-f - function reducing the data, :size-threshold - maximum size of chunk processed in a serial manner.
Obviously this task should be processed by run-fj-task functions without exclamation mark.
make-fj-task-vec
(make-fj-task-vec & {:keys [size-threshold process-f merge-f data]})Creates fj-task with vector data on the basis of keyword arguments: :data - data to be processed of type PersistentVector, :process-f - function processing chunk of data :size-threshold - maximum size of chunk processed by process-f :merge-f - function merging results of process-f ran on two chunks
:merge-f is optional. If you wish only to run for side effects using run-fj-task! it should not be provided.
run-fj-task!
(run-fj-task! fj-pool fj-task)Runs fj-task in a given fj-pool for side-effects only.
fj-task must not contain merge-f field. Any results returned by process-f are discarded, preferably it should return nil. Too much blocking I/O in process-f can result in suboptimal performance, since Fork/Join is designed to CPU intensive tasks.
run-fj-task-serial
(run-fj-task-serial fj-task)Runs fj-task in a serial mode. ForkJoin framework is not used.
run-fj-task-serial!
(run-fj-task-serial! fj-task)Runs fj-task in a serial mode for side effects only. ForkJoin framework is not used.