Home     Software     Services     About     Contact  

POSIX Pipelines (XFL)

POSIX Pipelines (XFL) is an implementation of "flow programming".
It is intended to work like CMS Pipelines as much as possible but it runs in a Unix or Unix-like environment (POSIX, including Linux).

If you are familiar with CMS Pipelines

XFL mimics CMS Pipelines as much as possible.
Unlike other CMS Pipelines work-alikes, this is a wholly POSIX package. Stages run as independent Unix processes, much like ordinary Unix commands. The initiating program 'pipe' is referred to as the "launcher" not the dispatcher. Stages are dispatched by the operating system kernel.

If you are familiar with shell pipes and Unix commands

XFL is not shell pipes.
Stages are spawned by the "launcher" with just argv[0] and argv[1]. It's not a shell environment so the usual parsing of command line arguments into argc and argv is different.

Connections between stages are not stdin and stdout. There can be any number of inputs and any number of outputs. Incoming traffic to a stage is read with either peekto() or readto(). Outbound traffic is sent downstream with output().

Data is passed from stage-to-stage as packets or records. The boundaries are out-of-band, so the data in transit remains pure. Where it matters (e.g., for text files), boundaries are interpolated as newline (NL), but within a pipeline newline is just another byte.

on GitHub

This package is available as a GitHub project:
https://github.com/trothtech/xfl/