Glossary

data stream

collection of data records produced and consumed by data components

data source
data destination
data endpoint

where you get data from or write to: SQL database, CSV file, REST api ...

data components
data component
component

Components are the basic building blocks of Jetstream pipes. There are four types of (data) components: Input, Inspector, Transformer or Output components.

data pipe
pipes
pipe

a “master” component composed of several data components piped together and run in order by a Jetstream Streamer

streamer

Part of Jetstream framework that:

1. composes a pipe from two or more components
2. adds instrumentation in between every ‘slot’ or ‘connection’ between two components
3. runs the pipe, indexing received records, performing logging and producing a report of the run

Input

a component reading data from an data source, for example from an SQL database

Inspector

a component that analyses the data stream and performs actions depending on data content; for example a validating Introspector can raise an error upon seeing incomplete data

Transformer

a component that modifies a data stream

Output

a component writing to a data source, for example to a CSV file