(Stream|Pipe|Message Queue)

1 - About

We should have some ways of coupling programs like garden hose–screw in another segment when it becomes when it becomes necessary to massage data in another way. This is the way of IO also. Malcolm Douglas McIlroy - Inventor of pipes and other wonders.

An Stream is a combination of:

  • an input source
  • an output destination.
  • and a sequence of data (that may be a byte of an object). In the general sense, it's called a message.
  • that then may be chained to form a pipeline

From an abstract point of view, a stream can be seen as an infinite iteration.

Stream processing lets model systems that have state without ever using assignment or mutable data.

The data structures involved in streams application are then:

  • and a queue to store messages.

Streams are a flexible system to implement a pipeline of data transformations.

An application that handle this message is called a messaging system.

In a stream architecture, the following rules: Something happened (Event), subscribe to it (Streams).

In Unix/Linux, streams are implemented by the shell with pipes.

A table is a stream with an infinite windows.

3 - Other definition

A stream is derivative of state over time. The product rule, (uv)' = u'v + uv', is analogous to the rule for joining streams. Martin Kleppmann and Julian Hyde

4 - Example

streams of data

  • user activity on a website
  • sensor readings from devices (IOT)
  • Bookmark "(Stream|Pipe|Message Queue)" at del.icio.us
  • Bookmark "(Stream|Pipe|Message Queue)" at Digg
  • Bookmark "(Stream|Pipe|Message Queue)" at Ask
  • Bookmark "(Stream|Pipe|Message Queue)" at Google
  • Bookmark "(Stream|Pipe|Message Queue)" at StumbleUpon
  • Bookmark "(Stream|Pipe|Message Queue)" at Technorati
  • Bookmark "(Stream|Pipe|Message Queue)" at Live Bookmarks
  • Bookmark "(Stream|Pipe|Message Queue)" at Yahoo! Myweb
  • Bookmark "(Stream|Pipe|Message Queue)" at Facebook
  • Bookmark "(Stream|Pipe|Message Queue)" at Yahoo! Bookmarks
  • Bookmark "(Stream|Pipe|Message Queue)" at Twitter
  • Bookmark "(Stream|Pipe|Message Queue)" at myAOL
stream/stream.txt · Last modified: 2017/06/07 20:56 by gerardnico