Stream - Messaging System

> (Data|State) Management and Processing > (Data Processing|Data Integration) > (Stream|Pipe|Message Queue|Event Processing)

1 - About

Messaging systems implements near-realtime asynchronous computation. A messaging system is a fairly low-level piece of infrastructure—it stores messages and waits for consumers to consume them.

Messages can be added to:

when something happens.

Downstream consumers read messages from these systems, and process them or take actions based on the message contents.

Can birds deliver messages ?


3 - Example

Suppose you have a website, and every time someone loads a page, you send a “user viewed page” event to a messaging system. You might then have consumers which do any of the following:

  • Store the message in Hadoop for future analysis
  • Count page views and update a dashboard
  • Trigger an alert if a page view fails
  • Send an email notification to another user
  • Join the page view event with the user’s profile, and send the message back to the messaging system

A messaging system lets you decouple all of this work from the actual web page serving.

4 - Documentation / Reference