Software Design - Idempotence (Idempotent)

Card Puncher Data Processing

About

Idempotence is the ability to apply multiple times an operation without changing the result beyond the initial application.

Given the same input, re-executing a task will always produce the same result and will not change other state.

Running an idempotent code (application, function,..) multiple times in a sequence should have the same effect as running it just once.

Implementation

One way to achieve idempotency is to have a check whether the desired final state has already been achieved, and if that's the case, to exit without performing any actions.

Example

  • If you send an insert, the application will search to see if it has been already applied. If this is the case, the operation is not applied.
  • When you can overwrite a script with its output. The output of any script is executable, and the output of a correct script is itself.

Map reduce

The individual tasks in a Map Reduce job:

  • are idempotent.
  • and have no side effects.

They can be restarted without changing the final result.

These two properties mean that given the same input, re-executing a task will always produce the same result and will not change other state. So, the results and end condition of the system are the same, whether a task is executed once or a thousand times.

Database Migration Script

A migration script will not try to add a column if it’s already there. Therefore, a specific script order isn't necessary needed in the database migration script.

1) 2)





Discover More
Card Puncher Data Processing
Ansible PlayBook - Task

A task is a call to an ansible module with arguments located in a play list. Variables can be used in arguments to modules. Tasks are executed top to bottom one at a time, against matched by the...
Data System Architecture
Database - Migration (Deployment|Versioning|Change Management)

with SQL database. Database migration is the process of changing the structure of the database in an other with mostly DDL statement. The flow sometimes will block any change that risks data-loss. See...
HTTP - Put Request

put is a value of the HTTP method header field. It's a attribute of a http request that can be used by the underlining server function. The exact meaning of the put value as given by the specification...
Map Reduce One Picture
Map Reduce (MR) Framework

Map reduce is a distributed execution . The MapReduce programming model (and a corresponding system) was proposed in a 2004 paper from a team at Google as a simpler abstraction for processing very large...
Card Puncher Data Processing
Software Design - Recovery (Restartable)

In really big system, there is always something that will go wrong. And it’s not possible to master all the different scenario that will arise. The file is delivered a little bit later, a mapping is...
Stream Vs Batch
Stream vs Batch

This article talks Stream Processing vs Batch Processing. The most important difference is that: in batch processing the size (cardinality) of the data to process is known whereas in a stream processing,...
What is the SameSite Cookie property? First-Party and third-party cookie control

What is the SameSite Cookie property? First-Party and third-party cookie control samesite is a cookie property that controls if a cookie should be sent along in a cross-site HTTP request ie: when...
What are the HTTP Request Methods (Get, Post, Put, )?

The http method is a mandatory header of http request that defines the type of operation. A minimal get request from this page It's used by the web server router to map a request to a function....



Share this page:
Follow us:
Task Runner