Data Processing - Buffer (Batch concept in code)

> (Data|State|Operand) Management and Processing > (Data Processing|Data Integration)

1 - About

Data Processing - Batch in code.

Buffers are containers of a (fixed|variable with max) amount of data or operations.

Buffers live in the memory (ie created by code structure) whereas a blocks lives in the disk.

Without buffer, each read or write request is handled directly by the underlying layer (OS, Database, …) . This can make a program much less efficient, since each such request often triggers other operations that may be relatively expensive.. For instance, for an OS layer

  • disk access,
  • network activity,
  • or some other operation

Furthermore, if requested data were sent as single big chunks, only one user could process it at a time, which obviously would make the application very inefficient and not much fun to use. (ie only one user would download data)

Example of buffered data:


3 - List of buffer implementation

4 - Flush Operation

Flushing the buffer means that you write data to an entity (mostly a file).

5 - Documentation / Reference