Data Processing - Buffer (Batch concept in code)

Data Processing - Batch in code.

Buffers are containers of a (fixed|variable with max) amount of data or operations.

Buffers live in the memory (ie created by code structure) whereas a blocks lives in the disk.

Without buffer, each read or write request is handled directly by the underlying layer (OS, Database, …) . This can make a program much less efficient, since each such request often triggers other operations that may be relatively expensive.. For instance, for an OS layer

• disk access,
• network activity,
• or some other operation

Example of buffered data:

• Strings,
• Byte arrays
• Database: Database - Fetch Size

• (Collection|Container) Abstract Data Type (Array, List, Set, ...) (queue, …)
• Java - IO - Buffer Streams

Flushing the buffer means that you write data to an entity (mostly a file).

• Data buffer