Data Processing - Buffer (Batch concept in code)
> (Data|State) Management and Processing > (Data Processing|Data Integration)
Table of Contents
1 - About
Data Processing - Batch in code.
Buffers are containers of a (fixed|variable with max) amount of data or operations.
Buffers live in the memory (ie created by code structure) whereas a blocks lives in the disk.
Without buffer, each read or write request is handled directly by the underlying layer (OS, Database, …) . This can make a program much less efficient, since each such request often triggers other operations that may be relatively expensive.. For instance, for an OS layer
- disk access,
- network activity,
- or some other operation
Furthermore, if requested data were sent as single big chunks, only one user could process it at a time, which obviously would make the application very inefficient and not much fun to use. (ie only one user would download data)
Example of buffered data:
- Byte arrays
- Database: Database - Fetch Size
- Network: Network - Data unit
2 - Articles Related
3 - List of buffer implementation
4 - Flush Operation
Flushing the buffer means that you write data to an entity (mostly a file).
