About
A data processing function takes an input and creates an output in a pipeline.
A data operator is also known as:
- transition in Automata
- functional interface in Functional Programming
- Filter in Data Processing (Shell and Log Pipeline)
Type
It may involve various function such as, including:
- Validation with data type conversion – Ensuring that supplied data is “clean, correct and useful”. See also: Data Quality
- Sorting – “arranging items in some sequence and/or in different sets.”
- Summarization – reducing detail data to its main points.
- Aggregation – combining multiple pieces of data.
- Analysis – the “collection, organization, analysis, interpretation and presentation of data.”.
- Reporting / visualization – list detail or summary data or computed information.
- Classification – separates data into various categories.
- Filtering - transform or delete data
- selection - select the data field