Yarn - Log (Container, Application) - Tfile
Table of Contents
1 - About
2 - Articles Related
3 - Format
Application logs are not saved in text format. They are saved in a binary format called TFile.
A TFile format is a container of key value pairs, and it is not readable by humans.
4 - Management
4.1 - View
4.1.1 - Command Line
yarn logs -applicationId <applicationId> yarn logs -applicationId <applicationId> -appOwner <user-who-started-the-application> yarn logs -applicationId <applicationId> -appOwner <user-who-started-the-application> -containerId <containerId> -nodeAddress <worker-node-address>
4.1.2 - Ui
With HortonWorks Ambari > Yarn > Quick Links:
4.1.3 - Other tfile reader
- https://github.com/rkanter/ALFExploder - explode any YARN application aggregated log files into their original separate log files.
4.2 - Configuration
4.2.1 - Aggregation
Logs for all the containers belonging to a single Application are aggregated and written out to a single (possibly compressed) log file at a configured location.
yarn.log-aggregation-enable to true
If disabled, node manager will keep the logs locally
4.2.2 - log-dirs
The log should be under the
yarn.nodemanager.log-dirs configuration properties
Screenshot from Ambari
4.2.3 - Location
184.108.40.206 - Application
The application log will be at:
220.127.116.11 - Container
The Containers log will be at