Yarn - Log (Container, Application) - Tfile
Table of Contents
1 - About
2 - Articles Related
3 - Format
Application logs are not saved in text format. They are saved in a binary format called TFile.
A TFile format is a container of key value pairs, and it is not readable by humans.
4 - Management
4.1 - Configuration
yarn.log-aggregation-enable to true
4.2 - View
4.2.1 - Command Line
yarn logs -applicationId <applicationId> yarn logs -applicationId <applicationId> -appOwner <user-who-started-the-application> yarn logs -applicationId <applicationId> -appOwner <user-who-started-the-application> -containerId <containerId> -nodeAddress <worker-node-address>
4.2.2 - Ui
With HortonWorks Ambari > Yarn > Quick Links:
4.2.3 - Other tfile reader
- https://github.com/rkanter/ALFExploder - explode any YARN application aggregated log files into their original separate log files.
4.3 - Location
The log should be under the
yarn.nodemanager.log-dirs configuration properties
Definition: Where to store container logs:
- An application's localized log directory will be found in
- Individual containers' log directories will be below this, in directories named
Each container directory will contain the files stderr, stdin, and syslog generated by that container.
Screenshot from Ambari