Spark - Log

> Database > Spark

1 - Management

1.1 - Directory

export SPARK_LOG_DIR=/var/log/spark2

1.2 - Cluster

1.2.1 - Standalone

Spark - Standalone installation (spark scheme) or Spark - Local Installation

Spark executor logs are located in the $SPARK_HOME/work/app-<AppName> directory (where <AppName> is the name of your application.

Advertising

1.2.2 - YARN

See Yarn - Log (Container, Application) - Tfile

Driver logs are by default printed to console