Spark - Web UI (Driver UI)

> Database > Spark

1 - About

Each driver program has a web UI, typically on port 4040, that displays information about:

The Spark UI will tell you which DataFrames and what percentages are in memory. Spark - (Executor) Cache

This UI is only available for the duration of the application. The cluster manager UI's (master en worker) are always available. Example with the built-in standalone cluster manager, the UI by default are available at the port 8080 for the master and 8081 for the workers.

Advertising

3 - Management

3.1 - URL

http://<driver-node>:4040

where: