Spark - Worker (Executor)

1 - About

Worker programs run:

  • on cluster nodes
  • or in local threads

There's no communication between workers.

When you perform transformations and actions that use functions, Spark will automatically push a closure containing that function to the workers so that it can run at the workers. One closure is send per worker for every task.

Any modifications to the global variables at the workers are not sent to the driver or to other workers.

db/spark/worker.txt ยท Last modified: 2017/09/06 20:15 by gerardnico