Spark - (Take|TakeOrdered)

1 - About

The take action returns an array of the first n elements (not ordered) whereas takeOrdered returns an array with the first n elements after a sort

It's a Top N function

3 - Take



rdd = sc.parallelize([1, 2, 3]) 
Value: [1,2] # as list 

4 - Takeordered


Takeordered is an action that returns n elements ordered in ascending order as specified by the optional key function:

If key function returns a negative value (-1), the order is a descending order.

Python List:

rdd = sc.parallelize([5,3,1,2])
rdd.takeOrdered(3,lambda s: ‐1 * s)
Value: [5,3,2] # as list

Python Tuple:

topTenErrURLs = endpointSum.takeOrdered(10,lambda (x,y): -1*y)
db/spark/take.txt · Last modified: 2017/09/06 20:15 by gerardnico