Spark1.4源码走读笔记之隐式转换
RDD源码中隐式转换def distinct(numPartitions: Int)(implicit ord: Ordering = null): RDD = withScope {
map(x => (x, null)).reduceByKey((x, y) => x, numPartitions).map(_._1)
}
使用了隐式参数，运行时不需要手动提供参数ord，运行时会根据上下文注入参数
def groupBy(f: T => K)(implicit kt: ClassTag): RDD[(K, Iterable)] = withScope {
groupBy(f, defaultPartitioner(this))
}
def groupBy(
f: T => K,
numPartitions: Int)(implicit kt: ClassTag): RDD[(K, Iterable)] = withScope {
groupBy(f, new HashPartitioner(numPartitions))
}
页:
[1]