spark shuffle优化-参数调优

spark shuffle优化-参数调优 常用参数列举： spark.reducer.maxSizeInFlight spark.shuffle.compress spark.shuffle.file.buffer spark.shuffle.io.maxRetries spark.shuffle.io.retryWait 调优参数一定要设置合理，效率才能事半功倍  

>>阅读原文<<