I recently got into a discussion of how to tune spark jobs.
This led me to learnings related to dynamic allocation and stuff.
Some interesting links:
- http://jerryshao.me/architecture/2015/08/22/spark-dynamic-allocation-investigation/
- http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-2/