Screencast Proposal: Speeding up batch jobs

Jun 27, 2013 • Simon Belluzzo

It’s not uncommon to see people either manually running a script or program over a series of files. I was thinking of showing how to automate and parallelise over multiple data with very little effort or knowledge of parallel programming. Basically, three stages:

  1. Shell for loops/xargs
  2. GNU Parallel on local machine
  3. GNU Parallel across remote machines

Let me know what you think!