tldr/pages/common/parallel.md

# GNU Parallel

> Run commands on multiple CPU cores

- gzip several files at once, using all cores

`parallel gzip ::: {{file1}} {{file2}} {{file3}}`

- read arguments from stdin, run 4 jobs at once

`ls *.txt | parallel -j4 gzip`

- Convert JPG images to PNG using replacement strings

`parallel convert {} {.}.png ::: *.jpg`

- parallel xargs, cram as many args as possible onto one command

`{{args}} | parallel -X {{command}}`

- break stdin into ~1M blocks, feed each block to stdin of new command

`cat {{bigfile.txt}} | parallel --pipe --block 1M {{command}}`

- run on multiple machines via SSH

`parallel -S {{machine1}},{{machine2}} {{command}} ::: {{arg1}} {{arg2}}`