How to get result from background process linux shell script? -
for example let's want count number of lines of 10 big files , print total.
for f in files #this creates background process each file wc -l $f | awk '{print $1}' & done
i trying like:
for f in files #this not work :/ n=$( expr $(wc -l $f | awk '{print $1}') + $n ) & done echo $n
you should use gnu parallel:
find . -maxdepth 1 -type f | parallel --gnu 'wc -l' | awk 'begin {n=0} {n += $1} end {print n}'
or else xargs in parallel mode:
find . -maxdepth 1 -type f | xargs -n1 -p4 wc -l | awk 'begin {n=0} {n += $1} end {print n}'
another option, if doesn't fit needs, write temp files. if don't want write disk, write /dev/shm. ramdisk on linux systems.
#!/bin/bash declare -a temp_files count=0 f in * if [[ -f "$f" ]]; temp_files[$count]="$(mktemp /dev/shm/${f}-xxxxxx)" ((count++)) fi done count=0 f in * if [[ -f "$f" ]]; cat "$f" | wc -l > "${temp_files[$count]}" & ((count++)) fi done wait cat "${temp_files[@]}" | awk 'begin {n=0} {n += $1} end {print n}' tf in "${temp_files[@]}" rm "$tf" done
by way, can though of map-reduce wc doing mapping , awk doing reduction.
Comments
Post a Comment