Skip to content

Split data file in Shell or Bash

Split file into equal size

filename=<your_data_file_full_path>
# e.g. filename=/data/orders.tbl
split --lines $(( $(expr $(expr 1024 * 1024 * 1024 * 1) / $(wc -L < ${filename})) )) ${filename} ${filename}.

Split file into number of files


split--lines $(wc -l <input file> | awk '{printf("%.0f", $1/<num_of_files>)}') <input file> <prefix>.