How to split a file (over certain amount) without breaking lines (data being split of a record)?

I am trying to develop a shell script with the requirement:

Find files in a directory with size over 250 MB - how to discover the file?

If it finds a file over 250 MB, split it to make it into smaller chunks without having data (records) being split apart.

Example - the script should be able to find a file that is 600 MB - I can expect to see 3 files (one that is 250 MB, 250 MB, and 100 MB). It should not exceed 250 MB per file and record should not be broken, the full record should be in the same file.

Any help would be appreciated!

Thank you

2 Answers

Relevance
  • oyubir
    Lv 6
    1 month ago

    find . -size +250MB -type f -exec split -C 250MB {} {}- \;

    should do

  • BigE
    Lv 7
    1 month ago

    I would use a programming language like perl  or python but it can done shell scripting.  FIrst, finding big files use the find command with -size +250MB  That will just give a list of big files.

    Oh, I assume Linux that you are using.

    Split (I think by default) does not split on records, but you certainly can test it with a test file.  I know split  uses some suffix naming convention, but I think it has the options you want.

Still have questions? Get your answers by asking now.