How to Split Large Csv Files Into Smaller Ones

Basically, you don’t need anything special to do it. You just need a Linux command split.

This is a tiny script I have writen to do it:

1
2
#!/bin/bash
sudo split -l 100 myfile.csv -d --additional-suffix=.csv eric-

Let me explain it a little bit.

  • the -l, -d, and –additional-suffix: man split and you can see the documentation
  • eric-: it’s the prefix and you don’t need option to specify it. Just add it.

2018年3月29日更新:

通过上面的命令,按照5000行一个小文件,切割了一个6.9G大小的CSV文件,一共切割了不到5分钟,生成708个小文件。

OK!

© 2018 awesome.js All Rights Reserved.
Theme by hiero