Surprising Alternatives to Pandas

Check Datatable, Vaex and Dask performance alongside Pandas

Image from Unsplash

Panda (Panel Data) is a very popular library for exploring datasets in a data frame. This library is packed with excellent data analysis tools and performs well will small to medium-sized datasets. With the rise of big data, the library is slowly being replaced with several other alternatives but not all of those come with the perks of pandas. Datatable is a very much familiar package to R users. A counterpart library to that of R is datatable in python. We will evaluate the time required to read the same amount of data using Dask and Vaex too.

I have a .csv file consisting of 2.27 millions of rows with 21 features. I will check the time required for panda, datatable, vaex and ask to read the file.

Time to read a file

Panda took 4.17s to read the file whereas datatble took 0.395s for that.

Image by Author

Image by Author