I have a dataframe “DF” with with 500,000 rows. Here are the data types per column:
each entry in the “data” column is an array with size = [5,500]
When I try to save this dataframe using
it returned me the following error:
13 with open(path, 'wb') as f:
---> 14 pkl.dump(obj, f, protocol=pkl.HIGHEST_PROTOCOL)
OSError: [Errno 22] Invalid argument
I also try this method but I get the same error:
with open('my_filename.pkl', 'wb') as f:
I try to save 10 rows of this dataframe:
and I have no error at all. Therefore, it can save small DF but not large DF.
I am using python 3, ipython notebook 3 in Mac.
Please help me to solve this problem. I really need to save this DF to a pickle file. I can not find the solution in the internet.