Tukukkk

I have a pandas dataframe (this is only a little piece)

>>> d1

   y norm test  y norm train  len(y_train)  len(y_test)  

0    64.904368    116.151232          1645          549   

1    70.852681    112.639876          1645          549   



                                    SVR RBF  

0   (35.652207342877873, 22.95533537448393)   

1  (39.563683797747622, 27.382483096332511)   



                                        LCV  

0  (19.365430594452338, 13.880062435173587)   

1  (19.099614489458364, 14.018867136617146)   



                                   RIDGE CV  

0  (4.2907610988480362, 12.416745648065584)   

1    (4.18864306788194, 12.980833914392477)   



                                         RF  

0   (9.9484841581029428, 16.46902345373697)   

1  (10.139848213735391, 16.282141345406522)   



                                           GB  

0  (0.012816232716538605, 15.950164822266007)   

1  (0.012814519804493328, 15.305745202851712)   



                                             ET DATA  

0  (0.00034337162272515505, 16.284800366214057)  j2m  

1  (0.00024811554516431878, 15.556506191784194)  j2m  

>>> 

I want to split all the columns that contain tuples. For example I want to replace the column LCV with the columns LCV-a and LCV-b .

How can I do that?

You can do this by apply(pd.Series) on that column:

In [13]: df = pd.DataFrame({'a':[1,2], 'b':[(1,2), (3,4)]})



In [14]: df

Out[14]:

   a       b

0  1  (1, 2)

1  2  (3, 4)



In [16]: df['b'].apply(pd.Series)

Out[16]:

   0  1

0  1  2

1  3  4



In [17]: df[['b1', 'b2']] = df['b'].apply(pd.Series)



In [18]: df

Out[18]:

   a       b  b1  b2

0  1  (1, 2)   1   2

1  2  (3, 4)   3   4

This works because it makes of each tuple a Series, which is then seen as a row of a dataframe.

On much larger datasets, I found that .apply() is few orders slower than pd.DataFrame(df['b'].values.tolist(), index=df.index)

This performance issue was closed in GitHub, although I do not agree with this decision:

https://github.com/pandas-dev/pandas/issues/11615

EDIT: based on this answer: https://stackoverflow.com/a/44196843/2230844

I know this is from a while ago, but a caveat of the second solution:

pd.DataFrame(df['b'].values.tolist())

is that it will explicitly discard the index, and add in a default sequential index, whereas the accepted answer

apply(pd.Series)

will not, since the result of apply will retain the row index. While the order is initially retained from the original array, pandas will try to match the indicies from the two dataframes.

This can be very important if you are trying to set the rows into an numerically indexed array, and pandas will automatically try to match the index of the new array to the old, and cause some distortion in the ordering.

A better hybrid solution would be to set the index of the original dataframe onto the new, i.e.

pd.DataFrame(df['b'].values.tolist(), index=df.index)

Which will retain the speed of using the second method while ensuring the order and indexing is retained on the result.