This is my first question at Stack Overflow.
I have a DataFrame of Pandas like this.
a b c d
one 0 1 2 3
two 4 5 6 7
three 8 9 0 1
four 2 1 1 5
five 1 1 8 9
I want to extract the pairs of column name and data whose data is 1 and each index is separate at array.
[ [(b,1.0)], [(d,1.0)], [(b,1.0),(c,1.0)], [(a,1.0),(b,1.0)] ]
I want to use gensim of python library which requires corpus as this form.
Is there any smart way to do this or to apply gensim from pandas data?
解决方案
Many gensim functions accept numpy arrays, so there may be a better way...
In [11]: is_one = np.where(df == 1)
In [12]: is_one
Out[12]: (array([0, 2, 3, 3, 4, 4]), array([1, 3, 1, 2, 0, 1]))
In [13]: df.index[is_one[0]], df.columns[is_one[1]]
Out[13]:
(Index([u'one', u'three', u'four', u'four', u'five', u'five'], dtype='object'),
Index([u'b', u'd', u'b', u'c', u'a', u'b'], dtype='object'))
To groupby each row, you could use iterrows:
from itertools import repeat
In [21]: [list(zip(df.columns[np.where(row == 1)], repeat(1.0)))
for label, row in df.iterrows()
if 1 in row.values] # if you don't want empty [] for rows without 1
Out[21]:
[[('b', 1.0)],
[('d', 1.0)],
[('b', 1.0), ('c', 1.0)],
[('a', 1.0), ('b', 1.0)]]
In python 2 the list is not required since zip returns a list.