pandas的使用

pandas中的解析函数：

在创建时,我们可以指定一下的属性:

• 路径 path
• 分隔符 sep
• 列名 header OR name
• 索引 index_col
• 跳行 skiprows
• 缺失值 na_values
• 行数 nrows
• 文件块 chunksize

还有很多,诸如comment, parse_dates, keep_date_col, converters, dayfirst, date_parser, iterator, skip_footer, verbose, encoding, squeeze, thousands

举例子

函数示范举例：

import pandas as pd

• 分隔符1

>>> pd.read_table('ex3.txt',sep='\s+')
            A         B         C
aaa -0.264438 -1.026059 -0.619500
bbb  0.927272  0.302904 -0.032399
ccc -0.264273 -0.386314 -0.217601
ddd -0.871858 -0.348382  1.100491

• 列名

列名为None:

>>> pd.read_csv('ex2.csv',header=None)
   0   1   2   3      4
0  1   2   3   4  hello
1  5   6   7   8  world
2  9  10  11  12    foo

自定义列名:

>>> pd.read_csv('ex2.csv', names=['a','b','c','d','message'])
   a   b   c   d message
0  1   2   3   4   hello
1  5   6   7   8   world
2  9  10  11  12     foo

• 索引

注:如果希望列名中的某一个列做成DataFrame的索引,通过index_col参数指定.

>>> pd.read_csv('ex2.csv', names=['a','b','c','d','message'], index_col='message')
         a   b   c   d
message               
hello    1   2   3   4
world    5   6   7   8
foo      9  10  11  12

当然我们也可以做一个层次化索引.

>>> pd.read_csv('csv_mindex.csv')
  key1 key2  value1  value2
0  one    a       1       2
1  one    b       3       4
2  one    c       5       6
3  one    d       7       8
4  two    a       9      10
5  two    b      11      12
6  two    c      13      14
7  two    d      15      16
>>> pd.read_csv('csv_mindex.csv', index_col=['key1', 'key2'])
           value1  value2
key1 key2                
one  a          1       2
     b          3       4
     c          5       6
     d          7       8
two  a          9      10
     b         11      12
     c         13      14
     d         15      16

• 跳行2

>>> pd.read_csv('ex4.csv', skiprows=[0,2,3])
   a   b   c   d message
0  1   2   3   4   hello
1  5   6   7   8   world
2  9  10  11  12     foo

• 缺失值

>>> res = pd.read_csv('ex5.csv')
>>> res
  something  a   b   c   d message
0       one  1   2   3   4     NaN
1       two  5   6 NaN   8   world
2     three  9  10  11  12     foo
>>> pd.isnull(res)
  something      a      b      c      d message
0     False  False  False  False  False    True
1     False  False  False   True  False   False
2     False  False  False  False  False   False
>>> pd.read_csv('ex5.csv', na_values=['NULL'])
  something  a   b   c   d message
0       one  1   2   3   4     NaN
1       two  5   6 NaN   8   world
2     three  9  10  11  12     foo

用字典为各列指定NA值

#可以看到[0][c],[2][message]变为NA值
>>> dict={'c':[1000,3],'message':['NA','foo']}
>>> pd.read_csv('ex5.csv', na_values=dict)
  something  a   b   c   d message
0       one  1   2 NaN   4     NaN
1       two  5   6 NaN   8   world
2     three  9  10  11  12     NaN

• 行数

>>> pd.read_csv('ex6.csv', nrows=10)
        one       two     three      four key
0  0.467976 -0.038649 -0.295344 -1.824726   L
1 -0.358893  1.404453  0.704965 -0.200638   B
2 -0.501840  0.659254 -0.421691 -0.057688   G
3  0.204886  1.074134  1.388361 -0.982404   R
4  0.354628 -0.133116  0.283763 -0.837063   Q
5  1.817480  0.742273  0.419395 -2.251035   Q
6 -0.776764  0.935518 -0.332872 -1.875641   U
7 -0.913135  1.530624 -0.572657  0.477252   K
8  0.358480 -0.497572 -0.367016  0.507702   S
9 -1.740877 -1.160417 -1.637830  2.172201   G

• 文件块

>>> chunk = pd.read_csv('ex6.csv', chunksize = 1000)
>>> from pandas import Series
>>> tot = Series([])#Series是一种类似于数组的对象，它由一组数据以及一组与之相关的数据标签（即索引）组成
>>> for piece in chunk:
    #value_counts()统计这个块中相同值的个数
    #块中缺失值几即赋值为0
...     tot = tot.add(piece['key'].value_counts(), fill_value=0)
... 
>>> tot = tot.sort_values(ascending=False) 
>>> tot[:10]
E    368
X    364
L    346
O    343
Q    340
M    338
J    337
F    335
K    334
H    330
dtype: float64

可移步参考DataFrame或者Series.