python - Why does pandas read_csv not support multiple comments (#,@,...)? -
i found pandas read_csv method faster numpy loadtxt. unfortunatly find myself in situation have go numpy because loadtxt has option of setting comments=['#','@']
. pandas read_csv method can take 1 comment string comment='#'
far can tell site. suggestions or workarounds make life easier , make me not pivot numpy? why pandas not support multiple comment indicators?
# save in test.dat @ bla # bla 1 2 3 4
minimal example:
# work, 1 type of comment accounted df = pd.read_csv('test.dat', index_col=0, header=none, comment='#') # not work (not suprising reading help) df = pd.read_csv('test.dat', index_col=0, header=none, comment=['#','@']) # work slow df = np.loadtxt('test.dat', comments=['#','@'])
the short answer nobody has implemented in pandas
yet. looking through github issues, looks else has suggested , maintainers open patch implements it: https://github.com/pandas-dev/pandas/issues/13948
could opportunity contribute pandas
project if feel comfortable that, or keep eye on issue if else it. part of codebase handles comments looks around here in _check_comments
: https://github.com/pandas-dev/pandas/blob/master/pandas/io/parsers.py#l2348
Comments
Post a Comment