Python: How to parse places in place? -
does not have headers several columns such:
john, 1, orange, chicken mary, 7, purple, hamburger joey, 2, yellow, chicken
you don't need csv
this, simple python. solution efficient regards memory, operates row row.
with open('temp.csv', 'r') fin, open('temp2.csv', 'w') fout: row in fin: if row.split()[3] == 'chicken': fout.write(row)
this more memory efficient list comprehension requires reading , storing in memory data written file out.
timings
# create sample data file of 30k rows, 2/3rds 'chicken'. open('temp.csv', 'w') f: _ in range(10000): f.write("john,1,orange,chicken\n") f.write("mary,7,purple,hamburger\n") f.write("joey,2,yellow,chicken\n") %%timeit open('temp.csv', 'r') fin, open('temp2.csv', 'w') fout: row in fin: if row.split()[3] == 'chicken': fout.write(row) # 10 loops, best of 3: 41 ms per loop %%timeit open('temp.csv') old_file, open('temp3.csv', 'w') new_file: cr = csv.reader(old_file) cw = csv.writer(new_file) cw.writerows([r r in cr if r[3] == 'chicken']) # 10 loops, best of 3: 58.7 ms per loop # turn previous list comprehension generator. %%timeit open('temp.csv') old_file, open('temp3.csv', 'w') new_file: cr = csv.reader(old_file) cw = csv.writer(new_file) cw.writerows((r r in cr if r[3] == 'chicken')) # 10 loops, best of 3: 66.5 ms per loop open('temp.csv') input_file, open('temp4.csv', 'w', newline='') output_file: reader = csv.reader(input_file, delimiter=',') writer = csv.writer(output_file, delimiter=',') writer.writerows(filter(lambda x: x[3].strip() != 'chcicken', reader)) --------------------------------------------------------------------------- typeerror traceback (most recent call last) <ipython-input-41-072d37a3ff92> in <module>() ----> 1 open('temp.csv') input_file, open('temp4.csv', 'w', newline='') output_file: 2 reader = csv.reader(input_file, delimiter=',') 3 writer = csv.writer(output_file, delimiter=',') 4 writer.writerows(filter(lambda x: x[3].strip() != 'chcicken', reader)) typeerror: 'newline' invalid keyword argument function
Comments
Post a Comment