python - pandas merge dataframes generated inside a loop -
suppose have dataframes (generated inside loop , added list):
column row data_503 plate 0 1 1 2 1 1 b 2 2 2 1 c 3 2 3 1 d 4 2 column row data_280 plate 0 1 1 2 1 1 b 2 2 2 1 c 3 2 3 1 d 4 2 column row data_503 plate 0 1 1 1 1 1 b 2 1 2 1 c 3 1 3 1 d 4 1 column row data_280 plate 0 1 1 1 1 1 b 2 1 2 1 c 3 1 3 1 d 4 1 i have layout file links maps measurements specific conditions:
column row cond plate 0 1 5 1 1 1 b 5 1 2 1 c 5 1 3 1 d 4 1 0 1 5 2 1 1 b 5 2 2 1 c 5 2 3 1 d 4 2 i can combine dataframes like:
for df in df_list: layout= pd.merge(layout, df, on=['plate', 'row', 'column'], how = 'outer') however, data_280_x , data_280_y columns obtain data_280 , data_503 columns. changing outer left not change anything.
any ideas how obtain like?:
column row cond plate data_280 data_503 0 1 5 1 1 1 1 1 b 5 1 2 2 2 1 c 5 1 3 3 3 1 d 4 1 4 4 0 1 5 2 1 1 1 1 b 5 2 2 2 2 1 c 5 2 3 3 3 1 d 4 2 4 4
you can combine _x , _y columns since not going have overlapping values (based on layout df), this:
df['data_208'] = df['data_208_x'] + df['data_208_y'] then can drop _x , _y columns.
update example:
df1 = pd.dataframe({"column": [1, 1, 1, 1], "row": ["a", "b", "c", "d"], "plate": [1, 1, 1, 1], "data_503": [4, 5, 6, 7]}) df2 = pd.dataframe({"column": [1, 1, 1, 1], "row": ["a", "b", "c", "d"], "plate": [1, 1, 1, 1], "data_280": [1, 2, 3, 4]}) df3 = pd.dataframe({"column": [1, 1, 1, 1], "row": ["a", "b", "c", "d"], "plate": [2, 2, 2, 2], "data_503": [4, 5, 6, 7]}) df4 = pd.dataframe({"column": [1, 1, 1, 1], "row": ["a", "b", "c", "d"], "plate": [2, 2, 2, 2], "data_280": [1, 2, 3, 4]}) layout = pd.dataframe({"column": [1, 1, 1, 1, 1, 1, 1, 1], "row": ["a", "b", "c", "d", "a", "b", "c", "d"], "cond": [5, 5, 5, 4, 5, 5, 5, 4], "plate": [1, 1, 1, 1, 2, 2, 2, 2]}) out = [] df in [df1, df2, df3, df4]: _ = pd.merge(layout, df, on=['column', 'row', 'plate'], how='outer').dropna() out.append(_) merged = out[0] df in out[1:]: merged = pd.merge(merged, df, on=['column', 'row', 'plate', 'cond'], how='outer') merged = merged.fillna(0) merged['data_280'] = merged['data_280_x'] + merged['data_280_y'] merged['data_503'] = merged['data_503_x'] + merged['data_503_y'] merged = merged.drop(['data_280_x','data_280_y','data_503_x','data_503_y'],1) gave me:
column cond plate row data_280 data_503 0 1 5 1 1.0 4.0 1 1 5 1 b 2.0 5.0 2 1 5 1 c 3.0 6.0 3 1 4 1 d 4.0 7.0 4 1 5 2 1.0 4.0 5 1 5 2 b 2.0 5.0 6 1 5 2 c 3.0 6.0 7 1 4 2 d 4.0 7.0
Comments
Post a Comment