I'm trying to append the data frame values as rows but its appending them as columns. I have 32 files that i would like to take the second column from (called dataset_code) and append it. But its creating 32 rows and 101 columns. I would like 1 column and 3232 rows.
import pandas as pd
import os
source_directory = r'file_path'
df_combined = pd.DataFrame(columns=["dataset_code"])
for file in os.listdir(source_directory):
if file.endswith(".csv"):
#Read the new CSV to a dataframe.
df = pd.read_csv(source_directory + '\\' + file)
df = df["dataset_code"]
df_combined=df_combined.append(df)
print(df_combined)
解决方案
You already have two perfectly good answers, but let me make a couple of recommendations.
If you only want the dataset_code column, tell pd.read_csv directly (usecols=['dataset_code']) instead of loading the whole file into memory only to subset the dataframe immediately.
Instead of appending to an initially-empty dataframe, collect a list of dataframes and concatenate them in one fell swoop at the end. Appending rows to a pandas DataFrame is costly (it has to create a whole new one), so your approach creates 65 DataFrames: one at the beginning, one when reading each file, one when appending each of the latter — maybe even 32 more, with the subsetting. The approach I am proposing only creates 33 of them, and is the common idiom for this kind of importing.
Here is the code:
import os
import pandas as pd
source_directory = r'file_path'
dfs = []
for file in os.listdir(source_directory):
if file.endswith(".csv"):
df = pd.read_csv(os.join.path(source_directory, file),
usecols=['dataset_code'])
dfs.append(df)
df_combined = pd.concat(dfs)