Ad

How To Split Dataframe Depending On The Values Of Multiple Columns

- 1 answer

I am using Python. I want to split my dataframe depending on the values of two columns. Every time the value pair changes, I want to split my dataframe at this position.

Example:

df = pd.DataFrame({'Distance':[1,1,1,1,3,3,3], 'labels':[1,2,2,2,4,4,5]})

df=

    Distance  labels
0       1       1
1       1       2
2       1       2
3       1       2
4       3       4
5       3       4
6       3       5

I want to get:

list_of_dfs[0]=

    Distance    labels
0       1       1



list_of_dfs[1]=

    Distance    labels
1       1       2
2       1       2
3       1       2



list_of_dfs[2]=

    Distance    labels
4       3       4
5       3       4    



list_of_dfs[3]=

    Distance    labels
6       3       5

This is how it works:

l = [1,4,6,7]
l_mod = [0] + l + [max(l)+1]
list_of_dfs = [df.iloc[l_mod[n]:l_mod[n+1]] for n in range(len(l_mod)-1)]

My question:

How can I automatically get the array l=[1,4,6,7] ? This is all I need to finish this task!

Ad

Answer

Use pd.duplicates to find the unique rows.

# use duplicated to determine rows that are original and create list
ind = df[~df.duplicated()].index.tolist()

# account for the last row, append value one greater than maximum index.
ind.append(df.shape[0])

# create dictionary for dataframe.
dfs = {}

# use iloc to create new dataframes, then add to dictionary.
for i in range(len(ind)-1): 
    df_temp = df.iloc[ind[i]:ind[i+1], :]
    dfs[i] = df_temp

Retrieve your dataframes from the dictionary:

df0 = dfs[0]
       Distance  labels
0         1       1

df1 = dfs[1]
print(df1)

  Distance  labels
1         1       2
2         1       2
3         1       2
Ad
source: stackoverflow.com
Ad