GroupBy With Ffill Deletes Group And Does Not Put Group In Index
I'm running into a very strange issue ever since I have ported my code from one computer to another. I'm using pandas version 0.25.1 on this system, but am unsure on the pandas version I was using previously.
The issue is as follows:
I create a simple, unsorted (mock) dataframe on which I want to sort values and forward-fill all the NaN values.
In [1]: import pandas as pd
...: import numpy as np
In [2]: test = pd.DataFrame({"group" : ["A", "A", "A", "B", "B", "B", "C", "C"],
...: "count" : [2, 3, 1, 2, 1, 3, 1, 2],
...: "value" : [10, np.nan, 30, np.nan, 19, np.nan, 25, np.nan]})
In [3]: test
Out[3]:
group count value
0 A 2 10.0
1 A 3 NaN
2 A 1 30.0
3 B 2 NaN
4 B 1 19.0
5 B 3 NaN
6 C 1 25.0
7 C 2 NaN
However, when I do that I lose the entire "group" column, and it does not reappear in my index either.
In [4]: test.sort_values(["group", "count"]).groupby("group").ffill()
Out[4]:
count value
2 1 30.0
0 2 10.0
1 3 10.0
4 1 19.0
3 2 19.0
5 3 19.0
6 1 25.0
7 2 25.0
I've also tried to use the following using fillna, but that gives me the same result:
In [5]: test.sort_values(["group", "count"]).groupby("group").fillna(method = "ffill")
Out[5]:
count value
2 1 30.0
0 2 10.0
1 3 10.0
4 1 19.0
3 2 19.0
5 3 19.0
6 1 25.0
7 2 25.0
Does anyone know what I am doing wrong? The issue seems to be with the ffill method, since I CAN use .mean() on the groupby and retain my groupings.
Answer
IICU, you have to use 'update` to get the results back to the dataframe
test.update(test.sort_values(["group", "count"]).groupby("group").ffill())
print(test)
Output
group count value
0 A 2 10.0
1 A 3 10.0
2 A 1 30.0
3 B 2 19.0
4 B 1 19.0
5 B 3 19.0
6 C 1 25.0
7 C 2 25.0
Related Questions
- → What are the pluses/minuses of different ways to configure GPIOs on the Beaglebone Black?
- → Django, code inside <script> tag doesn't work in a template
- → React - Django webpack config with dynamic 'output'
- → GAE Python app - Does URL matter for SEO?
- → Put a Rendered Django Template in Json along with some other items
- → session disappears when request is sent from fetch
- → Python Shopify API output formatted datetime string in django template
- → Can't turn off Javascript using Selenium
- → WebDriver click() vs JavaScript click()
- → Shopify app: adding a new shipping address via webhook
- → Shopify + Python library: how to create new shipping address
- → shopify python api: how do add new assets to published theme?
- → Access 'HTTP_X_SHOPIFY_SHOP_API_CALL_LIMIT' with Python Shopify Module