Saving Footer (last Few Rows) Of Csv As Separate File Using Pandas In Python
I have a csv file that contains extra rows at the very end (last 9 rows) that are important, but do not fit the schema at all and need to be processed differently. They just contain number of clicks for different sites. I want to split these last few rows from the original csv and save it as a different file.
So far, I can get the most important rows out using pandas, skipping the footer. If the number of rows was consistent, then I could do the same for saving the footer using skiprows=0-2000 (for example), but these rows will change.
The code to save all the main rows is as follows:
reader = pd.read_csv(os.path.join(DATA_DIR, file), encoding='utf8', header=0, skipfooter=9, index_col=0)
trimmed_file_name = 'trimmed_{}'.format(file)
path = os.path.join(DATA_DIR)
full_path = path + "\ ".strip(' ') + trimmed_file_name
# had to use this odd way of creating a path because it kept trying to use \ as an escape char, just ignore
print(full_path)
reader.to_csv(full_path, mode='a')
So how do I just get out those last 9 rows without 'skiprows'? Any ideas? The footer is consistently the last 9 rows if that helps.
Answer
After reading in the first dataframe, we know how many regular rows there are. So just read the remaining by
footer = pd.read_csv(file, skiprows=len(reader))
Related Questions
- → What are the pluses/minuses of different ways to configure GPIOs on the Beaglebone Black?
- → Django, code inside <script> tag doesn't work in a template
- → React - Django webpack config with dynamic 'output'
- → GAE Python app - Does URL matter for SEO?
- → Put a Rendered Django Template in Json along with some other items
- → session disappears when request is sent from fetch
- → Python Shopify API output formatted datetime string in django template
- → Can't turn off Javascript using Selenium
- → WebDriver click() vs JavaScript click()
- → Shopify app: adding a new shipping address via webhook
- → Shopify + Python library: how to create new shipping address
- → shopify python api: how do add new assets to published theme?
- → Access 'HTTP_X_SHOPIFY_SHOP_API_CALL_LIMIT' with Python Shopify Module