Ad

Python Df.to_csv Only Prints Last Line Even If For Loop Is Present

- 1 answer

I am writing a code where the final results(denoted by x) to be exported to csv file. I used a for loop to iterate over but stil it only exports the last line of the result. My full code below is:

import csv
import itertools
import requests
import json
import pandas as pd


domainfile=open('domainsinfo.csv',newline='',encoding='utf_8')
reader=csv.reader(domainfile)
w=[]
  for row in reader:
    w.extend(row)

domain = list(itertools.permutations(w,1))
print(domain)

 def url_report(domain):
    url = 'https://www.virustotal.com/vtapi/v2/url/report'
    params = {'apikey': '', 'resource':domain}
    response = requests.get(url, params=params)
    return response

def pp_json(json_thing, sort=True, indents=4):
        if type(json_thing) is str:
            print(json.dumps(json.loads(json_thing), sort_keys=sort, 
    indent=indents))
      else:
              print(json.dumps(json_thing, sort_keys=sort, 
 indent=indents))
      return None


for d in domain:
    response = url_report(d)
    json_response = response.json()
    pretty_json = pp_json(json_response)

    response_list=[]
    for key in json_response['scans']:   

        if json_response['scans'][key] ['detected'] is True:      
            response_list.append(True)
        else:
            response_list.append(False)
    x=any(response_list)
    print(x)


    for d in domain:        
      final_list=[]

    final_list.append(x)
    result=(final_list)
    result_table = {'Domain': [d], 'Result':result}

    df=pd.DataFrame(data=result_table) 
    print(df)

    export_csv = df.to_csv (r'C:\csv', index=None, header=True)

print(pretty_json)


input()

Can someone explain why this does not work even if the for loop is present

Ad

Answer

Like liakoyras mentioned, you are writing your dataframe in the CSV. There are many ways to overcome your problem. While he suggested a merge, I am giving you another alternative here using append. Read the documentation for more information.

Here is how you need to modify your code block, explanation in comments inside the code:

######## Declare a dataframe outside the loop
final = pd.DataFrame()

for d in domain:
    ........................
       your other codes
    ........................
    result_table = {'Domain': [d], 'Result':result}

    df=pd.DataFrame(data=result_table) 

    ###### keep appending df to final 
    ###### the final df now gets updated in every loop
    final = final.append(df)

#### now outside the loop, write final dataframe to your csv
final.to_csv('your/path/file.csv')
Ad
source: stackoverflow.com
Ad