Ad
How To Know If My Custom HTTP Headers Are Being Passed?
So, I have been struggling with passing custom HTTP header for some time now.
I am creating a script (Python) to Open a URL with Custom headers like
{'Referer': 'https://google.com', 'X-Forwarded-For': '47.29.76.109',
'User-Agent': 'Mozilla/5.0 (Linux; Android 7.1.1; CPH1723 Build/N6F26Q;
wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/67.0.3396.87
Mobile Safari/537.36', 'existing_proxy_port_to_use': '8090'}
I have been using BrowserMob-Proxy for this but I am unable to see the effect when I try checking Network field in Inspect of Google Chrome.
CODE:
def automation():
headers = pd.read_excel('Database/header.xlsx')
for i in range(0,headers.shape[0]):
dict = {}
header = headers.loc[i]
dict['Referer'] = header['Referrer']
dict[header['Option']] = header['IP']
dict['User-Agent'] = header['USERAGENT']
dict['existing_proxy_port_to_use'] = "8090"
print(dict)
URL = 'xyz'
data = pd.read_csv('Database/data.csv')
server = Server(path="./browsermob-proxy/bin/browsermob-proxy", options=dict)
server.start()
proxy = server.create_proxy()
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--proxy-server={0}".format(proxy.proxy)) #Configure chrome options
driver = webdriver.Chrome(chrome_options=chrome_options,executable_path='/home/.../chromedriver')
proxy.new_har("google")
for j in range(0,data.shape[0]):
datum = data.loc[j]
print(datum)
driver.get(URL)
driver.quit()
server.stop()
return None
automation()
I am reading these Parameters from header file and using Selenium to fill the Google Form.
So, Please help me in knowing how to pass the headers correctly and how to know if they are working.
Ad
Answer
I solved the problem of passing the header by removing the Browsermob-proxy and instead using seleniumwire and use its driver._client.set_header_overrides(headers=dict_headers)
to override the default HTTP headers.
def automation():
headers = pd.read_excel('Database/header.xlsx')
data = pd.read_csv('Database/data.csv')
for i in range(0,headers.shape[0]):
dict_headers = {}
header = headers.loc[i]
dict_headers['Referer'] = header['Referrer']
dict_headers[header['Option']] = header['IP']
dict_headers['User-Agent'] = header['USERAGENT']
URL = 'xyz'
user_agent = "user-agent="+header['USERAGENT']
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument(user_agent)
driver = webdriver.Chrome(
chrome_options=chrome_options,
executable_path='/home/.../chromedriver')
driver._client.set_header_overrides(headers=dict_headers)
datum = data.loc[i]
driver.get(URL)
driver.quit()
return None
automation()
Ad
source: stackoverflow.com
Related Questions
- → What are the pluses/minuses of different ways to configure GPIOs on the Beaglebone Black?
- → Django, code inside <script> tag doesn't work in a template
- → React - Django webpack config with dynamic 'output'
- → GAE Python app - Does URL matter for SEO?
- → Put a Rendered Django Template in Json along with some other items
- → session disappears when request is sent from fetch
- → Python Shopify API output formatted datetime string in django template
- → Can't turn off Javascript using Selenium
- → WebDriver click() vs JavaScript click()
- → Shopify app: adding a new shipping address via webhook
- → Shopify + Python library: how to create new shipping address
- → shopify python api: how do add new assets to published theme?
- → Access 'HTTP_X_SHOPIFY_SHOP_API_CALL_LIMIT' with Python Shopify Module
Ad