Simpler way to write headers-CodePudding

I am looking for a simplier way to write these headers in the example below. The reason I have one for each loop is so that it'll generate a new User Agent at random for each request it makes; however, I am looking for a way to not have to write it so many times for each loop.

Code Example:

for example in examples:
    headers = {
    'accept': '*/*',
    'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6',
    'referer': 'https://www.google.com/',
    'user-agent': random.choice(all_user_agents),
    }
    response = request.get(url, headers=headers)

    while 10 > i:
        headers = {
        'accept': '*/*',
        'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6',
        'referer': 'https://www.google.com/',
        'user-agent': random.choice(all_user_agents),
        }
        response = request.get(url, headers=headers)

    for test in tests:
        headers = {
        'accept': '*/*',
        'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6',
        'referer': 'https://www.google.com/',
        'user-agent': random.choice(all_user_agents),
        }
        response = request.get(url, headers=headers)

CodePudding user response：

You can extract common header values like accept-language and referer into a common dict object, and then just clone and update those default headers as needed:

import random
import string

import requests as req

default_headers = {
    'accept': '*/*',
    'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6',
    'referer': 'https://www.google.com/',
    # 'user-agent': None,
}

# - SETUP -
examples = ['a']
tests = []
url = 'TODO'
all_user_agents = string.ascii_lowercase
i = 5

# - LOOP -
for _ in examples:
    # >  .copy() is optional
    headers = default_headers.copy()

    response = req.get(url, headers=headers)

    while 10 > i:
        headers['user-agent'] = random.choice(all_user_agents)
        response = req.get(url, headers=headers)
        # >  I added the line below, so we can avoid an infinite loop
        i  = 1

    for _ in tests:
        headers['user-agent'] = random.choice(all_user_agents)
        response = req.get(url, headers=headers)

CodePudding user response：

You can extract your headers into a function that generates a new user-agent each time it is called.

def get_headers():
    return {
        'accept': '*/*',
        'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6',
        'referer': 'https://www.google.com/',
        'user-agent': random.choice(all_user_agents),
    }

# ...

headers = get_headers()
response = request.get(url, headers=headers)