2, If you want permanent variables, there are two situations.

1' When based on classic pipeline, you need to use these REST API to achieve your requirements:

A python script that can change the variables of the classic pipeline:

import json
import requests


org_name = "xxx"
project_name = "xxx"
pipeline_definition_id = "xxx"
personal_access_token = "xxx"

key = 'variables'
var_name = 'BUILDNUMBER'

url = "https://dev.azure.com/" org_name "/" project_name "/_apis/build/definitions/" pipeline_definition_id "?api-version=6.0"

payload={}
headers = {
  'Authorization': 'Basic ' personal_access_token
}

response = requests.request("GET", url, headers=headers, data=payload)
print(response.text)
json_content = response.text
def get_content_of_json(json_content, key, var_name):
    data = json.loads(json_content)
    return data[key][var_name].get('value')

def change_content_of_json(json_content, key, var_name):
    data = json.loads(json_content)
    data[key][var_name]['value'] = str(int(get_content_of_json(json_content,key,var_name))   1)
    return data

json_data = change_content_of_json(json_content, key, var_name)


url2 = "https://dev.azure.com/" org_name "/" project_name "/_apis/build/definitions/" pipeline_definition_id "?api-version=6.0"

payload2 = json.dumps(json_data)
headers2 = {
  'Authorization': 'Basic ' personal_access_token,
  'Content-Type': 'application/json'
}

response2 = requests.request("PUT", url2, headers=headers2, data=payload2)

2' If your pipeline is based on non-classic pipeline, then you need to change the variables part of YAML file definition.

After you change the YAML definition during pipeline run, you can follow this to push back the changes:

Push Back Changes to Repository

Also, you can make your pipeline based on variables group and update the variables group via REST API:

Variablegroups - Update