How do I divide lines into dictionaries and assign key-value pairs to them, and append an ID for eac-CodePudding

I have a text file with the following column names. I want these to be the keys in my dictionary, and grab the value data from separating lines in the txt file, which contains a series of lines like such:

   VLAN1100    84:03:28:a8:b3:18   D   -   ge-0/0/45.0            0         0       10.68.8.189:

key_list = ['vlan name', 'mac address', 'mac flags', 'age', 'logical interface', 'nh index', 'rtr id', 'IP']

with open('prog-input.txt', 'r') as file:
    lines = file.readlines()`
#separate column data into lines
    for line in lines:
        if 'VLAN' in line:
#skip headers
            entry = line.split()
            res = {}
            for key in key_list:
                for value in entry:
                    res[key] = value
                    entry.remove(value)
                    break
            print(res)
file.close()

This is correctly splitting the data into key value pairs, but not assigning them to individual dictionaries.

OUTPUT:

{'vlan name': 'VLAN1100', 'mac address': '94:40:c9:3a:44:1a', 'mac flags': 'D', 'age': '-', 'logical interface': 'ge-0/0/16.0', 'nh index': '0', 'rtr id': '0', 'IP': '10.68.14.67'}
{'vlan name': 'VLAN1100', 'mac address': '94:40:c9:3a:91:b6', 'mac flags': 'D', 'age': '-', 'logical interface': 'ge-0/0/8.0', 'nh index': '0', 'rtr id': '0', 'IP': '10.68.14.59'}
{'vlan name': 'VLAN1100', 'mac address': '94:40:c9:3a:e5:b2', 'mac flags': 'D', 'age': '-', 'logical interface': 'ge-0/0/17.0', 'nh index': '0', 'rtr id': '0', 'IP': '10.68.14.68'}
{'vlan name': 'VLAN1100', 'mac address': 'f4:a7:39:9c:4c:e0', 'mac flags': 'D', 'age': '-', 'logical interface': 'ge-0/0/47.0', 'nh index': '0', 'rtr id': '0', 'IP': '10.68.8.191'}`

I only care about the key:value pairs of 0, 1, 4, 7. I would like to assign each line entry with an ID, possibly based on the 'mac address', which is unique.

I also tried using this code, but I don't understand how to use map, so I'd rather it be spelled out:

        if 'VLAN' in line:
            device = {k:v for k, *v in map(line.split, file)}
            for key in key_list:
                for value in device:
                    device.append()
print(device)

But it doesn't work. I want to be able to create a dictionary for each line item, and then put them inside a List, to query later.

CodePudding user response：

import io    
import csv
import pandas as pd

file_content="""
VLAN1100    94:40:c9:3a:44:1a   D   -   ge-0/0/16.0 0   0   10.68.14.67
VLAN1100    94:40:c9:3a:91:b6   D   -   ge-0/0/8.0  0   0   10.68.14.59
VLAN1100    94:40:c9:3a:e5:b2   D   -   ge-0/0/17.0 0   0   10.68.14.68
VLAN1100    f4:a7:39:9c:4c:e0   D   -   ge-0/0/47.0 0   0   10.68.8.191
"""

key_list = ['vlan name', 'mac address', 'mac flags', 'age',
            'logical interface', 'nh index', 'rtr id', 'IP']


# with open("csv.abc") as fobj: #
#     records = csv.DictReader(fobj, fieldnames=key_list, delimiter="\t")

file_buffer = io.StringIO(file_content)
records = csv.DictReader(file_buffer, fieldnames=key_list, delimiter="\t")

df = pd.DataFrame(records)
df = df.iloc[:, [0,1,4,7]]
df.to_dict("records")

[{'vlan name': 'VLAN1100', 'mac address': '94:40:c9:3a:44:1a', 'logical interface': 'ge-0/0/16.0', 'IP': '10.68.14.67'}, {'vlan name': 'VLAN1100', 'mac address': '94:40:c9:3a:91:b6', 'logical interface': 'ge-0/0/8.0', 'IP': '10.68.14.59'}, {'vlan name': 'VLAN1100', 'mac address': '94:40:c9:3a:e5:b2', 'logical interface': 'ge-0/0/17.0', 'IP': '10.68.14.68'}, {'vlan name': 'VLAN1100', 'mac address': 'f4:a7:39:9c:4c:e0', 'logical interface': 'ge-0/0/47.0', 'IP': '10.68.8.191'}]

CodePudding user response：

Here is a slightly generalised method, that only uses built-ins, which you can use to perform similar tasks to the one you have specified.

import operator


def split_lines_to_dicts(lines, column_map, line_filter, id_generator):
    value_getter = operator.itemgetter(*list(column_map.keys()))
    column_names = list(column_map.values())

     return [
        dict(
            zip(column_names, value_getter(line.split())),
            id=next(id_generator)
        )
        for line in lines
        if line_filter(line)
    ]

To show how you might use that I'll show it with details plugged in from your situation:

import io
import itertools


print(split_lines_to_dicts(
    # You would use a file handle here
    lines=io.StringIO("""
        # Ignore me
        VLAN1100    84:03:28:a8:b3:18   D   -   ge-0/0/45.0            0         0       10.68.8.189
        VLAN1100    84:03:28:a8:b3:18   D   -   ge-0/0/45.0            0         0       10.68.8.189
        VLAN1100    84:03:28:a8:b3:18   D   -   ge-0/0/45.0            0         0       10.68.8.189
        ...
    """),
    # The position to the name we want in the output
    column_map={
        0: "vlan name",
        1: "mac address",
        4: "logical interface",
        7: "IP"
    },
    # A filter to decide which lines to keep
    line_filter=lambda line: "VLAN" in line,
    # A generator of ids, you could change this to do something else
    id_generator=itertools.count()
))

This would give you a list like this:

[
    {'vlan name': 'VLAN1100', 'mac address': '84:03:28:a8:b3:18', 'logical interface': 'ge-0/0/45.0', 'IP': '10.68.8.189', 'id': 0}, 
    {'vlan name': 'VLAN1100', 'mac address': '84:03:28:a8:b3:18', 'logical interface': 'ge-0/0/45.0', 'IP': '10.68.8.189', 'id': 1}, 
    {'vlan name': 'VLAN1100', 'mac address': '84:03:28:a8:b3:18', 'logical interface': 'ge-0/0/45.0', 'IP': '10.68.8.189', 'id': 2}
]