How to create a dictionary from a CSV file-CodePudding

I have a CSV file like this:

          w                                                syn
0     abaca   http://kaiko.getalp.org/dbnary/fra/Musa_textilis
1     abaca  http://kaiko.getalp.org/dbnary/fra/chanvre_de_...
2     abaca           http://kaiko.getalp.org/dbnary/fra/tagal
3     abaca   http://kaiko.getalp.org/dbnary/fra/Musa_textilis
4     abaca  http://kaiko.getalp.org/dbnary/fra/chanvre_de_...
..      ...                                                ...
95  abandon       http://kaiko.getalp.org/dbnary/fra/apostasie
96  abandon    http://kaiko.getalp.org/dbnary/fra/capitulation
97  abandon  http://kaiko.getalp.org/dbnary/fra/cession_de_...
98  abandon       http://kaiko.getalp.org/dbnary/fra/confiance
99  abandon       http://kaiko.getalp.org/dbnary/fra/défection

[100 rows x 2 columns]
6
{'abaca': 'tagal', 'abdomen': 'ventre', 'abricot': 'michemis', 'ADN': 'acide désoxyribonucléique', 'Indien': 'sauvage', 'abandon': 'défection'}

I ma trying to create a dictionary which each word and its synonym. I came up with this code but the final dictionary only contains one synonym for the word but as you can see in the csv file, a word can have more than one synonym.

# read specific columns of csv file using Pandas

df = pd.read_csv("sparql.csv", usecols = ["w","syn"]) #usecols = ["l","f","s","w","syn","synonyme"]
print(df)
liste_mot = df['w'].tolist()
liste_mot = set(liste_mot)
print(len(liste_mot))


liste_sys = []
dict_syn = {}


for index, row in df.iterrows():
    k, v = row
    sys = os.path.basename(v)
    if "_" in sys:
        sys = sys.split("_")
        sys = " ".join(sys)
        dict_syn[k] = sys
    else:
        dict_syn[k] = sys

print(dict_syn)

What I want to get is each word as key and a list of all their synonyms as its value but so far I only get one synonym (syn) per word (w) and not all of them.

CodePudding user response：

Here's a working example based partly on your code. Synonyms are put in a list:

from io import StringIO

import pandas as pd


text = """
          w                                                syn
0     abaca   http://kaiko.getalp.org/dbnary/fra/Musa_textilis
1     abaca  http://kaiko.getalp.org/dbnary/fra/chanvre_de_...
2     abaca           http://kaiko.getalp.org/dbnary/fra/tagal
3     abaca   http://kaiko.getalp.org/dbnary/fra/Musa_textilis
4     abaca  http://kaiko.getalp.org/dbnary/fra/chanvre_de_...
95  abandon       http://kaiko.getalp.org/dbnary/fra/apostasie
95  abandon       http://kaiko.getalp.org/dbnary/fra/apostasie
96  abandon    http://kaiko.getalp.org/dbnary/fra/capitulation
97  abandon  http://kaiko.getalp.org/dbnary/fra/cession_de_...
98  abandon       http://kaiko.getalp.org/dbnary/fra/confiance
99  abandon       http://kaiko.getalp.org/dbnary/fra/défection
"""

# read in data
df = pd.read_csv(StringIO(text), sep='\s ')

# get the synonym out of the url
df['real_syn'] = df['syn'].str.extract('.*/(.*)')

# dictionary to write results to
result = {}

# loop over every row of the dataframe
for _, row in df[['w', 'real_syn']].iterrows():
    word = row['w']
    syn = row['real_syn']
   
    # check if word is already in result dictionary and make sure words are not added twice
    if result.get(word) and syn not in result[word]:
            result[word] = result[word]   [syn]
    else:
        # if word is not yet in dictionary, then add it a key, and add the synonym as a list
        result[word] = [syn] 
        
print(result)

CodePudding user response：

I'm not sure if your CSV is actually fixed-width, or if that's just a nice printout.

If you don't need Pandas, Python's standard CSV module is up to the job.

import csv
import os
import pprint

from collections import defaultdict

def syn_splitter(s):
    syn = os.path.basename(s)
    syn = syn.replace('_', ' ')
    return syn

# So we can just start appending syns, without having to "prime" the dictionary with an empty list
word_syn_map = defaultdict(list)

with open('sample.csv', 'r', newline='') as f:
    reader = csv.reader(f)
    next(reader)  # discard header

    for row in reader:
        w, syn = row
        syn = syn_splitter(syn)
        word_syn_map[w].append(syn)

pprint.pprint(word_syn_map)

# word_syn_map = dict(word_syn_map) if you want to get rid of the defaultdict wrapper

I mocked up sample.csv:

w,syn
abaca,http://kaiko.getalp.org/dbnary/fra/Musa_textilis
abaca,http://kaiko.getalp.org/dbnary/fra/tagal
abaca,http://kaiko.getalp.org/dbnary/fra/Musa_textilis
abandon,http://kaiko.getalp.org/dbnary/fra/apostasie
abandon,http://kaiko.getalp.org/dbnary/fra/capitulation
abandon,http://kaiko.getalp.org/dbnary/fra/confiance
abandon,http://kaiko.getalp.org/dbnary/fra/défection

and I got:

defaultdict(<class 'list'>,
            {'abaca': ['Musa textilis', 'tagal', 'Musa textilis'],
             'abandon': ['apostasie',
                         'capitulation',
                         'confiance',
                         'défection']})

CodePudding user response：

Another approach:

import os

df = pd.read_csv("sparql.csv", usecols=["w","syn"])
df["syn_new"] = df.syn.map(os.path.basename).str.replace("_", " ")
dict_syn = {
    key: group.syn_new.to_list()
    for key, group in df[["w", "syn_new"]].groupby("w")
}

Result for your sample:

{'abaca': ['Musa textilis',
           'chanvre de ...',
           'tagal',
           'Musa textilis',
           'chanvre de ...'],
 'abandon': ['apostasie',
             'capitulation',
             'cession de ...',
             'confiance',
             'défection']}

You could try if

df["syn_new"] = df.syn.str.rsplit("/", 1, expand=True)[1].str.replace("_", " ")

works too, could be faster.

And maybe you don't want lists but sets as dict_syn values to avoid duplicates:

...
    key: set(group.syn_new.to_list())
...