Reduce time for Loop over multiple objects to get value Django-CodePudding

I have a Dataframe in which a column Code is increased significantly everyday and these codes are to be converted into object description for which I am doing something like the following:

    product = []
    beacon = []
    count = []
    c_start = time.time()
    for i, v in df["D Code"].iteritems():
        product.append(Product.objects.get(short_code=v[:2]).description)  #how to optimize this?
        beacon.append("RFID")
        count.append(v[-5:])
    c_end = time.time()
    print("D Code loop time ", c_end-c_start)

Now initially when the rows were less it used to work in no time but as the data increased the combined Database call for every code takes too much time. Is there any more efficient Django method to loop over a list and get the value?

The df['D Code]` looks something like this:

['TRRFF.1T22AD0029',
  'TRRFF.1T22AD0041',
  'TRRFF.1T22AD0009',
  'TRRFF.1T22AD0032',
  'TRRFF.1T22AD0028',
  'TRRFF.1T22AD0026',
  'TRRFF.1T22AD0040',
  'HTRFF.1T22AD0003',
  'TRRFF.1T22AD0048',
  'PPRFP.1T22AD0017',
  'TRRFF.1T22AD0047',
  'TRRFF.1T22AD0005',
  'TRRFF.1T22AD0033',
  'TRRFF.1T22AD0024',
  'TRRFF.1T22AD0042'],

CodePudding user response：

You can create a lookup dict with just one query. Then use that dict to find your description.

description_dict = {}
for product in Product.objects.values('short_code', 'description'):
    description_dict[product['short_code'] = product['description']

for i, v in df["D Code"].iteritems():
        product.append(description_dict[v[:2]])
        ...