Home > Back-end >  I want to remove specifc data from rows in pandas column
I want to remove specifc data from rows in pandas column

Time:12-25

Here is how my data looks like.

    Complaint Type  Created Date
0   Noise - Street/Sidewalk 10/31/2013 02:08:41 AM
1   Illegal Parking 10/31/2013 02:01:04 AM
2   Noise - Commercial  10/31/2013 02:00:24 AM
3   Noise - Vehicle 10/31/2013 01:56:23 AM
4   Rodent  10/31/2013 01:53:44 AM
... ... ...
111064  Maintenance or Facility 10/04/2013 12:01:13 AM
111065  Illegal Parking 10/04/2013 12:01:05 AM
111066  Noise - Street/Sidewalk 10/04/2013 12:00:45 AM
111067  Noise - Commercial  10/04/2013 12:00:28 AM
111068  Blocked Driveway    10/04/2013 12:00:10 AM

What I want to do here to remove all the complaints except these:

complaint_list=['Blocked Driveway','DOF Literature Request','GENRAL CONSTRUCTION','HEATING','Illegal Parking','NONCONST','PAINT-PLASTER','PLUMBING','Street Condition','Street Light Condition']

here is some portion of data in dictionary format.

{'Complaint Type': {0: 'Noise - Street/Sidewalk',
  1: 'Illegal Parking',
  2: 'Noise - Commercial',
  3: 'Noise - Vehicle',
  4: 'Rodent',
  5: 'Noise - Commercial',
  6: 'Blocked Driveway',
  7: 'Noise - Commercial',
  8: 'Noise - Commercial',
  9: 'Noise - Commercial',
  10: 'Noise - House of Worship',
  11: 'Noise - Commercial',
  12: 'Illegal Parking',
  13: 'Noise - Vehicle',
  14: 'Rodent',
  15: 'Noise - House of Worship',
  16: 'Noise - Street/Sidewalk',
  17: 'Illegal Parking',
  18: 'Street Light Condition',
  19: 'Noise - Commercial',
  20: 'Noise - House of Worship',
  21: 'Noise - Commercial',
  22: 'Noise - Vehicle',
  23: 'Noise - Commercial',
  24: 'Blocked Driveway',
  25: 'Noise - Street/Sidewalk',
  26: 'Street Light Condition',
  27: 'Harboring Bees/Wasps',
  28: 'Noise - Street/Sidewalk',
  29: 'Street Light Condition',
  30: 'Blocked Driveway',
  31: 'Noise - Street/Sidewalk',
  32: 'Taxi Complaint',
  33: 'Noise - House of Worship',
  34: 'Homeless Encampment',
  35: 'Blocked Driveway',
  36: 'Traffic Signal Condition',
  37: 'Noise - Commercial',
  38: 'Traffic Signal Condition',
  39: 'Blocked Driveway',
  40: 'Noise - Commercial',
  41: 'Food Establishment',
  42: 'Noise - Commercial',
  43: 'Noise - Commercial',
  44: 'Noise - Park',
  45: 'Noise - Street/Sidewalk',
  46: 'Noise - Commercial',
  47: 'Taxi Complaint',
  48: 'Noise - Commercial',
  49: 'Noise - Street/Sidewalk',
  50: 'Noise - Commercial',
  51: 'Broken Muni Meter',
  52: 'Blocked Driveway',
  53: 'Noise - Commercial',
  54: 'Benefit Card Replacement',
  55: 'Noise - Commercial',
  56: 'Sanitation Condition',
  57: 'ELECTRIC',
  58: 'PLUMBING',
  59: 'HEATING',
  60: 'ELECTRIC',
  61: 'HEATING',
  62: 'HEATING',
  63: 'GENERAL CONSTRUCTION',
  64: 'HEATING',
  65: 'ELECTRIC',
  66: 'GENERAL CONSTRUCTION',
  67: 'Street Condition',
  68: 'Consumer Complaint',
  69: 'Blocked Driveway',
  70: 'Derelict Vehicles',
  71: 'Noise - Commercial',
  72: 'Derelict Vehicles',
  73: 'Noise',
  74: 'Noise',
  75: 'Blocked Driveway',
  76: 'Noise - Commercial',
  77: 'Drinking',
  78: 'Indoor Air Quality',
  79: 'Noise'},
 'Created Date': {0: '10/31/2013 02:08:41 AM',
  1: '10/31/2013 02:01:04 AM',
  2: '10/31/2013 02:00:24 AM',
  3: '10/31/2013 01:56:23 AM',
  4: '10/31/2013 01:53:44 AM',
  5: '10/31/2013 01:46:52 AM',
  6: '10/31/2013 01:46:40 AM',
  7: '10/31/2013 01:44:19 AM',
  8: '10/31/2013 01:44:14 AM',
  9: '10/31/2013 01:34:41 AM',
  10: '10/31/2013 01:25:12 AM',
  11: '10/31/2013 01:24:14 AM',
  12: '10/31/2013 01:20:57 AM',
  13: '10/31/2013 01:20:13 AM',
  14: '10/31/2013 01:19:54 AM',
  15: '10/31/2013 01:14:02 AM',
  16: '10/31/2013 12:54:03 AM',
  17: '10/31/2013 12:52:46 AM',
  18: '10/31/2013 12:51:00 AM',
  19: '10/31/2013 12:46:27 AM',
  20: '10/31/2013 12:43:47 AM',
  21: '10/31/2013 12:41:17 AM',
  22: '10/31/2013 12:39:55 AM',
  23: '10/31/2013 12:38:00 AM',
  24: '10/31/2013 12:37:16 AM',
  25: '10/31/2013 12:35:18 AM',
  26: '10/31/2013 12:33:00 AM',
  27: '10/31/2013 12:32:44 AM',
  28: '10/31/2013 12:32:08 AM',
  29: '10/31/2013 12:32:00 AM',
  30: '10/31/2013 12:31:17 AM',
  31: '10/31/2013 12:30:36 AM',
  32: '10/31/2013 12:30:31 AM',
  33: '10/31/2013 12:29:47 AM',
  34: '10/31/2013 12:28:30 AM',
  35: '10/31/2013 12:23:24 AM',
  36: '10/31/2013 12:23:00 AM',
  37: '10/31/2013 12:20:44 AM',
  38: '10/31/2013 12:20:00 AM',
  39: '10/31/2013 12:19:48 AM',
  40: '10/31/2013 12:18:05 AM',
  41: '10/31/2013 12:16:25 AM',
  42: '10/31/2013 12:15:06 AM',
  43: '10/31/2013 12:14:42 AM',
  44: '10/31/2013 12:12:08 AM',
  45: '10/31/2013 12:11:58 AM',
  46: '10/31/2013 12:09:07 AM',
  47: '10/31/2013 12:08:47 AM',
  48: '10/31/2013 12:07:45 AM',
  49: '10/31/2013 12:05:10 AM',
  50: '10/31/2013 12:04:50 AM',
  51: '10/31/2013 12:03:27 AM',
  52: '10/31/2013 12:02:01 AM',
  53: '10/31/2013 12:01:47 AM',
  54: '10/31/2013 12:01:45 AM',
  55: '10/31/2013 12:01:34 AM',
  56: '10/31/2013 12:01:00 AM',
  57: '10/31/2013 12:00:00 AM',
  58: '10/31/2013 12:00:00 AM',
  59: '10/31/2013 12:00:00 AM',
  60: '10/31/2013 12:00:00 AM',
  61: '10/31/2013 12:00:00 AM',
  62: '10/31/2013 12:00:00 AM',
  63: '10/31/2013 12:00:00 AM',
  64: '10/31/2013 12:00:00 AM',
  65: '10/31/2013 12:00:00 AM',
  66: '10/31/2013 12:00:00 AM',
  67: '10/30/2013 11:58:43 PM',
  68: '10/30/2013 11:57:57 PM',
  69: '10/30/2013 11:57:00 PM',
  70: '10/30/2013 11:57:00 PM',
  71: '10/30/2013 11:55:03 PM',
  72: '10/30/2013 11:55:00 PM',
  73: '10/30/2013 11:53:00 PM',
  74: '10/30/2013 11:53:00 PM',
  75: '10/30/2013 11:50:51 PM',
  76: '10/30/2013 11:50:30 PM',
  77: '10/30/2013 11:49:10 PM',
  78: '10/30/2013 11:48:22 PM',
  79: '10/30/2013 11:47:00 PM'}}

What I have done so far is that I have selected unique values from the complaint type and then added them to list and created a list with the complains given above in complaint_list.

    all_complaint_list=list(task3['Complaint Type'].unique())
    complaint_list=['Blocked Driveway','DOF Literature Request','GENRAL CONSTRUCTION','HEATING','Illegal Parking','NONCONST','PAINT-PLASTER','PLUMBING','Street Condition','Street Light Condition']
    for i in complaint_list:
        if i not in all_complaint_list:
            continue
        else:
            all_complaint_list.remove(i)

I have used pandas.drop method but it doesn't work.

CodePudding user response:

First make that dictionary a Pandas Dataframe:

df = pd.DataFrame.from_dict(dict)

Then you can use isin:

df[df["Complaint Type"].isin(complaint_list)]

This should give you all the rows that match the items in complaint_list.

  • Related