Data frame with Hidden column values-CodePudding

The data frame consists of column 'value' which has some hidden characters.
When I write the data frame to PostgreSQL I get the below error
ValueError: A string literal cannot contain NUL (0x00) characters.
I some how found the cause of error. Refer table below (missing column value

|       | datetime                 | mc | tagname | value      | quality |
|-------|--------------------------|----|---------|------------|---------|
| 19229 | 16-12-2021 02:31:29.083  | L  | VIN     |            | 192     |
| 19230 | 16-12-2021 02:35:28.257  | L  | VIN     | C4A 173026 | 192     |

Checked the length of string- it was same 10 character like below rows

df.value.str.len()

Requirement:
I want to replace that empty area with text 'miss', i tried different method in pandas. I'm not able to do.

df['value'] = df['value'].str.replace(r"[\"\',]", '')<br />
df.replace('\'','', regex=True, inplace=True)

|       | datetime                 | mc | tagname | value      | quality |
|-------|--------------------------|----|---------|------------|---------|
| 19229 | 16-12-2021 02:31:29.083  | L  | VIN     | miss       | 192     |
| 19230 | 16-12-2021 02:35:28.257  | L  | VIN     | C4A 173026 | 192     |

CodePudding user response：

Try this:

df['value'] = df['value'].str.replace(r'[\x00-\x19]', '').replace('', 'miss')