Executing cmd command over subprocess.getstatusoutput does not include lang special characters-CodePudding

For a few days I am trying to solve my following problem but with no success.

So I have a bash script which successfully executes by calling command through a subprocess.getstatusoutput method. The bash script itself returns some results by logging out the string on stdout. I do indeed get the output of executed bash script from the subprocess.getstatusoutput but the output does not include intended Slovenian special language characters, but instead replaces them with the '?'.

By executing the bash script from the terminal, there is no problem with special language characters, only when I run it from the .py script by calling getstatusoutput method.

Due to described problem, here is an example:

država -> dr?ava

I have a Ubuntu 18.04 server with installed python version 3.8.9.

CodePudding user response：

The easy way to avoid this problem is to switch from getstatusoutput to run (without text=True), and to handle the returned stdout in a way that avoids trying to convert it from a bytestring to Unicode text.

For example:

#!/usr/bin/env python3
import subprocess, sys

cmd = [ 'echo', 'država' ] # this should really start your script
result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

exitcode = result.returncode
sys.stdout.write(f'Result had status {exitcode}\n')
sys.stdout.flush()
sys.stdout.buffer.write(b'Result had output: '   result.stdout)

See this working at https://replit.com/@CharlesDuffy2/ActiveEffectiveLocations#main.py

CodePudding user response：

Thanks for your answer. I have executed this code locally, and the result that gets logged out on stdout is the same. Meanwhile, I've tried to run the exact same code that you provided in my production environment, on the server and following error is thrown:

UnicodeEncodeError: 'ascii' codec can't encode character '\u017e' in position 2: ordinal not in range(128)

There is something wrong going on while executing subprocess.run method because if I do comment this line, there is no error thrown.

Server is using default terminal and the result of grep -i "fontface" /etc/default/console-setup returns FONTFACE="fixed".

We do use Apache Server with mod_wsgi: apache

Locale details are following: locale