Wrong encoding for tomcat/spring server REST input-CodePudding

I have a REST service made with Tomcat 8 and Spring v5.3.18. The service gets and stores a string in DB.

I have an issue with this input:

{"name": "Pawe\u0142"}

It is a valid UTF8 char. The string saved is Pawe? so there is something wrong...

I started to check where it can be the missing encoding, but I found that it depends on the tomcat environment. Two servers with the same software, linked to a common database but with different operative system, first is Linux, second is Windows have different responses.

The Windows server is working and store the valid UTF8 char, the linux server is not working and stores the char as "?".

So I deduced it is not related to the software or missing encoding in sw configuration, but probably it is related to tomcat or java.

Do you have any idea about what can be wrong?

CodePudding user response：

Probable causes could be LC_CTYPE linux locale or file.encoding JVM property:

Linux LC_CTYPE (or LC_ALL) locale is set to other than UTF-8. Simulated as

LC_CTYPE="ISO-8859-1" jshell

jshell> import java.nio.charset.Charset;

jshell> Charset.defaultCharset()
$2 ==> US-ASCII

jshell> String a = "adf \u0142";
a ==> "adf ?"

JVM has file.encoding property set to other than UTF-8

jshell -J-Dfile.encoding="ISO-8859-1" --execution="local"

jshell> import java.nio.charset.Charset;

jshell> Charset.defaultCharset()
$2 ==> ISO-8859-1

jshell> String a = "adf \u0142";
    a ==> "adf ?"


jshell -J-Dfile.encoding="UTF-8" --execution="local"

jshell> String a = "adf \u0142";
a ==> "adf ł"

Linux quick tests

java -Dfile.encoding="ISO-8859-1" -XshowSettings:properties -version 2>&1 | grep 'encoding'
    file.encoding = ISO-8859-1
    sun.io.unicode.encoding = UnicodeLittle
    sun.jnu.encoding = UTF-8

java -XshowSettings:properties -version 2>&1 | grep 'encoding'
    file.encoding = UTF-8
    sun.io.unicode.encoding = UnicodeLittle
    sun.jnu.encoding = UTF-8

ps -lf -C jshell | grep 'file.encoding'
0 S lmc      7266  2225 10  80   0 - 895886 -     14:48 pts/2    00:00:05 jshell -J-Dfile.encoding=ISO-8859-1 --execution=local


LC_ALL="es_ES.ISO-8859-1" java -help 2>&1 | tail -n-3
Para especificar un argumento para una opci�n larga, puede usar --<nombre>=<valor> o
--<nombre> <valor>.

CodePudding user response：

Finally the error was in the connection string with DB / JDBC.

I had to append some params to the connection string URL for Mysql driver:

dbConnectionString=jdbc:mysql://localhost/test?useSSL=false&useUnicode=true&characterEncoding=utf-8