0

This is my first time asking at StackOverFlow. I'm not good at English. Please excuse me.

I'm having a problem that my application is returning a strange character.

PlayStation\ufffd\ufffd4 Pro

It has to be like this:

PlayStation®4 Pro

I think '\ufffd' character represents this, 'REPLACE CHARACTER'.

My application is using jdk 1.6.

I found that when I change my application's jdk to 1.7, it prints the character correctly.

PlayStation®4 Pro

More Information

My application uses ibatis, and the problem is occurring after queryForObject.

public class A { private String content; public String getContent() { return content; } } 
A a = (A)queryForObject("mapper.getSomething", params); return a; // jdk1.6 - a.getContent() : PlayStation\ufffd\ufffd4 Pro // jdk1.7 - a.getContent() : PlayStation®4 Pro

JDBC connection property is like this.

driverClassName=com.mysql.jdbc.Driver url=jdbc:mysql://{IPADDRESS}/{DBNAME}?Unicode=true&characterEncoding=MS949&zeroDateTimeBehavior=convertToNull&socketTimeout=500000&connectTimeout=500000

More Information 2

  • I tested without ibatis and others. Directly using jdbc connection, but the same result.
public class CharacterEncodeTest { // JDBC driver name and database URL static final String DB_URL = "jdbc:mysql://{IPADDRESS}/{DBTNAME}}?Unicode=true&characterEncoding=MS949&zeroDateTimeBehavior=convertToNull&socketTimeout=500000&connectTimeout=500000"; // Database credentials static final String USER = "{USER}"; static final String PASS = "{PASSWORD}"; public static void main(String[] args) { Connection conn = null; Statement stmt = null; try { //STEP 2: Register JDBC driver Class.forName("com.mysql.jdbc.Driver"); //STEP 3: Open a connection System.out.println("Connecting to a selected database..."); conn = DriverManager.getConnection(DB_URL, USER, PASS); System.out.println("Connected database successfully..."); //STEP 4: Execute a query System.out.println("Creating statement..."); stmt = conn.createStatement(); String sql = "SELECT * from TABLE"; ResultSet rs = stmt.executeQuery(sql); //STEP 5: Extract data from result set while (rs.next()) { //Retrieve by column name String content = rs.getString("content"); //Display values System.out.print("content: " + content); // jdk1.6 : PlayStation\ufffd\ufffd4 Pro // jdk1.7 : PlayStation®4 Pro } rs.close(); } catch (SQLException se) { // something } finally { // something }//end try } }

Question

The only difference is just changing jdk version.

  1. What difference is the matter between jdk 1.6 and 1.7 about this problem?

  2. Is there any solution to solve this problem in jdk 1.6?

Improve this question
4
  • Please explain what you are doing to get this string (maybe with a few lines of code). Commented Jan 29, 2019 at 11:47
  • Hello @Henry ! I edited little bit more code. I'm not sure it'll be helpful. When more information is needed, please tell me. Thanks for your comment. Commented Jan 29, 2019 at 12:08
  • This means, the string is already wrong when you get it back from queryForObject. The problem must be be in there or in even deeper layers. Use a debugger to track down where exactly it gets wrong. Commented Jan 29, 2019 at 12:12
  • How do you know that the System.out.print prints \ufffd\ufffd ? I don't know of any terminal or console that outputs unicode escapes. And have you considered that, since the result doesn't come from the JDK but from the mysql driver, that it may be something in there? Commented Jan 30, 2019 at 6:42

4 Answers 4

Reset to default
1

No idea what \ufffd is, but the ® symbol is \u00ae: https://www.fileformat.info/info/unicode/char/00ae/index.htm

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

\ufffd is the replacement character, it's used when a Unicode aware system cannot process a character
1

If one sees two replacement chars (� or ?) for one special char, then UTF-8 binary data was converted to a two-byte sequence, every byte > 127, and unconvertable to a char in a single-byte encoding only knowing 256 chars.

So a String (Unicode) was converted to UTF-8 bytes, and then those bytes converted to some single byte encoding.

This could be an URL parameter encoded as UTF-8, received as ISO-8859-1. Or some other meddling. URL. URL decoding/encoding got an encoding parameter. Most likely though there were changes in the environment too. If using ® in the java code, the editor must have the same encoding as the javac compiler, and able to represent the symbol (check by using \u00AE instead).

Search default encoding usages:

  • string.getBytes()
  • new String(bytes)
  • URLDecoder.decode(string)
  • URLEncoder.encode(string)
  • FileReader/FileWriter
  • InputStreamReader(inputStream)
  • OutputStreamWriter(outputStream)

Also zip handling got Unicode support for file names.

Anti-pattern:

  • new String(string.getBytes(...), ...)
Improve this answer

Comments

0

No idea, but i think jdk 1.6 and jdk 1.7 use different types of encoding for character. Please visit the below links :

Does Java 1.7 use a different character encoding?

Why is my String returning "\ufffd\ufffdN a m e"

Improve this answer

1 Comment

Both Java 6 and 7 use the same encoding for String: UTF-16 (see Javadoc: docs.oracle.com/javase/6/docs/api/java/lang/String.html). Source code files can be encode in the encoding of your choice you just have to indicate which one is used to the Java compiler using -encoding option (see docs.oracle.com/javase/6/docs/technotes/tools/windows/…).
0

You got two question mark characters initially. This looks like there was one UTF8 character, but your code was not able to read the 4-byte sequence and thus showed 2 question marks - each representing an unknown 2-byte character. Are you sure that the data did not change while your code was never able to process UTF8? It might have been this 4-byte character before: https://en.wikipedia.org/wiki/Enclosed_R ?

Improve this answer

1 Comment

The character in question is 2 bytes in UTF-8, not 4 bytes. The 2 question marks (more accurately, replacement chars) are more likely due to those 2 bytes being misinterpreted individually.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.