Java file encoding detection and converter (UTF-8, ASCII)

Question

I have a simple txt. file and i'm looking to know if there's a way in java to do what, for example, notepad++ does with file enconding. It can detect the encoding of the file (UTF-8, ASCII, UTF-16, ...) and, if we want to, it can convert it to another encoding without transform the special characters like 'ç' or '€' in strange characters.

Thanks.

You need to check for the Byte Order Mark(BOM) msdn.microsoft.com/en-us/library/windows/desktop/… — user1244949
– user1244949, Commented Oct 30, 2015 at 9:34
Thank you for your comment. Yes, there are encodings with the BOM which are easily detected. But, for example, there's UTF-8 and UTF-8 without BOM. And if doesn't have BOM, the problem remains the same. — JCasper
– JCasper, Commented Oct 30, 2015 at 9:39

wero · Accepted Answer · 2016-09-30 16:19:45Z

1

Apache Tika has an EncodingDetector with implementations for different contexts. Typically these implementations use heuristics to determine the charset with some probability. If you are interested in the details you can dive into the source.

edited Sep 30, 2016 at 16:19

answered Oct 30, 2015 at 9:41

wero

33.1k3 gold badges63 silver badges87 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

JCasper Over a year ago

Thank you for your answer! I already wrote a simple code including Apache Tika. But the probability to guess it right it's around 60%. I really wanted to know how notepad does it :\

wero Over a year ago

@JCaspar but this is not your original question. It is a little bit disappointing to make the effort of an answer just to learn that you meant something different. In the end you will need to dive into the sources of notepad++ when you want to know how it is implemented.

JCasper Over a year ago

Of course i don't mean the exact code of notepad++, probably i will never reach it. It was just an outflow regarding the probability of my class, and that's why i mentioned notepad++ in the original question, because it has an high sucess guess rate. The question is the one i posted in the first place, and thank you for your attention.

Community · Accepted Answer · 2017-05-23 11:48:19Z

0

You can do that in java.Already there is an another discussion about this topic on another thread. Best way to convert text files between character sets?

edited May 23, 2017 at 11:48

CommunityBot

11 silver badge

answered Oct 30, 2015 at 9:39

RatheeshTS

4195 silver badges18 bronze badges

1 Comment

RatheeshTS Over a year ago

stackoverflow.com/questions/10093178/…

Collectives™ on Stack Overflow

Java file encoding detection and converter (UTF-8, ASCII)

2 Answers 2

3 Comments

1 Comment

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

1 Comment

Linked

Related