2

I am having trouble with nordic characters (å, ä and ö) not rendering correctly in Emacs. In text files, they render correctly, but in dired and other non-file buffers (like counsel-find-file) they instead render as o¨ instead of ö, etc. See the picture below.

enter image description here

I have the following settings related to character encoding in my init.el file:

(prefer-coding-system 'utf-8) (set-default-coding-systems 'utf-8) (set-terminal-coding-system 'utf-8) (set-keyboard-coding-system 'utf-8)

My LANG is set to en_US.UTF-8 and LC_ALL is en_IE.UTF-8 in my .zshrc.

System information

  • OS: macOS Sonoma 14.2
  • Emacs version: 29.1 (railwaycat's Emacs Mac Port)

What I've tried

  • Changing LC_ALL to sv_SE.UTF-8
  • The official GNU Emacs for macOS & d12frosted's Emacs Plus

Update

As per @NickD's comment, here is the help information for the o character in the file name:

 position: 898 of 983 (91%), column: 3 character: o (displayed as o) (codepoint 111, #o157, #x6f) charset: ascii (ASCII (ISO646 IRV)) code point in charset: 0x6F script: latin syntax: w which means: word category: .:Base, L:Strong L2R, a:ASCII, l:Latin, r:Roman to input: type "C-x 8 RET 6f" or "C-x 8 RET LATIN SMALL LETTER O" buffer code: #x6F file code: #x6F (encoded by coding system utf-8) display: by this font (glyph code): mac-ct:-*-Ubuntu Mono-medium-normal-normal-*-20-*-*-*-m-0-iso10646-1 (#x52)

And here is the ö character inside a file:

 position: 101 of 13753 (1%), column: 0 character: ö (displayed as ö) (codepoint 246, #o366, #xf6) charset: unicode (Unicode (ISO10646)) code point in charset: 0xF6 script: latin syntax: w which means: word category: .:Base, L:Strong L2R, j:Japanese, l:Latin to input: type "C-x 8 RET f6" or "C-x 8 RET LATIN SMALL LETTER O WITH DIAERESIS" buffer code: #xC3 #xB6 file code: #xC3 #xB6 (encoded by coding system utf-8-unix) display: by this font (glyph code): mac-ct:-*-Ubuntu Mono-medium-normal-normal-*-20-*-*-*-m-0-iso10646-1 (#xD1)
Improve this question
4
  • 2
    Place your cursor on the o of sökmotorer (the first one, the one that's supposed to have the umlaut). Do C-u C-x = (or M-x describe-char) and edit your question to add the resulting *Help* screen. I suspect it's a font problem: have you tried a different font? Commented Dec 20, 2023 at 16:55
  • What @NickD said. Sounds like your font can't display those chars. Commented Dec 20, 2023 at 17:01
  • It seems to be the font, thanks for the help! I do not understand, however, why the characters can be displayed correctly in files? Commented Dec 20, 2023 at 17:43
  • Check the font that is used there: open a file with a ö, place the cursor there and say C-u C-x =. Add it it to your question, so we can check for differences. Commented Dec 20, 2023 at 20:25

2 Answers 2

Reset to default
4

The fact that you could paste the characters in to your question and have them show up exactly the same as in your screenshot shows us exactly what is going on.

Unicode has two diaeresis characters. One of them combines with the previous character in order to accent it, while the other takes up space as a character all of its own. Those filenames have the U+00A8 SPACING DIAERESIS character in them, while your other buffers where the character shows up as you wanted it to have the U+308 COMBINING DIAERESIS character instead. Emacs is not rendering anything incorrectly, and there is no problem with your fonts. You just need to rename the files.

As an aside, you should also clean up your Emacs settings. Not only is UTF-8 the default encoding, but setting those four variables leaves several other related variables on their default values. If you were setting these variables to something other than their defaults, that would leave the others with potentially conflicting, or at least confusing, values. You can just delete all four of those lines from your config. If you really want to have something explicit in your config file, use (set-language-environment "UTF-8") instead.

Improve this answer
3
  • Thanks for your answer! I tried renaming the files, which works until I restart my computer, after which it reverts to o¨. Also, thanks for pointing that out regarding my Emacs settings, I am now using the setting you recommended. Commented Dec 21, 2023 at 19:43
  • 1
    That sounds like a problem with OSX to me. Or perhaps you have some other program that modifies these files? In principle, either could be renormalizing the names. You might also try using the precomposed character “ö” (U+00F6 LATIN SMALL LETTER O WITH DIAERESIS) instead. You should definitely consult the OSX documentation to see what sort of normalization it does, and when (renormalizing only after a reboot and not when the file is created or renamed is actually pretty weird, now that you mention it). Commented Dec 22, 2023 at 0:58
  • The only other program I could think of that might modify the files is Syncthing. I have never had this happen before I started using Emacs though, and I've been using Syncthing for over six months. Commented Dec 22, 2023 at 7:19
2

As @db48x says, the core problem is in the input method you use. Instead of creating precomposed Unicode characters, you keep producing the decomposed character pairs. This should not happen if your macOS keyboard language is set up to Swedish.

I usually run into this problem when I copy text from a browser or some other text source like a PDF. The text has been converted to that format and the conversion does the sloppy job of coding the Scandinavian characters using decomposed Unicode characters.

As a quick fix, here is a naive brute force emacs-lisp function I use to fix those texts:

(defun my-fix-umlauts (@begin @end) "Replace 「ä」and 「å」 and 「ö」 by 「ä」 and 「å」and 「ö」 in the current line or selection." (interactive (if (use-region-p) (list (region-beginning) (region-end)) (list (line-beginning-position) (line-end-position)))) (save-excursion (save-restriction (narrow-to-region @begin @end) (goto-char (point-min)) (while (re-search-forward "ä" nil t) (replace-match "ä" "FIXEDCASE")) (goto-char (point-min)) (while (re-search-forward "ö" nil t) (replace-match "ö" "FIXEDCASE")) (goto-char (point-min)) (while (re-search-forward "å" nil t) (replace-match "å" "FIXEDCASE")))))
Improve this answer
2
  • 1
    P.S. Use wdired-mode to fix those filenames with this function directly within emacs (gnu.org/software/emacs/manual/html_node/emacs/Wdired.html) Commented Dec 22, 2023 at 4:24
  • Thanks for the help. I did have my input language set to Swerty (johanegustafsson.net/projects/swerty), but after changing it to the built-in Swedish input language, renaming the file, and restarting, it still reverts to o¨. Commented Dec 22, 2023 at 7:15

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.