I ran into the same problem and found the following to be a good enough solution. Try including this snippet in your `olivetti-mode` configuration:
```
(setq olivetti-style 'fancy
 olivetti-margin-width 3)

(defun custom-olivetti-mode-on-hook ()
 (setq-local flycheck-indication-mode 'left-margin))

(defun custom-olivetti-mode-off-hook ()
 (kill-local-variable 'flycheck-indication-mode))

(add-hook 'olivetti-mode-on-hook 'custom-olivetti-mode-on-hook)
(add-hook 'olivetti-mode-off-hook 'custom-olivetti-mode-off-hook)
```
This solves the problem for Flycheck indicators, which are likely the only ones you will see with `olivetti-mode` active.

The bottom buffer below shows the result after enabling `olivetti-mode` (using the above configuration):
[![enter image description here][1]][1]


 [1]: https://i.sstatic.net/YSO9f.png