The picture looks like it's a custom module, so it might not be a bug in Joomla.
Maybe the module doesn't honour the variable "jform_frontediting" in it's display part. It happend to me too when I used standard includes/renders and forgot to get those variables that might suppress output.
The hiding of front editing on my Joomla sites works well with standard moduls when disabled.
Edit:
I searched for *frontediting* in the Joomla files. There's a file libraries\src\Document\Renderer\Html\ModulesRenderer.php where frontediting is used to generate or not the icons.
So I guess if you use the renderer classes of Joomla in your module, the icons should disappear. Alternatively you can switch them off in CSS, if you don't need them any time.
HTH, I'm not an expert and starting myself to understand the programming of Joomla 4