It looks like the correct answer is easier than expected.
In order to cleanup all dumped core stored by systemd-coredump you can run (as root):
systemd-tmpfiles --clean
You will still get the list of dumped cores with:
coredumpctl
but the dumped cores will be actually gone and the output will tell you that they are missing. Please, refer to the relevant man pages to get more details.
In case you want to completely disable this systemd feature altogether, you can do it via systemctl (as root):
systemctl disable systemd-coredump.socket systemctl stop systemd-coredump.socket systemctl status systemd-coredump.socket systemctl disable systemd-coredump.socket
Be warned that this move won't survive the reboot: systemd will re-enable it at reboot!
To actually make this happen you have to "shadow" that systemd module. Which in turn translates into:
sudo ln -fs /dev/null /etc/sysctl.d/50-coredump.conf
Keep in mind that, at least on my Arch Linux box, my /proc/sys/kernel/core_pattern reads |/bin/false which completely avoids any core dumping by piping the core dump through the /bin/false binary (which does nothing at all).
Please check the relevant man page and your distribution documentation for more details.