Both .htaccess and robots.txt are a part of Drupal core, they are called "Scaffold Files". When you do composer update it will download the repository version of these files.
You have to re-add your custom changes every time you do composer update. Fortunately there is an automated way to do this:
Option 1. Append your changes:
"name": "my/project", ... "extra": { "drupal-scaffold": { "file-mapping": { "[web-root]/robots.txt": { "append": "assets/my-robots-additions.txt", } } } } Option 2. Create patches:
"name": "my/project", ... "scripts": { "post-drupal-scaffold-cmd": [ "cd docroot && patch -p1 <../patches/htaccess-ssl.patch" ] } Option 3. Exclude (not recommended):
The official example:
"name": "my/project", ... "extra": { "drupal-scaffold": { "file-mapping": { "[web-root]/robots.txt": false } } } But this (option 3) approach is not recommended, because you might miss important core changes/security fixes.
The code, that you provided looks correct.
Make sure that you are using the https://github.com/drupal/core-composer-scaffold package and not the outdated https://github.com/drupal-composer/drupal-scaffold.
But this approach is not recommended, because you might miss important core changes/security fixes.
More details on Altering Scaffold Files.