1

I have a weird issue where either Nginx or Apache (I can't tell which - but I'm 90% sure it's Apache - Nginx is reverse proxying) is matching a partial URL to a full, random URL.

For example: https://example.com/php

Is being rewritten/redirected to: https://example.com/php-full-article-url

But as far as I'm aware, I don't have anything in my config's that would result in this behaviour!

Nginx conf:

server { server_name example.com; root /var/www/html/examplecom/; index index.php index.html index.htm; set $skip_cache 0; set $skip_reason ""; proxy_cache_key $scheme$host$request_uri; if ($http_cookie ~* "comment_author_|wordpress_(?!test_cookie)|wp-postpass_" ) { set $skip_cache 1; set $skip_reason Cookie; } if ($request_uri ~* "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|sitemap(_index)?.xml") { set $skip_cache 1; set $skip_reason URI; } add_header Cache-BYPASS-Reason $skip_reason; location / { try_files $uri $uri/ /index.php?$args; proxy_buffering off; proxy_cache edge-cache; proxy_cache_revalidate on; proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504; proxy_cache_bypass $skip_cache; proxy_no_cache $skip_cache; proxy_cache_valid 200 301 302 500m; proxy_cache_valid 404 1m; add_header X-Cache-Status $upstream_cache_status; #add_header X-Handled-By $proxy_host; add_header Cache-BYPASS-Reason $skip_reason; } location ~ \.php$ { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $host; proxy_set_header Host $host; proxy_set_header X-Forwarded-Proto $scheme; proxy_pass http://127.0.0.1:8080$request_uri; add_header Front-End-Https on; proxy_redirect off; proxy_buffering off; proxy_cache edge-cache; proxy_cache_revalidate on; proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504; proxy_cache_bypass $skip_cache; proxy_no_cache $skip_cache; proxy_cache_valid 200 301 302 500m; proxy_cache_valid 404 1m; add_header X-Cache-Status $upstream_cache_status; #add_header X-Handled-By $proxy_host; add_header Cache-BYPASS-Reason $skip_reason; } location ~ /\.ht { deny all; } listen 443 ssl http2; # managed by Certbot #listen [::]:443 ssl http2 ipv6only=off; ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; # managed by Certbot ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; # managed by Certbot #include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot #ssl_ciphers EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5; ssl_ciphers TLS13-CHACHA20-POLY1305-SHA256:TLS13-AES-256-GCM-SHA384:TLS13-AES-128-GCM-SHA256:EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH:EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5; ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot location = /favicon.ico { log_not_found off; access_log off; } location = /robots.txt { log_not_found off; access_log off; allow all; } location ~* .(css|gif|ico|jpeg|jpg|js|png)$ { expires max; log_not_found off; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Host $host; proxy_set_header Host $host; proxy_set_header X-Forwarded-Proto $scheme; proxy_redirect off; proxy_buffering off; proxy_cache edge-cache; proxy_cache_revalidate on; proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504; proxy_cache_bypass $skip_cache; proxy_no_cache $skip_cache; proxy_cache_valid 200 301 302 500m; proxy_cache_valid 404 1m; add_header X-Cache-Status $upstream_cache_status; add_header Cache-BYPASS-Reason $skip_reason; proxy_pass http://127.0.0.1:8080$request_uri; } } server { if ($host = example.com) { return 301 https://$host$request_uri; } # managed by Certbot if ($host = www.example.com) { return 301 https://example.com$request_uri; } if ($host = dev.example.com) { return 301 https://example.com$request_uri; } # managed by Certbot listen 80; server_name example.com; return 404; # managed by Certbot } 

Apache conf:

<VirtualHost 127.0.0.1:8080> #ServerName example.com ServerAdmin webmaster@localhost DocumentRoot /var/www/html/examplecom/ #LogLevel info ssl:warn ErrorLog ${APACHE_LOG_DIR}/error.log CustomLog ${APACHE_LOG_DIR}/access.log combined #Include conf-available/serve-cgi-bin.conf Header unset ETag FileETag None CacheQuickHandler off #CacheRoot /var/cache/apache2/mod_cache_disk #CacheEnable disk / #CacheDirLevels 2 #CacheDirLength 1 #CacheMaxFileSize 2000000 #CacheIgnoreNoLastMod On #CacheDefaultExpire 7200 #CacheIgnoreCacheControl On #CacheLastModifiedFactor 0.5 CacheIgnoreHeaders Set-Cookie Cookie #CacheHeader on #CacheLock on CacheDisable /wp-admin CacheDisable /wp-login.php CacheDisable /wp-cron.php #SetOutputFilter CACHE <FilesMatch \.php$> SetHandler "proxy:unix:/var/run/php/php7.2-fpm.sock|fcgi://localhost/" </FilesMatch> </VirtualHost> 

Wordpress .htaccess:

# BEGIN W3TC CDN <IfModule mod_headers.c> Header set Access-Control-Allow-Origin "*" </IfModule> # END W3TC CDN # BEGIN W3TC Browser Cache <IfModule mod_mime.c> AddType text/css .css AddType text/x-component .htc AddType application/x-javascript .js AddType application/javascript .js2 AddType text/javascript .js3 AddType text/x-js .js4 AddType text/html .html .htm AddType text/richtext .rtf .rtx AddType image/svg+xml .svg AddType text/plain .txt AddType text/xsd .xsd AddType text/xsl .xsl AddType text/xml .xml AddType video/asf .asf .asx .wax .wmv .wmx AddType video/avi .avi AddType image/bmp .bmp AddType application/java .class AddType video/divx .divx AddType application/msword .doc .docx AddType application/vnd.ms-fontobject .eot AddType application/x-msdownload .exe AddType image/gif .gif AddType application/x-gzip .gz .gzip AddType image/x-icon .ico AddType image/jpeg .jpg .jpeg .jpe AddType image/webp .webp AddType application/json .json AddType application/vnd.ms-access .mdb AddType audio/midi .mid .midi AddType video/quicktime .mov .qt AddType audio/mpeg .mp3 .m4a AddType video/mp4 .mp4 .m4v AddType video/mpeg .mpeg .mpg .mpe AddType video/webm .webm AddType application/vnd.ms-project .mpp AddType application/x-font-otf .otf AddType application/vnd.ms-opentype ._otf AddType application/vnd.oasis.opendocument.database .odb AddType application/vnd.oasis.opendocument.chart .odc AddType application/vnd.oasis.opendocument.formula .odf AddType application/vnd.oasis.opendocument.graphics .odg AddType application/vnd.oasis.opendocument.presentation .odp AddType application/vnd.oasis.opendocument.spreadsheet .ods AddType application/vnd.oasis.opendocument.text .odt AddType audio/ogg .ogg AddType application/pdf .pdf AddType image/png .png AddType application/vnd.ms-powerpoint .pot .pps .ppt .pptx AddType audio/x-realaudio .ra .ram AddType image/svg+xml .svg .svgz AddType application/x-shockwave-flash .swf AddType application/x-tar .tar AddType image/tiff .tif .tiff AddType application/x-font-ttf .ttf .ttc AddType application/vnd.ms-opentype ._ttf AddType audio/wav .wav AddType audio/wma .wma AddType application/vnd.ms-write .wri AddType application/font-woff .woff AddType application/font-woff2 .woff2 AddType application/vnd.ms-excel .xla .xls .xlsx .xlt .xlw AddType application/zip .zip </IfModule> <IfModule mod_expires.c> ExpiresActive On ExpiresByType text/css A31536000 ExpiresByType text/x-component A31536000 ExpiresByType application/x-javascript A31536000 ExpiresByType application/javascript A31536000 ExpiresByType text/javascript A31536000 ExpiresByType text/x-js A31536000 ExpiresByType text/html A3600 ExpiresByType text/richtext A3600 ExpiresByType image/svg+xml A3600 ExpiresByType text/plain A3600 ExpiresByType text/xsd A3600 ExpiresByType text/xsl A3600 ExpiresByType text/xml A3600 ExpiresByType video/asf A31536000 ExpiresByType video/avi A31536000 ExpiresByType image/bmp A31536000 ExpiresByType application/java A31536000 ExpiresByType video/divx A31536000 ExpiresByType application/msword A31536000 ExpiresByType application/vnd.ms-fontobject A31536000 ExpiresByType application/x-msdownload A31536000 ExpiresByType image/gif A31536000 ExpiresByType application/x-gzip A31536000 ExpiresByType image/x-icon A31536000 ExpiresByType image/jpeg A31536000 ExpiresByType image/webp A31536000 ExpiresByType application/json A31536000 ExpiresByType application/vnd.ms-access A31536000 ExpiresByType audio/midi A31536000 ExpiresByType video/quicktime A31536000 ExpiresByType audio/mpeg A31536000 ExpiresByType video/mp4 A31536000 ExpiresByType video/mpeg A31536000 ExpiresByType video/webm A31536000 ExpiresByType application/vnd.ms-project A31536000 ExpiresByType application/x-font-otf A31536000 ExpiresByType application/vnd.ms-opentype A31536000 ExpiresByType application/vnd.oasis.opendocument.database A31536000 ExpiresByType application/vnd.oasis.opendocument.chart A31536000 ExpiresByType application/vnd.oasis.opendocument.formula A31536000 ExpiresByType application/vnd.oasis.opendocument.graphics A31536000 ExpiresByType application/vnd.oasis.opendocument.presentation A31536000 ExpiresByType application/vnd.oasis.opendocument.spreadsheet A31536000 ExpiresByType application/vnd.oasis.opendocument.text A31536000 ExpiresByType audio/ogg A31536000 ExpiresByType application/pdf A31536000 ExpiresByType image/png A31536000 ExpiresByType application/vnd.ms-powerpoint A31536000 ExpiresByType audio/x-realaudio A31536000 ExpiresByType image/svg+xml A31536000 ExpiresByType application/x-shockwave-flash A31536000 ExpiresByType application/x-tar A31536000 ExpiresByType image/tiff A31536000 ExpiresByType application/x-font-ttf A31536000 ExpiresByType application/vnd.ms-opentype A31536000 ExpiresByType audio/wav A31536000 ExpiresByType audio/wma A31536000 ExpiresByType application/vnd.ms-write A31536000 ExpiresByType application/font-woff A31536000 ExpiresByType application/font-woff2 A31536000 ExpiresByType application/vnd.ms-excel A31536000 ExpiresByType application/zip A31536000 </IfModule> <IfModule mod_brotli.c> <IfModule mod_filter.c> AddOutputFilterByType BROTLI_COMPRESS text/css text/x-component application/x-javascript application/javascript text/javascript text/x-js text/html text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/bmp application/java application/msword application/vnd.ms-fontobject application/x-msdownload image/x-icon image/webp application/json application/vnd.ms-access video/webm application/vnd.ms-project application/x-font-otf application/vnd.ms-opentype application/vnd.oasis.opendocument.database application/vnd.oasis.opendocument.chart application/vnd.oasis.opendocument.formula application/vnd.oasis.opendocument.graphics application/vnd.oasis.opendocument.presentation application/vnd.oasis.opendocument.spreadsheet application/vnd.oasis.opendocument.text audio/ogg application/pdf application/vnd.ms-powerpoint image/svg+xml application/x-shockwave-flash image/tiff application/x-font-ttf application/vnd.ms-opentype audio/wav application/vnd.ms-write application/font-woff application/font-woff2 application/vnd.ms-excel <IfModule mod_mime.c> # BROTLI_COMPRESS by extension AddOutputFilter BROTLI_COMPRESS js css htm html xml </IfModule> </IfModule> </IfModule> <IfModule mod_deflate.c> <IfModule mod_filter.c> AddOutputFilterByType DEFLATE text/css text/x-component application/x-javascript application/javascript text/javascript text/x-js text/html text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/bmp application/java application/msword application/vnd.ms-fontobject application/x-msdownload image/x-icon image/webp application/json application/vnd.ms-access video/webm application/vnd.ms-project application/x-font-otf application/vnd.ms-opentype application/vnd.oasis.opendocument.database application/vnd.oasis.opendocument.chart application/vnd.oasis.opendocument.formula application/vnd.oasis.opendocument.graphics application/vnd.oasis.opendocument.presentation application/vnd.oasis.opendocument.spreadsheet application/vnd.oasis.opendocument.text audio/ogg application/pdf application/vnd.ms-powerpoint image/svg+xml application/x-shockwave-flash image/tiff application/x-font-ttf application/vnd.ms-opentype audio/wav application/vnd.ms-write application/font-woff application/font-woff2 application/vnd.ms-excel <IfModule mod_mime.c> # DEFLATE by extension AddOutputFilter DEFLATE js css htm html xml </IfModule> </IfModule> </IfModule> <FilesMatch "\.(css|htc|less|js|js2|js3|js4|CSS|HTC|LESS|JS|JS2|JS3|JS4)$"> FileETag MTime Size <IfModule mod_headers.c> Header unset Set-Cookie </IfModule> </FilesMatch> <FilesMatch "\.(html|htm|rtf|rtx|svg|txt|xsd|xsl|xml|HTML|HTM|RTF|RTX|SVG|TXT|XSD|XSL|XML)$"> FileETag MTime Size <IfModule mod_headers.c> Header append Vary User-Agent env=!dont-vary </IfModule> </FilesMatch> <FilesMatch "\.(asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|eot|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|webp|json|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|webm|mpp|otf|_otf|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|png|pot|pps|ppt|pptx|ra|ram|svg|svgz|swf|tar|tif|tiff|ttf|ttc|_ttf|wav|wma|wri|woff|woff2|xla|xls|xlsx|xlt|xlw|zip|ASF|ASX|WAX|WMV|WMX|AVI|BMP|CLASS|DIVX|DOC|DOCX|EOT|EXE|GIF|GZ|GZIP|ICO|JPG|JPEG|JPE|WEBP|JSON|MDB|MID|MIDI|MOV|QT|MP3|M4A|MP4|M4V|MPEG|MPG|MPE|WEBM|MPP|OTF|_OTF|ODB|ODC|ODF|ODG|ODP|ODS|ODT|OGG|PDF|PNG|POT|PPS|PPT|PPTX|RA|RAM|SVG|SVGZ|SWF|TAR|TIF|TIFF|TTF|TTC|_TTF|WAV|WMA|WRI|WOFF|WOFF2|XLA|XLS|XLSX|XLT|XLW|ZIP)$"> FileETag MTime Size <IfModule mod_headers.c> Header unset Set-Cookie </IfModule> </FilesMatch> <FilesMatch "\.(bmp|class|doc|docx|eot|exe|ico|webp|json|mdb|webm|mpp|otf|_otf|odb|odc|odf|odg|odp|ods|odt|ogg|pdf|pot|pps|ppt|pptx|svg|svgz|swf|tif|tiff|ttf|ttc|_ttf|wav|wri|woff|woff2|xla|xls|xlsx|xlt|xlw|BMP|CLASS|DOC|DOCX|EOT|EXE|ICO|WEBP|JSON|MDB|WEBM|MPP|OTF|_OTF|ODB|ODC|ODF|ODG|ODP|ODS|ODT|OGG|PDF|POT|PPS|PPT|PPTX|SVG|SVGZ|SWF|TIF|TIFF|TTF|TTC|_TTF|WAV|WRI|WOFF|WOFF2|XLA|XLS|XLSX|XLT|XLW)$"> <IfModule mod_headers.c> Header unset Last-Modified </IfModule> </FilesMatch> <IfModule mod_headers.c> Header set Referrer-Policy "no-referrer-when-downgrade" </IfModule> # END W3TC Browser Cache # BEGIN W3TC Page Cache core <IfModule mod_rewrite.c> RewriteEngine On RewriteBase / RewriteCond %{HTTPS} =on RewriteRule .* - [E=W3TC_SSL:_ssl] RewriteCond %{SERVER_PORT} =443 RewriteRule .* - [E=W3TC_SSL:_ssl] RewriteCond %{HTTP:X-Forwarded-Proto} =https [NC] RewriteRule .* - [E=W3TC_SSL:_ssl] RewriteCond %{HTTP:Accept-Encoding} br RewriteRule .* - [E=W3TC_ENC:_br] RewriteCond %{HTTP_COOKIE} w3tc_preview [NC] RewriteRule .* - [E=W3TC_PREVIEW:_preview] RewriteCond %{REQUEST_METHOD} !=POST RewriteCond %{QUERY_STRING} ="" RewriteCond %{REQUEST_URI} \/$ RewriteCond %{HTTP_COOKIE} !(comment_author|wp\-postpass|w3tc_logged_out|wordpress_logged_in|wptouch_switch_toggle) [NC] RewriteCond "%{DOCUMENT_ROOT}/wp-content/cache/page_enhanced/%{HTTP_HOST}/%{REQUEST_URI}/_index%{ENV:W3TC_SSL}%{ENV:W3TC_PREVIEW}.html%{ENV:W3TC_ENC}" -f RewriteRule .* "/wp-content/cache/page_enhanced/%{HTTP_HOST}/%{REQUEST_URI}/_index%{ENV:W3TC_SSL}%{ENV:W3TC_PREVIEW}.html%{ENV:W3TC_ENC}" [L] RewriteCond %{REQUEST_METHOD} !=POST RewriteCond %{QUERY_STRING} ="" RewriteCond %{REQUEST_URI} \/$ RewriteCond %{HTTP_COOKIE} !(comment_author|wp\-postpass|w3tc_logged_out|wordpress_logged_in|wptouch_switch_toggle) [NC] RewriteCond "%{DOCUMENT_ROOT}/wp-content/cache/page_enhanced/%{HTTP_HOST}/%{REQUEST_URI}/_index%{ENV:W3TC_SSL}%{ENV:W3TC_PREVIEW}.xml%{ENV:W3TC_ENC}" -f RewriteRule .* "/wp-content/cache/page_enhanced/%{HTTP_HOST}/%{REQUEST_URI}/_index%{ENV:W3TC_SSL}%{ENV:W3TC_PREVIEW}.xml%{ENV:W3TC_ENC}" [L] </IfModule> # END W3TC Page Cache core # BEGIN WordPress <IfModule mod_rewrite.c> RewriteEngine On RewriteBase / RewriteRule ^index\.php$ - [L] RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule . /index.php [L] </IfModule> # END WordPress 

Any help here would be GREATLY appreciated. https://example.com/php should not be rewriting to https://example.com/php-full-article-url - because it's not a legitimate match/binding/whatever it's called.

I have just checked my Apache access logs: 127.0.0.1 - - [21/Feb/2019:14:42:14 +0000] "GET /php HTTP/1.0" 301 - so it is doing a 301 redirect within Apache!!! I just don't know why

I've cleared Nginx caching, restarted both Apache and Nginx (and PHP for good measure).

Happy to give some real world example url's if that helps? I have a set of posts that all start /review- however if I just visit /review (expecting to get a 404 error - which is exactly correct!), it seems to just pick a random post and redirects/rewrites to serve that up instead (200 OK!)

Note: Originally asked on SO (https://stackoverflow.com/questions/54809631/) but got no responses - I feel it's probably more appropriate here

4
  • You have a "front controller" rewrite rule for WordPress: RewriteRule . /index.php [L] This assigns every URL that isn't an actual file or directory to be handled by WordPress. It is WordPress itself that is doing this, not your config files. Commented Feb 22, 2019 at 10:23
  • Thanks @StephenOstermiller - is this standard wordpress behaviour? I'd never noticed this before, but it seems... "dangerous" to me to "match" part of a url to a full resource. For instance, I have a lot of posts that start /review-, but if I just visit example.com/review - I get one random post. If these were products (let's say /blue where I have /blue-tshirt, /blue-trousers & /blue-jumper), how does WP decide which page to use? Surely in that scenario, if there are multiple matches it should just be forwarding to example.com/?s=blue not picking a single page?? Commented Feb 22, 2019 at 12:12
  • The front controller is standard WordPress stuff (almost all content management systems do it.) I know that WordPress has some settings for how your URLs should look, but I'm not well versed in them. Ideally it would show a 404 error or redirect for URLs that aren't in exactly the right format. I'm not sure if there are plugins that would help make that happen. Commented Feb 22, 2019 at 12:40
  • Thanks @StephenOstermiller - do you want to pop that in as an answer and I'll accept it? :) Commented Feb 22, 2019 at 14:14

1 Answer 1

1

Per @StephenOstermiller's comments - this is actually something WordPress is handling (albeit, not very well).

It's basically guessing the full URI by picking a post/page that happens to have the same word in it!

I asked the question on WP's support forum and as it turns out, fortunately, there's a very easy way to disable this behaviour. One of the forum mod's gave me the following resource: https://www.bloggersignal.com/stop-wordpress-from-guessing-urls/

TL;DR - if you want to retain maximum SEO friendliness, pop the following in your functions.php

function wmse_121146_no_redirect_404($redirect_url) { if ( is_404() ) { return false; } return $redirect_url; } add_filter('redirect_canonical', 'wmse_121146_no_redirect_404'); 

See WordPress forum question: https://wordpress.org/support/topic/wordpress-front-controller-redirecting-partial-urls

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.