when I spide a website ,I got a lot of bad url like these. http://example.com/../../.././././1.htm http://example.com/test/../test/.././././1.htm http://example.com/.//1.htm http://example.com/../test/..//1.htm
all of these should be http://example.com/1.htm. how to use PHP codes to do this ,thanks.
PS: I use http://snoopy.sourceforge.net/ I get a lot of repeated link in my database , the 'http://example.com/../test/..//1.htm' should be 'http://example.com/1.htm' .
You could do it like this, assuming all the urls you have provided are expected tobe http://example.com/1.htm:
$test = array('http://example.com/../../../././.\./1.htm', 'http://example.com/test/../test/../././.\./1.htm', 'http://example.com/.//1.htm', 'http://example.com/../test/..//1.htm'); foreach ($test as $url){ $u = parse_url($url); $path = $u['scheme'].'://'.$u['host'].'/'.basename($u['path']); echo $path.'<br />'.PHP_EOL; } /* result http://example.com/1.htm<br /> http://example.com/1.htm<br /> http://example.com/1.htm<br /> http://example.com/1.htm<br /> */ //or as a function @lpc2138 function getRealUrl($url){ $u = parse_url($url); $path = $u['scheme'].'://'.$u['host'].'/'.basename($u['path']); $path .= (!empty($u['query'])) ? '?'.$u['query'] : ''; return $path; } 
http://www.example.com/datasheet/ds11015.html should be http://www.example.com/datasheet/ds11015.html
You seem to be looking for a algorithm to remove the dot segments:
function remove_dot_segments($abspath) { $ib = $abspath; $ob = ''; while ($ib !== '') { if (substr($ib, 0, 3) === '../') { $ib = substr($ib, 3); } else if (substr($ib, 0, 2) === './') { $ib = substr($ib, 2); } else if (substr($ib, 0, 2) === '/.' && ($ib[2] === '/' || strlen($ib) === 2)) { $ib = '/'.substr($ib, 3); } else if (substr($ib, 0, 3) === '/..' && ($ib[3] === '/' || strlen($ib) === 3)) { $ib = '/'.substr($ib, 4); $ob = substr($ob, 0, strlen($ob)-strlen(strrchr($ob, '/'))); } else if ($ib === '.' || $ib === '..') { $ib = ''; } else { $pos = strpos($ib, '/', 1); if ($pos === false) { $ob .= $ib; $ib = ''; } else { $ob .= substr($ib, 0, $pos); $ib = substr($ib, $pos); } } } return $ob; } This removes the . and .. segments. Any removal of any other segment like an empty one (//) or .\. is not as per standard as it changes the semantics of the path.
You could do some fancy regex but this works just fine.
fixUrl('http://example.com/../../../././.\./1.htm'); function fixUrl($str) { $str = str_replace('../', '', $str); $str = str_replace('./', '', $str); $str = str_replace('\.', '', $str); return $str; } 
$str = str_replace(array('../','./','\.'), '', $str);http://example.com/../test/..//1.htm
/test/../ into /)
/.\./and//both become/? They are semantically different.