Curl get header with get not head request

Question

okey I made a script checking if file exist but the header I got from Curl is bad

 $curl = curl_init( $url ); curl_setopt( $curl, CURLOPT_NOBODY, true ); curl_setopt( $curl, CURLOPT_HEADER, true ); curl_setopt( $curl, CURLOPT_RETURNTRANSFER, true ); curl_setopt( $curl, CURLOPT_FOLLOWLOCATION, true ); curl_setopt( $curl, CURLOPT_FRESH_CONNECT, true); $data = curl_exec( $curl ); var_dump($data);

and it gets me response 200 while @file_headers() gets me boolean false, which is right file cant be downloaded (connection reset by peer)

i'd use file_headers but its muuuch slower than curl (checking about 10k files)

any idea how to fix curl to force it showing it correctly if file exist?

FYI curl_getinfo($curl, CURLINFO_HTTP_CODE); shows 200 too

*I mean if file is reset by peer, because cURL detects if file exist flawlessly (making Head request) while get_headers() use get request by default, is there a way to force cURL make get request?

after adding curl_setopt( $curl, CURLOPT_HTTPGET,true); When i log curl I see: * Empty reply from server, while getting "reset by peer" file but this is downloading my whole file

ok I think Ive made workaround adding curl_setopt( $curl, CURLOPT_RANGE, "1-1"); and downloading 1 byte instead getting header and if false then file does not exist or reset by peer

Community · Accepted Answer · 2017-05-23 12:14:00Z

0

The reset happens after de http status, so yes, 200 status is possible.

The error seems a forbid or slow handling downloads.

Do this:

Specify an user agent in your curl request
Set a (random) waiting time between the requests

See the following for examples: PHP cURL how to add the User Agent value OR overcome the Servers blocking cURL requests?

edited May 23, 2017 at 12:14

CommunityBot

11 silver badge

answered Aug 6, 2015 at 9:18

schellingerht

5,8062 gold badges30 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

klystianek Over a year ago

no I just simplified the code here, I got useragent added, and the reset by peer is right answer for this file, I just want to catch this kind of error by cURL, im testing one file only Using curl I get: HTTP/1.1 200 OK Server: nginx Date: Thu, 06 Aug 2015 09:25:05 GMT Content-Type: application/octet-stream Content-Length: 2147483648 Last-Modified: Sun, 10 May 2015 23:05:19 GMT Connection: close and I sould catch error instead of 200

schellingerht Over a year ago

Like I said, connection reset has no effect for http status. Secondly, a http status 200 is no guarantee that the file exists ;-)

klystianek Over a year ago

so how to check it properly by curl if file exist (even in cases where connection is reset by peer) ?? file_headers() answer is good for me I can catch this error with ease, but it is slow as hell, cURLing is about 20-30% faster for me. now I realized that my main question should be different :)

klystianek Over a year ago

wait again my words are wrong, cURL actually detects properly if file exist but does not detect if connection is reset by peer, how to detect this error?

schellingerht Over a year ago

Check this: stackoverflow.com/questions/981954/…

|

Collectives™ on Stack Overflow

Curl get header with get not head request

1 Answer 1

6 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Linked

Related