1

Intro

I'm sending POST request to server which responses with chunked messages. So I'm trying to make writecallback being called on each received chunked http message.

Code

#include <iostream> #include <string> #include <curl/curl.h> using namespace std; size_t write_callback(char *d, size_t n, size_t l, void *userp) { cerr << ""--- Called once" << endl; return n*l; } string xml_msg() { return "<<some request data>>"; } curl_slist* get_header(size_t content_length) { auto list = curl_slist_append(nullptr, "<<protocol version>>"); list = curl_slist_append(list, "Content-Type: text/xml"); list = curl_slist_append(list, "Content-Length: " + content_length); return list; } void main() { auto xml = xml_msg(); curl_global_init(CURL_GLOBAL_ALL); auto curl = curl_easy_init(); curl_easy_setopt(curl, CURLOPT_URL, "<<server url>>"); curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_callback); curl_easy_setopt(curl, CURLOPT_WRITEDATA, nullptr); curl_easy_setopt(curl, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0)"); curl_easy_setopt(curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC); curl_easy_setopt(curl, CURLOPT_USERPWD, "<<user credentials>>"); curl_easy_setopt(curl, CURLOPT_HTTPHEADER, get_header(xml.size())); curl_easy_setopt(curl, CURLOPT_POST, 1); curl_easy_setopt(curl, CURLOPT_POSTFIELDS, xml.data()); curl_easy_setopt(curl, CURLOPT_HTTP_CONTENT_DECODING, 0L); curl_easy_perform(curl); curl_easy_cleanup(curl); curl_global_cleanup(); }

Verbose log

* STATE: INIT => CONNECT handle 0x15c4de0; line 1422 (connection #-5000) * Added connection 0. The cache now contains 1 members * STATE: CONNECT => WAITRESOLVE handle 0x15c4de0; line 1458 (connection #0) * Trying xxx.xxx.xxx.xxx... * TCP_NODELAY set * STATE: WAITRESOLVE => WAITCONNECT handle 0x15c4de0; line 1539 (connection #0) * Connected to <<host>> (xxx.xxx.xxx.xxx) port 80 (#0) * STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x15c4de0; line 1591 (connection #0) * Marked for [keep alive]: HTTP default * STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x15c4de0; line 1605 (connection #0) * STATE: PROTOCONNECT => DO handle 0x15c4de0; line 1626 (connection #0) * Server auth using Basic with user '<<credentials>>' > POST <<URL>> Host: <<host>> Authorization: Basic <<base64 credentials>> User-Agent: Mozilla/4.0 (compatible; MSIE 6.0) Accept: */* Content-Type: text/xml Content-Length: 204 * upload completely sent off: 204 out of 204 bytes * STATE: DO => DO_DONE handle 0x15c4de0; line 1688 (connection #0) * STATE: DO_DONE => WAITPERFORM handle 0x15c4de0; line 1813 (connection #0) * STATE: WAITPERFORM => PERFORM handle 0x15c4de0; line 1823 (connection #0) * HTTP 1.1 or later with persistent connection, pipelining supported < HTTP/1.1 200 OK < Date: Tue, 08 May 2018 12:29:49 GMT * Server is not blacklisted < Server: <<server>> < Expires: Thu, 01 Jan 1970 00:00:00 GMT < Content-Language: en-US < Cache-Control: no-cache, no-store < Pragma: no-cache < Content-Type: application/xml;charset=UTF-8 < Set-Cookie: <<cookie>> < Transfer-Encoding: chunked < --- Called once * STATE: PERFORM => DONE handle 0x15c4de0; line 1992 (connection #0) * multi_done * Connection #0 to <<server>> left intact

Problem

Writecallback has been called when connection had been closed by server due to timeout with FIN tcp packet instead of moment when chunked http response has been received.

It is about 30 secs interval between this events.

Question

What am I doing wrong?

Update 1

Server returns a tcp segment with PUSH flag and http message with chunked transfer encoding containing XML. Message ends with CLRF. Meanwhile Win API socket does not allow to read it and select() returns 0, which means that there is nothing to read/write on this socket.

After 30 secs delay before closing connection due to heartbeat timeout (that is internal implementation of the server), server sends finalizing http message with chunked transfer encoding, which contains 0 and CLRF. After that message select() displays new socket state and libcurl calls write callback with returning chunked message content.

That is what I see after debuging libcurl. I need to find out the way to get chunked http message returned by libcurl once it is received, not after getting final http message.

Improve this question
3
  • As the only time write_callback has been called is when the timeout period is hit, it sounds to me that the web server was not able to process the request at all, rather than the issue being with the client code. Did you confirm that the server returns any response at all? Commented May 9, 2018 at 13:12
  • Yes, server returns tcp segment with PUSH flag containing chunked http message right after the request . That what I see using wireshark. Bur callback is being called when FIN tcp segment is received. BTW wireshark displays HTTP message right before FIN tcp segment as well. Commented May 9, 2018 at 18:25
  • I see that WinApi select returns 0 even when tcp segment with chunked http message is received. Commented May 10, 2018 at 13:25

1 Answer 1

Reset to default
1

Ok, I was able to find out that the problem is with Win Api sockets. On linux builds libcurl calls writecallback right after receiving chuncked message. I'm not sure how to fix that issue with Win builds, but at least I found rootcause of problem.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

This doesn't sound like "the answer", this sounds like some clues into debugging what's actually going on...
I'm a newcomer here. If you can, please make this answer as comment. Meanwhile I want to try build libcurl with USE_LWIPSOCK and check if will fix an isuue with Win sockets. Or this will use Win sockets internally?
Hi newcomer, we don't ask extra questions in comments to answers to other queestions...
I've tried to ask extra question as a new question and it was marked as duplicate :) stackoverflow.com/questions/50279343/…

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.