NSURLCache: inconsistent behaviour

Question

I was observirng some strange behaviour of my app sometime caching responses and sometime not caching them (all the responses have Cache-Control: max-age=600).

The test is simple: I did a test.php script that was just setting the headers and returning a simple JSON:

<?php header('Content-Type: application/json'); header('Cache-Control: max-age=600'); ?> { "result": { "employeeId": "<?php echo $_GET['eId']; ?>", "dateTime": "<?php echo date('Y-m-d H:i:s'); ?>'" } }

This is the response I get from the PHP page:

HTTP/1.1 200 OK Date: Thu, 28 Nov 2013 11:41:55 GMT Server: Apache X-Powered-By: PHP/5.3.17 Cache-Control: max-age=600 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Transfer-Encoding: chunked Content-Type: application/json { "result": { "employeeId": "", "dateTime": "2013-11-28 11:41:55'" } }

Then I've created a simple app and added AFNetworking library.

When I call the script with few parameters, the cache works properly:

AFHTTPRequestOperationManager *manager = [AFHTTPRequestOperationManager manager]; NSDictionary *params = @{ @"oId": @"4011", @"eId": self.firstTest ? @"1" : @"0", @"status": @"2031", }; [manager GET:@"http://www.mydomain.co.uk/test.php" parameters:params success:^(AFHTTPRequestOperation *operation, id responseObject) { NSLog(@"JSON: %@", responseObject); NSLog(@"Cache current memory usage (after call): %d", [cache currentMemoryUsage]); } failure:^(AFHTTPRequestOperation *operation, NSError *error) { NSLog(@"Error: %@", error); }];

But when I increase the number of parameters, like:

NSDictionary *params = @{ @"organizationId": @"4011", @"organizationId2": @"4012", @"organizationId3": @"4013", @"organizationId4": @"4014", @"organizationId5": @"4015", @"organizationId6": @"4016", @"eId": self.firstTest ? @"1" : @"0", @"status": @"2031", };

it doesn't work anymore and it execute a new request each time it is called.

I've done many tests and it seems to me that it is related to the length of the URL, because if I includes this set of params:

NSDictionary *params = @{ @"oId": @"4011", @"oId2": @"4012", @"oId3": @"4013", @"oId4": @"4014", @"oId5": @"4015", @"oId6": @"4016", @"eId": self.firstTest ? @"1" : @"0", @"status": @"2031", };

It works!!

I've done many tests and that's the only pattern I've found...

To exclude AFNetworking from the equation, I've created another test program that uses NSURLConnection only and I can see the same behaviour so it's not AFNetworking and definitely NSURLCache. This is the other test:

NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:@"http://www.mydomain.co.uk/test.php?eId=%@&organizationId=4011&organizationId2=4012&organizationId3=4013&organizationId4=4014&organizationId5=4015&organizationId6=4016", self.firstTest ? @"1" : @"0"]]; // doesn't work //NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:@"http://www.mydomain.co.uk/test.php?eId=%@&oId=4011&oId2=4012&oId3=4013&oId4=4014&oId5=4015&oId6=4016", self.firstTest ? @"1" : @"0"]]; // work //NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:@"http://www.mydomain.co.uk/test.php?eId=%@", self.firstTest ? @"1" : @"0"]]; // work NSURLRequest *request = [NSURLRequest requestWithURL:url]; NSURLResponse *response = nil; NSError *error = nil; NSData *data = [NSURLConnection sendSynchronousRequest:request returningResponse:&response error:&error]; if (error == nil) { // Parse data here NSString *responseDataStr = [NSString stringWithUTF8String:[data bytes]]; NSLog(@"Response data: %@", responseDataStr); }

I've also tried to establish how many characters in the URL will trigger the problem but even in this case I've got strange results:

This one is 112 characters long and it doesn't work:

http://www.mydomain.co.uk/test.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&orgaId4=4

This one is 111 characters long and it works:

http://www.mydomain.co.uk/test.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&orgId4=4

Ive renamed the PHP script to see if the first part of the URL would matter and I've got a strange behaviour again:

This one is 106 characters long and it doesn't work:

http://www.mydomain.co.uk/t.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&org=40

This one is 105 characters long and it works:

http://www.mydomain.co.uk/t.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&org=4

So I've removed 3 characters from the page name and I've got a working threshold 6 characters lower.

Any suggestion?

Thanks, Dem

@EugeneProkoshev thanks, I've added the complete RAW response that I get from the PHP page — demetrio812
– demetrio812, Commented Nov 28, 2013 at 11:46

m_katsifarakis · Accepted Answer · 2015-08-31 17:05:27Z

I am witnessing something similar with certain responses not being cached by NSURLCache and I have come up with another possible reason:

In my case I have been able to ascertain that the responses not being cached are the ones that are returned using Chunked transfer-encoding. I've read elsewhere that NSURLCache should cache those after iOS 6 but for some reason it doesn't in my case (iOS 7.1 and 8.1).

I see that your example response shown here, also has the Transfer-Encoding: chunked header.

Could it be that some of your responses are returned with chunked encoding (those that are not cached) and some are not (those that are cached)?

My back-end is also running PHP on Apache and I still can't figure out why it does that... Probably some Apache extension...

Anyway, I think it sounds more plausible than the request URL length scenario.

EDIT:

It's been a while, but I can finally confirm that in our case, it is the chunked transfer encoding that causes the response not to be cached. I have tested that with iOS 7.1, 8.1, 8.3 and 8.4.

Since I understand that it is not always easy to change that setting on your server, I have a solution to suggest, for people who are using AFNetworking 2 and subclassing AFHTTPSessionManager.

You could add your sub-class as an observer for AFNetworking's AFNetworkingTaskDidCompleteNotification, which contains all the things you will need to cache the responses yourself. That means: the session data task, the response object and the response data before it has been processed by the response serializer.

If your server uses chunked encoding for only a few of its responses, you could add code in -(void)didCompleteTask: to only cache responses selectively. So for example you could check for the transfer-encoding response header, or cache the response based on other criteria.

The example HTTPSessionManager sub-class below caches all responses that return any data:

MyHTTPSessionManager.h

@interface MyHTTPSessionManager : AFHTTPSessionManager @end

MyHTTPSessionManager.m

#import "MyHTTPSessionManager.h" @implementation MyHTTPSessionManager + (instancetype)sharedClient { static MyHTTPClient *_sharedClient = nil; static dispatch_once_t onceToken; dispatch_once(&onceToken, ^{ [[NSNotificationCenter defaultCenter] addObserver:_sharedClient selector:@selector(didCompleteTask:) name:AFNetworkingTaskDidCompleteNotification object:nil]; }); return _sharedClient; } - (void)didCompleteTask:(NSNotification *)notification { NSURLSessionDataTask *task = notification.object; NSHTTPURLResponse *response = (NSHTTPURLResponse *)task.response; NSData *responseData = notification.userInfo[AFNetworkingTaskDidCompleteResponseDataKey]; if (!responseData.length) { // Do not cache empty responses. // You could place additional checks above to cache responses selectively. return; } NSCachedURLResponse *cachedResponse = [[NSCachedURLResponse alloc] initWithResponse:response data:responseData]; [[NSURLCache sharedURLCache] storeCachedResponse:cachedResponse forRequest:task.currentRequest]; }

I tried to come up with some sort of cleaner solution, but it seems that AFNetworking does not provide a callback or a delegate method that returns everything we need early enough - that is, before it has been serialized by the response serializer.

Hope people will find this helpful :)

We are seeing this same issue on our end! Did you end up resolving this, by chance? Is your assumption that there is a bug in NSURLCache when handling a response with transfer-encoding: chunked?
Unfortunately we did not have enough time to test it more on iOS, so we ended up disabling chunked responses in Apache. Here's a relevant post: stackoverflow.com/questions/10300446/… I did not have enough time to test if chunked transfers are the only reason why NSURLCache breaks. Perhaps it's that, combined with gzip compression (which we also use). I will do some more tests when I have time and post the results here.
Hi goatrance, any update on this one? Should NSURLCache handle chunked transfers? I'm facing the same problem, thanks for sharing.
I can finally confirm that in our case, what prevents responses from being cached, is the chunked transfer-encoding. Sorry for taking so long to respond to this. I could not find anything in the Apple docs, but I have tested it thoroughly by disabling gzip compression and turning chunked transfer-encoding on and off.

Eugene P · Accepted Answer · 2013-11-27 18:14:35Z

Did you try to configure NSURLRequestCachePolicy for NSURLRequest

+ (id)requestWithURL:(NSURL *)theURL cachePolicy:(NSURLRequestCachePolicy)cachePolicy timeoutInterval:(NSTimeInterval)timeoutInterval

These constants are used to specify interaction with the cached responses.

enum { NSURLRequestUseProtocolCachePolicy = 0, NSURLRequestReloadIgnoringLocalCacheData = 1, NSURLRequestReloadIgnoringLocalAndRemoteCacheData =4, NSURLRequestReloadIgnoringCacheData = NSURLRequestReloadIgnoringLocalCacheData, NSURLRequestReturnCacheDataElseLoad = 2, NSURLRequestReturnCacheDataDontLoad = 3, NSURLRequestReloadRevalidatingCacheData = 5 }; typedef NSUInteger NSURLRequestCachePolicy;

I've done tests by playing with all these values, expecially with NSURLRequestReturnCacheDataElseLoad. Same results.

Matt J · Accepted Answer · 2013-11-27 18:19:33Z

You could investigate what your cached response is from the sharedURLCache by subclassing NSURLProtocol and overriding startLoading:

add in AppDelegate application:didFinishLaunchingWithOptions:

[NSURLProtocol registerClass:[CustomURLProtocol class]];

Then create a subclass of NSURLProtocol (CustomURLProtol) and override startLoading

- (void)startLoading { self.cachedResponse = [[NSURLCache sharedURLCache] cachedResponseForRequest:self.request]; if (self.cachedResponse) { [self.client URLProtocol:self didReceiveResponse:[self.cachedResponse response] cacheStoragePolicy:[self.cachedResponse storagePolicy]]; [self.client URLProtocol:self didLoadData:[self.cachedResponse data]]; } [self.client URLProtocolDidFinishLoading:self]; }

self.cachedResponse is a property NSCachedURLResponse i've added. You can see if anything is wrong with any cachedResponse here.

Collectives™ on Stack Overflow

NSURLCache: inconsistent behaviour

3 Answers 3

4 Comments

2 Comments

Comments

Linked

Hot Network Questions