1

I recently started using the new Outlook/Office 365 Rest API and got some great help regarding the authentication (new v2.0 app model), scopes, using the OutlookServicesClient and writing better (linq-)queries.

Unfortunately, some users reported some memory-issues, which resulted in the following System.OutOfMemoryException:

 Exception of type 'System.OutOfMemoryException' was thrown. : at Microsoft.OData.Core.Json.JsonReader.ReadInput() at Microsoft.OData.Core.Json.JsonReader.ParseStringPrimitiveValue(Boolean& hasLeadingBackslash) at Microsoft.OData.Core.Json.JsonReader.ParseValue() at Microsoft.OData.Core.Json.JsonReader.Read() at Microsoft.OData.Core.Json.BufferingJsonReader.ReadInternal() at Microsoft.OData.Core.Json.BufferingJsonReader.ReadNextAndCheckForInStreamError() at Microsoft.OData.Core.Json.BufferingJsonReader.ReadInternal() at Microsoft.OData.Core.Json.BufferingJsonReader.Read() at Microsoft.OData.Core.JsonLight.ODataJsonLightDeserializer.ParseProperty(DuplicatePropertyNamesChecker duplicatePropertyNamesChecker, Func`2 readPropertyAnnotationValue, String& parsedPropertyName) at Microsoft.OData.Core.JsonLight.ODataJsonLightDeserializer.ProcessProperty(DuplicatePropertyNamesChecker duplicatePropertyNamesChecker, Func`2 readPropertyAnnotationValue, Action`2 handleProperty) at Microsoft.OData.Core.JsonLight.ODataJsonLightPropertyAndValueDeserializer.ReadComplexValue(IEdmComplexTypeReference complexValueTypeReference, String payloadTypeName, SerializationTypeNameAnnotation serializationTypeNameAnnotation, DuplicatePropertyNamesChecker duplicatePropertyNamesChecker) at Microsoft.OData.Core.JsonLight.ODataJsonLightPropertyAndValueDeserializer.ReadNonEntityValueImplementation(String payloadTypeName, IEdmTypeReference expectedTypeReference, DuplicatePropertyNamesChecker duplicatePropertyNamesChecker, CollectionWithoutExpectedTypeValidator collectionValidator, Boolean validateNullValue, Boolean isTopLevelPropertyValue, Boolean insideComplexValue, String propertyName, Nullable`1 isDynamicProperty) at Microsoft.OData.Core.JsonLight.ODataJsonLightEntryAndFeedDeserializer.ReadEntryDataProperty(IODataJsonLightReaderEntryState entryState, IEdmProperty edmProperty, String propertyTypeName) at Microsoft.OData.Core.JsonLight.ODataJsonLightEntryAndFeedDeserializer.ReadEntryPropertyWithValue(IODataJsonLightReaderEntryState entryState, String propertyName) at Microsoft.OData.Core.JsonLight.ODataJsonLightEntryAndFeedDeserializer.<>c__DisplayClass2.<ReadEntryContent>b__0(PropertyParsingResult propertyParsingResult, String propertyName) at Microsoft.OData.Core.JsonLight.ODataJsonLightDeserializer.ProcessProperty(DuplicatePropertyNamesChecker duplicatePropertyNamesChecker, Func`2 readPropertyAnnotationValue, Action`2 handleProperty) at Microsoft.OData.Core.JsonLight.ODataJsonLightEntryAndFeedDeserializer.ReadEntryContent(IODataJsonLightReaderEntryState entryState) at Microsoft.OData.Core.JsonLight.ODataJsonLightReader.ReadEntryStart(DuplicatePropertyNamesChecker duplicatePropertyNamesChecker, SelectedPropertiesNode selectedProperties) at Microsoft.OData.Core.JsonLight.ODataJsonLightReader.ReadAtEntryEndImplementationSynchronously() at Microsoft.OData.Core.JsonLight.ODataJsonLightReader.ReadAtEntryEndImplementation() at Microsoft.OData.Core.ODataReaderCore.ReadImplementation() at Microsoft.OData.Core.ODataReaderCore.ReadSynchronously() at Microsoft.OData.Core.ODataReaderCore.InterceptException[T](Func`1 action) at Microsoft.OData.Core.ODataReaderCore.Read() at Microsoft.OData.Client.Materialization.ODataReaderWrapper.Read() at Microsoft.OData.Client.Materialization.FeedAndEntryMaterializerAdapter.TryRead() at Microsoft.OData.Client.Materialization.FeedAndEntryMaterializerAdapter.TryStartReadFeedOrEntry() at Microsoft.OData.Client.Materialization.FeedAndEntryMaterializerAdapter.TryReadEntry(MaterializerEntry& entry) at Microsoft.OData.Client.Materialization.FeedAndEntryMaterializerAdapter.<LazyReadEntries>d__0.MoveNext() at Microsoft.OData.Client.Materialization.FeedAndEntryMaterializerAdapter.Read() at Microsoft.OData.Client.Materialization.ODataReaderEntityMaterializer.ReadNextFeedOrEntry() at Microsoft.OData.Client.Materialization.ODataEntityMaterializer.ReadImplementation() at Microsoft.OData.Client.MaterializeAtom.MoveNextInternal() at Microsoft.OData.Client.MaterializeAtom.MoveNext() at System.Linq.Enumerable.<CastIterator>d__94`1.MoveNext() at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection) at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source) at Microsoft.OData.ProxyExtensions.PagedCollection`2..ctor(DataServiceContextWrapper context, QueryOperationResponse`1 qor) at Microsoft.OData.ProxyExtensions.DataServiceContextWrapper.<>c__DisplayClass38`2.<ExecuteAsync>b__36(IAsyncResult r) at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.OData.ProxyExtensions.DataServiceContextWrapper.<ExecuteAsync>d__3a`2.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task) at Shared.Data.Office365.Office365Api.<GetEmailsReceived>d__23.MoveNext()

My query (to get the number of emails received for a date) looks like this:

var dtStart = date.Date.ToUniversalTime(); var dtEnd = date.Date.AddDays(1).ToUniversalTime(); var mailResults = await _client.Me.Messages .OrderByDescending(m => m.ReceivedDateTime) .Where(m => m.ReceivedDateTime.Value >= dtStart && m.ReceivedDateTime.Value <= dtEnd) //todo: filter if not in Junk Email and Deleted Folder (maybe with ParentFolderId) .Take(500) // todo: add paging to (and maybe make more efficient?) .Select(m => new DisplayEmail(m)).ExecuteAsync(); // todo: add paging to (and maybe make more efficient?) foreach (var message in mailResults.CurrentPage) { emails.Add(message); }

I didn't find a way to dispose the sources and also don't think it's necessary, as there are just local variables. The result from the emails-list is then stored in the database. Memory usage for my tool was stable before I added this feature, and is now growing at around 10Mb per hour (depending on how many emails someone has).

Any suggestions are very much appreciated!

Improve this question
2
  • If you just need the number of emails received on a date do you need to store them at all? That would probably save quite a bit of time and space. Commented Feb 3, 2016 at 16:53
  • No I don't need it. But what do I need to change? Thanks! Commented Feb 3, 2016 at 18:39

1 Answer 1

Reset to default
2

You might try a few things here:

  • Reduce your page size. You're doing 500 at a time here (.Take(500)), you could try reducing that.
  • Only request the properties you're interested in. If you modify your .Select to reference specific properties of your m variable, the API should use a $select query parameter in the API request to limit what comes back. See this tutorial for an example.
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Thanks Jason! Even though I just selected a few properties in the Select statement, I can remove all of them as I don't need them just to count. Furthermore, I now implemented the proper paging (and reduced the Take(500) to Take(20). I just released the update and hope this fixes all memory issues for the users.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.