1

I've created some code that identifies all of the site collections on a farm and then loops through all of the sites and lists within those collections . It's not ideal, but I've been tasked with it, and need to do so.

Unfortunately the code is performing very slowly, and I believe it's due to the following code which finds all of the items within a list:

foreach (SPList list in web.Lists) { query.RecordLists(list.ID.ToString(), list.Author.ToString(), list.Title, list.DefaultViewUrl, list.ParentWeb.Title, list.ParentWebUrl, list.ItemCount, list.LastItemModifiedDate, list.LastItemDeletedDate); // check if files exist in the document library, if they don't break out to next level if (list.ItemCount <= 0) continue; for (int i = 0; i < list.Items.Count; i++) { try { SPListItem item = list.Items[i]; if (!item["Created By"].ToString().Contains("System Account") && !item.Url.Contains("_catalogs") && !item.Url.Contains("Style Library") && !item.Url.Contains("Cache Profiles") && !item.ContentType.ToString().Contains("Theme Gallery") && !item.ContentType.ToString().Contains("Converted Forms") && !item.ContentType.ToString().Contains("Page Output Cache") && !item.ContentType.ToString().Contains("Master Page")) { query.RecordItems(item.ID.ToString(), item.ContentType.Name, item.DisplayName, item.Name, "", item.Url, item["Created By"].ToString(), "", Convert.ToDateTime(item["Modified"]), item["Modified By"].ToString(), Convert.ToDateTime(item["Created"]), item["Created By"].ToString()); } } catch (NullReferenceException) { SPListItem item = list.Items[i]; Logger.Error( "[{0}] Filed moving on file {1} as not all content was present", item.Name); } } }

Is there a faster way to do this? At its current rate this will take hours to execute.

Improve this question
1
  • 3
    My eyes started bleeding reading that code. I don't know what you do with the data, but I would deffinently advice that you use PowerShell to do that, or if you have to use it within a web part or timerjob, use CAMLquery as that works faster than what you have there. Commented Oct 2, 2013 at 5:38

3 Answers 3

Reset to default
6

You can use SPQuery to retreive the items, and loop over the results instead of looping directly over list.Items which is a very slow operation.
To retrieve all the items of the list - do not put anything in SPQuery.Query.
In addition - set in SPQuery.ViewFields only the fields you want to retrieve.
Like this: 

SPQuery query = new SPQuery(); query.ViewFields = string.Concat( "<FieldRef Name='Field1' />", "<FieldRef Name='Field2' />", "<FieldRef Name='Field3' />", "<FieldRef Name='Field4' />"); SPListItemCollection items = list.GetItems(query);

Then change:

for (int i = 0; i < list.Items.Count; i++)

to:

foreach(SPListItem item in items)

(foreach is faster than for in this case.)

Edit:
As Robert Lindgren said, you can use SPQuery.Query to include your filter, and thereby reduce the amount of the returned items.
This is a good reference for SPQuery.Query options.

Improve this answer
2
  • 1
    Why not put all his checks also in the Query? :) Commented Oct 2, 2013 at 10:28
  • @RobertLindgren - You're right, I did not notice it. Commented Oct 2, 2013 at 10:45
1

can you clarify what your trying todo? I can give you part of the reason its taking long! your looping within a loop and looking in every list and every item (all data), this equates to huge huge ammounts! hence why it crashes or stops or just goes amazingly slow!

the key here is to know what data you want! if for example you only want to check on one specific colunm for all lists in a site collection than you need to look into camlquery and be more specific!

Do you need to iterate though every cell in a list?

also dont forget your going though each site, meaning a new connection so make sure your disposing of the site once your finished though each loop:

SPWebApplication webApplication = SPContext.Current.Site.WebApplication; SPSiteCollection siteCollections = webApplication.Sites; foreach (SPSite siteCollection in siteCollections) { //do your code with list stuff //once complete with site close it before next loop siteCollection.Close(); }

the above is one way of improving speeds

http://msdn.microsoft.com/en-us/library/ms462161(v=office.14).aspx

Improve this answer
1

Hello two simple change would run you code like a charm: While iterating through the lists use a SPListCollection object to store the Web.lists. Second create SPListItemCollection object to store all the list items in to it. Then iterate each items from the SPListItemCollection object. User foreach in both the cases.

If your list would have too many entries then each time calling web.Lists and list.Items would make your performance slower.

If you are familiar with client object model then that would increase your performance to a greater extent.

-Vikram

Improve this answer
3
  • Do you have an example of this you can post? Commented Oct 2, 2013 at 22:38
  • 1
    SPListCollection listCol = web.Lists; foreach (SPList list in listCol) { SPListItemCollection itemCol = list.Items; foreach (SPListItem item in itemCol) { //Do your coding here } } Commented Oct 3, 2013 at 2:36
  • Hope this helps Commented Oct 3, 2013 at 2:38

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.