Thread get killed without exception

Question

I get cItems which is a IReadOnlyCollection<IWebElement> provided by Selenium. However. I make the function Single Thread based, it works nice. But now I wanted to increase performance and so I choosed the .Net ThreadPool.

ThreadPool.SetMaxThreads(16, 16); IEnumerator<IWebElement> iter = cItems.GetEnumerator(); while(iter.MoveNext()) { cThreadJobObj.Item = iter.Current; ThreadPool.QueueUserWorkItem(new WaitCallback(GetThreadJob), cThreadJobObj.Clone()); // .Clone() is a deep clone }

The Problem now is, that the Threads disappear, in Line 5. There isn't any Exception thrown, I think because the } catch (Exception Ex) { block isn't called.

public static void GetThreadJob(object ThreadJob) { try { var cThreadJob = ThreadJob as ThreadJobObj; var IWebElement = cThreadJob.Item; var cElem = cItem.FindElement(By.CssSelector("span.im_message_date_text")); } catch (Exception Ex) { Rupert.Logger.E("{Thread.GetCurrentProcessorId()} on Obj {cThreadJob.iCount}", Ex, false); } }

The GetThreadJob is called by all 16 Threads, and all disappear at the same line.

Update

I separated the line and the Threads disappear in this method ISearchContext.FindElement Method.

IWebElement.FindElement(By)

If I make a quickwatch on this line, this is the value:

cItem.FindElement(cSel) Function evaluation disabled because a previous function evaluation timed out. You must continue execution to reenable function evaluation. OpenQA.Selenium.IWebElement

If you run it in a debugger with "Break on thrown" (debug menu.. exceptions.. tick next to CLR) set, does it break? — Caius Jard
– Caius Jard, Commented Feb 18, 2021 at 17:52
@CaiusJard Now, I ticked every single. They are all disappearing on the same line. — ru4ert
– ru4ert, Commented Feb 18, 2021 at 20:43
@CaiusJard nothing is thrown. var cElem = cItem.FindElement(By.CssSelector("span.im_message_date_text")); in here the thread disappear. I can see that because I make a Console.WriteLine before and after. Further, during debugging, the process switches to the next waiting thread. If all passed this line, the program is waiting for I/O because the callback, which is in the bottom getting reached. — ru4ert
– ru4ert, Commented Feb 19, 2021 at 8:07

ru4ert · Accepted Answer · 2021-02-19 17:47:19Z

Ok, after a long night I found out, that the .Net Limits the Sockets which could be opened, during the Selenium Process.

The default value for ServicePointManager.DefaultConnectionLimit is 2 and 10 for Asp.net. More Info

This can be avoided through this in the init of the script:

 ThreadPool.SetMinThreads(Environment.ProcessorCount / 2, 25); ThreadPool.SetMaxThreads(Environment.ProcessorCount * 4, 100); ServicePointManager.UseNagleAlgorithm = true; ServicePointManager.Expect100Continue = true; ServicePointManager.DefaultConnectionLimit = 65000; ServicePointManager.MaxServicePointIdleTime = 500;

Collectives™ on Stack Overflow

Thread get killed without exception

Update

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

Update

1 Answer 1

Comments

Related