Skip to main content
added 27 characters in body
Source Link
Tony
  • 151
  • 1
  • 4

Gert Arnold gave a great answer but let me suggest one more soution to try. Yes, data which is got from another source (rather than DB) can be processed in 2 ways:

  1. Download as small data part from DB to local as possible and join locally (usually using AsEnumerable() or basically ToList()). You got many good thoughts on this in other answers.
  2. Another one is different - upload your local data to server somehow and perform query on DB side. Uploading can be done differently: using some temp tables OR using VALUES. Fortunately there is a small extension for EF now (for both EF6 and EF Core) which you could try (It is written by me). It is EntityFrameworkCore.MemoryJoin (name might be confusing, but it supports both EF6 and EF Core). As stated in author's article (me) it modifies SQL query passed to server and injects VALUES construction with data from your local list. And query is executed on DB server.

So in your case you may try the following:

var query= from e in _context.Employees where (...filters...) select e; // change 1: no need to use AsEnumerable now var employees = query; // change 2: get IQueryable representation using EntityFrameworkCore.MemoryJoin var offices = context.FromLocalList(_officeService.GetAllOffices()); var employeeData = from e in employees join o in offices on e.Office equals o.Code select new EmployeeData { EmployeeId = e.EmployeeId, FullName = e.FullName, Office = e.Office, Area = o.Area, Region = o.Region, OfficeName = o.Name, Position = e.Position, Languages = e.Languages }; // change 3 (suggested), let's return result list instead of IQueryable return employeeData.ToList(); 

Using code above, query will be done on DB side. 2500 records should be ok to process (I used with 20k), but of course need to ensure this works fine for you.

Gert Arnold gave a great answer but let me suggest one more soution to try. Yes, data which is got from another source (rather than DB) can be processed in 2 ways:

  1. Download as small data part from DB to local as possible and join locally (usually using AsEnumerable() or basically ToList()). You got many good thoughts on this in other answers.
  2. Another one is different - upload your local data to server somehow and perform query on DB side. Uploading can be done differently: using some temp tables OR using VALUES. Fortunately there is a small extension for EF now (for both EF6 and EF Core) which you could try. It is EntityFrameworkCore.MemoryJoin (name might be confusing, but it supports both EF6 and EF Core). As stated in author's article it modifies SQL query passed to server and injects VALUES construction with data from your local list. And query is executed on DB server.

So in your case you may try the following:

var query= from e in _context.Employees where (...filters...) select e; // change 1: no need to use AsEnumerable now var employees = query; // change 2: get IQueryable representation using EntityFrameworkCore.MemoryJoin var offices = context.FromLocalList(_officeService.GetAllOffices()); var employeeData = from e in employees join o in offices on e.Office equals o.Code select new EmployeeData { EmployeeId = e.EmployeeId, FullName = e.FullName, Office = e.Office, Area = o.Area, Region = o.Region, OfficeName = o.Name, Position = e.Position, Languages = e.Languages }; // change 3 (suggested), let's return result list instead of IQueryable return employeeData.ToList(); 

Using code above, query will be done on DB side. 2500 records should be ok to process (I used with 20k), but of course need to ensure this works fine for you.

Gert Arnold gave a great answer but let me suggest one more soution to try. Yes, data which is got from another source (rather than DB) can be processed in 2 ways:

  1. Download as small data part from DB to local as possible and join locally (usually using AsEnumerable() or basically ToList()). You got many good thoughts on this in other answers.
  2. Another one is different - upload your local data to server somehow and perform query on DB side. Uploading can be done differently: using some temp tables OR using VALUES. Fortunately there is a small extension for EF now (for both EF6 and EF Core) which you could try (It is written by me). It is EntityFrameworkCore.MemoryJoin (name might be confusing, but it supports both EF6 and EF Core). As stated in author's article (me) it modifies SQL query passed to server and injects VALUES construction with data from your local list. And query is executed on DB server.

So in your case you may try the following:

var query= from e in _context.Employees where (...filters...) select e; // change 1: no need to use AsEnumerable now var employees = query; // change 2: get IQueryable representation using EntityFrameworkCore.MemoryJoin var offices = context.FromLocalList(_officeService.GetAllOffices()); var employeeData = from e in employees join o in offices on e.Office equals o.Code select new EmployeeData { EmployeeId = e.EmployeeId, FullName = e.FullName, Office = e.Office, Area = o.Area, Region = o.Region, OfficeName = o.Name, Position = e.Position, Languages = e.Languages }; // change 3 (suggested), let's return result list instead of IQueryable return employeeData.ToList(); 

Using code above, query will be done on DB side. 2500 records should be ok to process (I used with 20k), but of course need to ensure this works fine for you.

Source Link
Tony
  • 151
  • 1
  • 4

Gert Arnold gave a great answer but let me suggest one more soution to try. Yes, data which is got from another source (rather than DB) can be processed in 2 ways:

  1. Download as small data part from DB to local as possible and join locally (usually using AsEnumerable() or basically ToList()). You got many good thoughts on this in other answers.
  2. Another one is different - upload your local data to server somehow and perform query on DB side. Uploading can be done differently: using some temp tables OR using VALUES. Fortunately there is a small extension for EF now (for both EF6 and EF Core) which you could try. It is EntityFrameworkCore.MemoryJoin (name might be confusing, but it supports both EF6 and EF Core). As stated in author's article it modifies SQL query passed to server and injects VALUES construction with data from your local list. And query is executed on DB server.

So in your case you may try the following:

var query= from e in _context.Employees where (...filters...) select e; // change 1: no need to use AsEnumerable now var employees = query; // change 2: get IQueryable representation using EntityFrameworkCore.MemoryJoin var offices = context.FromLocalList(_officeService.GetAllOffices()); var employeeData = from e in employees join o in offices on e.Office equals o.Code select new EmployeeData { EmployeeId = e.EmployeeId, FullName = e.FullName, Office = e.Office, Area = o.Area, Region = o.Region, OfficeName = o.Name, Position = e.Position, Languages = e.Languages }; // change 3 (suggested), let's return result list instead of IQueryable return employeeData.ToList(); 

Using code above, query will be done on DB side. 2500 records should be ok to process (I used with 20k), but of course need to ensure this works fine for you.