Suppose I have several nested structs in C++, like this: struct Size { int width, height; }; struct Position { int x, y; }; struct Layout { Size size; Position position; }; class ...

I am using a BlockingCollection<T> in a producer/consumer scenario, where the Ts are big objects with variable memory size. Some Ts can be as large as 700 MB, and others can be as small as 10 MB....

Context: The last section of the amazing new Joins in data.table vignette explains how to use the merge sintax ( x[i,on=.(id),...]) to update the LEFT table by reference (also described in this SO ...

Say I have the below table library(jsonify) library(data.table) df <- data.table(col = c('{"geo":"USA","class":"A","score":"99"}' ...

I'm working on a Python project where I need to process a very large file (e.g., a multi-gigabyte CSV or log file) in parallel to speed up processing. However, I have three specific requirements that ...

I have inherited some code that is doing the following steps: Starting with a byte array of compressed data, stream and unzip it Deserialize it into a list of objects Add to the list Serialize the ...

I've a simple ASP.NET Core Web API that processes large size documents (> 10 MB < 50 MB). Basically it reads a document from CRM like Salesforce processes it with Aspose and send the processed ...

I'm working on a project when I do format preserving encryption (that includes the three types alphabetic, alphanumeric and numeric ) well to achieve this I wrote several methods, then I wrote a ...

I have two numpy.ndarray instances with different shapes. If I add these two arrays, broadcasting will occur between them: import numpy as np x = np.array([1, 2, 3]) y = np.array([[2, 3, 5], ...

I want to run a task with a multiprocessing.Pool and return only the minimum element, without taking the memory to store every output. My code so far: with Pool() as pool: programs = pool.map(task, ...

I have two large 2D numpy arrays A and B (each array has dimensions (18000,18000)). I want to calculate the Pearson correlation between corresponding columns of the two arrays (i.e. naively calculate ...

When attempting to transfer files from an external server to a Google Cloud Storage (GCS) bucket using the provided code snippet (code to be executed on Google Cloud Functions), the files are ...

I have a square matrix with dimension ranging from 100x100 to 10,000x10,000. The matrix represents parameter values for a function. I go through a loop where I try various combinations of parameters....

Is it feasible to streamline the counting sort algorithm by exclusively utilizing the counting array (C) after determining the frequency of each element in the input array? Instead of creating an ...

I am trying to generate last day of the month for a date field in my dataframe: so there is field start date, I want to add say 5 columns like nep_0, nep_1, nep_2, nep_3, nep_4. So for nep_0 logic is ...