40 questions
0 votes
0 answers
26 views
How to define correlation structure for relational data in glmmTMB?
I have relational data, i.e. observations for pairs of objects. More specifically these are migration rates between plant populations, which I would like to explain by a predictor. The migration rates ...
1 vote
1 answer
70 views
cosine_distances on large matrices
I have an embedding matrix in the size of (100000, 100). I want to compute all the pairwise cosine distances in the matrix. I've tried using sklearn.metrics.pairwise.cosine_distances function, but it ...
1 vote
1 answer
88 views
split-half reliability using Jensen-Shannon Divergence
I have two pandas dataframes in which each row is a person and their response data in the form of a list: df_1 = pd.DataFrame({'ID': ['a', 'b', 'c', 'd', 'e', 'f'], 'response': [["apple", &...
0 votes
1 answer
259 views
Set the range of pairwise distance and cosine similarity between 0 and 1
I write a BiLSTM-Siamese Network to measure the string similarities using pairwise distance and cosine similarities with the detail as follows: class SiameseNetwork(nn.Module): def __init__(self, ...
1 vote
2 answers
180 views
Vectorize nestled loops for pairwise distance calculation
How to make the script below more efficient? This is a follow-up to my previous post Python nested loop issue It currently takes the best part of two hours to process input tables consisting in about ...
0 votes
1 answer
53 views
find pairwise row differences of numpy array (with perdioic boundaries)
I have a (N,3)-dimensional numpy array holding the 3-dim. positions of N particles in its rows. I want to obtain the pairwise distances of these particles using periodic boundaries (the latter means ...
3 votes
1 answer
918 views
Is there an efficient way to compute Hamming distances between every possible column in a binary matrix?
In NumPy, the command numpy.corrcoef(X.T) is amazingly efficient at computing correlations between every possible pair of columns in a matrix X. I am looking for a similarly efficient method to ...
1 vote
1 answer
92 views
How can I loop a function through every combination of levels of a factor?
I have a dataset containing a set of variables and the coordinates describing their distributions in geographic space: set.seed(123) #example dataset: d <- data.frame(var=as.factor(rep(LETTERS[1:...
0 votes
0 answers
150 views
Passing sparse distance matrix to AgglomerativeClustering is giving TypeError
I was getting MemoryError when I imported 100,000 documents to pairwise_distances function. For this reason, I sparsely calculated the distance matrix piece by piece and combined it finally. But ...
1 vote
1 answer
2k views
Calculating pairwise distances between entries in a `torch.tensor`
I'm trying to implement a manifold alignment type of loss illustrated here. Given a tensor representing a batch of embeddings of shape (L,N) for example with L=256: tensor([[ 0.0178, 0.0004, -0.0217, ...
0 votes
1 answer
380 views
Torch custom pairwise distance
i'm working on big distance matrix (10-80k row ; 3k cols) and i want to get custom pairwise distance on that matrix ; and do it fast. I have trying with armadillo but with huge data it still "...
0 votes
0 answers
196 views
Gurobi/python euclidean/manhattan distance calculations inside of a constraint, addConstr(...)
Is it possible to use the sklearn pairwise_distances function inside an addConstr(...) for computing the distance between 2 D-dimensional points in the constraint? I'd like to do something like this: ...
1 vote
2 answers
323 views
Compute differences between all variable pairs in R
I have a dataframe with 4 columns. set.seed(123) df <- data.frame(A = round(rnorm(1000, mean = 1)), B = rpois(1000, lambda = 3), C = round(rnorm(1000, mean = -1)), ...
0 votes
1 answer
637 views
Pairwise Mahalanobis Distance Between Multiple Independent Variables and a Dependent Variable With Three Classes in R
Issue I'm having some trouble calculating the Mahalanobis Distance between three classes of the independent variable Country. My aim is to calculate the Mahalanobis distance among dolphin whistle ...
0 votes
1 answer
951 views
Creating a pairwise similarity matrix of a list of words using Gensim Fasttext in python
I have a list of words, and I need to create a pairwise similarity matrix using the Fasttext word embedding. This is what I am currently doing: from gensim.models import fasttext as ft from sklearn....