6 
import pandas as pd import numpy as np df = pd.DataFrame({'group': ['a'] * 5 + ['b'] * 5, 'x1': np.random.normal(0, 1, 10), 'x2': np.random.normal(0, 1, 10), 'y': np.random.normal(0, 1, 10)}) df Out[4]: group x1 x2 y 0 a -0.468746 1.254817 -1.629483 1 a -1.849347 -2.776032 1.413563 2 a 1.186306 0.766866 0.163395 3 a -0.314397 -0.531984 0.473665 4 a 0.278961 0.510429 1.484343 5 b 2.240489 0.856263 0.369464 6 b 2.029284 1.020894 -0.042139 7 b 1.571930 -0.415627 0.865577 8 b 0.609133 1.370543 0.450230 9 b -1.820421 -0.211467 0.704480

I would like to calculate the correlations between y and some specific(not all) columns of the same dataframe by group to produce an output dataframe that looks like:

Out[5]: x1 x2 a -0.168390 -0.622155 b -0.467561 -0.771757

I had tried to use one-liner like:

df.groupby('group')[['x1', 'x2']].apply(...some function here that takes y as argument...)

However, I am having difficulty on how to write the function so that it will iterate through the specified columns(x1 and x2) and how to specify y as the fixed column.

Does anyone know an elegant one-liner that can achieve this?

Improve this question

1 Answer 1

Reset to default
7

use groupby + corrwith

df.groupby('group').apply(lambda d: d.filter(like='x').corrwith(d.y)) x1 x2 group a 0.127141 0.434080 b -0.892755 0.524215
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Thanks. I was not aware of the corrwith function
np, glad I could help