How to compute correlation between all variables?

Question

I cleaned previously my dataset and I am trying to compute correlation between some variables from my csv files. This is all of my columns :

Index(['ID', 'age', 'sex', 'city', 'province', 'country', 'latitude', 'longitude', 'geo_resolution', 'date_onset_symptoms', 'date_admission_hospital', 'date_confirmation', 'symptoms', 'lives_in_Wuhan', 'travel_history_dates', 'travel_history_location', 'reported_market_exposure', 'additional_information', 'chronic_disease_binary', 'chronic_disease', 'source', 'sequence_available', 'outcome', 'date_death_or_discharge', 'notes_for_discussion', 'location', 'admin3', 'admin2', 'admin1', 'country_new', 'admin_id', 'data_moderator_initials', 'travel_history_binary'], dtype='object')

I computed correlation by doing df.corr and I had this output :

 latitude longitude chronic_disease_binary admin_id latitude 1.000000 0.180020 -0.027188 0.172725 longitude 0.180020 1.000000 0.016198 0.242909 chronic_disease_binary -0.027188 0.016198 1.000000 -0.007604 admin_id 0.172725 0.242909 -0.007604 1.000000

I tried to compute correlation between sex and latitude to understand why I cannot have all columns and I have casting error :

TypeError: unsupported operand type(s) for /: 'str' and 'int'

How can I compute correlation between all variables ?

One of the columns is of string type. You can't correlate string and integer type features. — Vishnudev Krishnadas
– Vishnudev Krishnadas, Commented May 20, 2020 at 11:17

jezrael · Accepted Answer · 2020-05-20 11:18:14Z

I think there are some columns filled by string repr of numbers.

You can use to_numeric for all columns for convert non numeric to missing values and then remove only NaNs columns:

df = df.apply(lambda x: pd.to_numeric(x, errors='coerce')).dropna(how='all', axis=1).corr()

Or if necesary specify columns in list for processing:

cols = ['age','latitude','longitude'] df = df[cols].apply(lambda x: pd.to_numeric(x, errors='coerce')).corr()

Collectives™ on Stack Overflow

How to compute correlation between all variables?

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related