Removing empty rows from dataframe

Question

I have a dataframe with empty values in rows

How can I remove these empty values? I have already tried data.replace('', np.nan, inplace=True) and data.dropna() but that didn't change anything. What other ways are there to drop empty rows from a dataframe?

Have you tried dropna?

Yevhen Kuzmovych
– Yevhen Kuzmovych

2021-03-31 13:42:22 +00:00
Commented Mar 31, 2021 at 13:42 — Yevhen Kuzmovych
– Yevhen Kuzmovych, Commented Mar 31, 2021 at 13:42
I tried it right away, but there was no effect

Sam324
– Sam324

2021-03-31 13:47:55 +00:00
Commented Mar 31, 2021 at 13:47 — Sam324
– Sam324, Commented Mar 31, 2021 at 13:47

BENY · Accepted Answer · 2021-03-31 13:59:16Z

1

Try with

data = data.replace('', np.nan).dropna()

Update

data = data.apply(pd.to_numeric,errors='coerce').dropna()

edited Mar 31, 2021 at 13:59

answered Mar 31, 2021 at 13:42

BENY

324k22 gold badges176 silver badges250 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Sam324 Over a year ago

Tried it, no effect

BENY Over a year ago

@Sam324 is that empty or it is ' '?

Sam · Accepted Answer · 2021-03-31 14:18:56Z

As you have spaces in a numeric variable, I'm assuming it got read in as a string. The way I would solve this in a robust way is following the following:

data = {'lattitude': ['', '38.895118', '', '', '', '45.5234515', '', '40.764462'], 'longitude': ['', '-77.0363658', '', '', '', '-122.6762071', '', '-11.904565']} df = pd.DataFrame(data)

Change the fields to a numeric field. errors='coerce' will change the values it can not convert to a numeric to pd.NaN.

df = df.apply(lambda x: pd.to_numeric(x, errors='coerce'))

The only thing you'll have to do now is drop the NA's

df.dropna(inplace=True)

Another possible solution is to use regular expressions. In this case it's a negative match to any character. So if the field does not have a character, it'll be caught here. Of course there are multiple regex possible here.

mask = (df['lattitude'].str.contains(r'(^\S)') & df['longitude'].str.contains(r'(^\S)')) df = df[mask]

Simon · Accepted Answer · 2021-03-31 14:06:08Z

suppose latitude is between -90 and 90.

data = data[data['latitude'] <= 90]

this should work, no matter they are Nan or ''

Sathish Pandurangan · Accepted Answer · 2024-02-24 20:17:01Z

0

Try this. Hacky, but works.

data.fillna("").replace('', pd.NA, inplace=True)

answered Feb 24, 2024 at 20:17

Sathish Pandurangan

488 bronze badges

Collectives™ on Stack Overflow

Removing empty rows from dataframe

4 Answers 4

2 Comments

Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

2 Comments

Comments

Comments

Comments

Related