drop all rows that contain even one alphabet in pyspark [duplicate]

Question

pyspark 2.3.1

my rows to col1 should only contain integers. I am trying to filter out any row that have even one character. How can I do this in pyspark?

I've tried

df.select('col1').filter(df.col1.rlike(^[a-zA-Z]))

however rows that contain alphabet also contain integers therefore not filtered.

How can I do this?

过过招 · Accepted Answer · 2021-12-23 08:22:36Z

1

You can try to select pure digital rows.

df = df.filter('col1 rlike "^[0-9]+$"') df.show(truncate=False)

answered Dec 23, 2021 at 8:22

4,3372 gold badges7 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1