split-apply-combine to sklearn pipeline

Question

I am trying to generate a pipeline using sklearn, and am not really sure how to go about it. Here is a minimal example:

def numFeat(data): return data[['AGE', 'WASTGIRF']] def catFeat(data): return pd.get_dummies(data[['PAI', 'smokenow1']]) features = FeatureUnion([('f1',FunctionTransformer(numFeat)), ('f2',FunctionTransformer(catFeat)) ] ) pipeline = Pipeline( [('f', features), ('lm',LinearRegression())] ) data = pd.DataFrame({'AGE':[1,2,3,4], 'WASTGIRF': [23,5,43,1], 'PAI':['a','b','a','d'], 'smokenow1': ["lots", "some", "none", "some"]}) pipeline.fit(data, y) print pipeline.transform(data)

In the above example, data is a Pandas DataFrame that contains the columns ['AGE', 'WASTGIRF', 'PAI', 'smokenow1'] among others.

Of course, in the FeatureUnion example, I want to supply many more transformation operations, but, all of them take a Pandas DataFrame and return another Pandas DataFrame. So in effect, I want to do something like this ...

data --+-->num features-->num transforms--+-->FeatureUnion-->model | | +-->cat features-->cat transforms--+

How do I go about doing this?

For the example above, the error i get is ...

TypeError: float() argument must be a string or a number

Community · Accepted Answer · 2017-05-23 12:32:43Z

You need to initialise FunctionTransformer with validate=False (IMO this is a bad default that should be changed):

features = FeatureUnion([('f1',FunctionTransformer(numFeat, validate=False)), ('f2',FunctionTransformer(catFeat, validate=False))] )

See also sklearn pipeline - how to apply different transformations on different columns

Collectives™ on Stack Overflow

split-apply-combine to sklearn pipeline

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related