ParsimonY contains the following features:

• parsimony provides structured and sparse penalties in machine learning. It contains, among other things:
  • Loss functions:
    • Linear regression
    • Logistic regression
  • Penalties:
    • L1 (Lasso)
    • L2 (Ridge)
    • Total variation (TV)
    • Overlapping Group LASSO (GL)
    • Any combination of the above
  • Algorithms:
    • I terative S hrinkage-T hresholding A lgorithm (fista)
    • F ast I terative S hrinkage-T hresholding A lgorithm (fista)
    • CO ntinuation of NEST erov’s smoothing A lgorithm (conesta)
    • Excessive gap method
  • Estimators
    • LinearRegression
    • Lasso
    • ElasticNet
    • LinearRegressionL1L2TV
    • LinearRegressionL1L2GL
    • LogisticRegressionL1L2TL
    • LogisticRegressionL1L2GL
    • LinearRegressionL2SmoothedL1TV

The reference environment for pylearn-parsimony was Ubuntu 14.04 LTS with Python 2.7.6, Numpy 1.8.2 and Scipy 0.13.3. We have converted the package to Python 3, but have not finished this transition yet. It appears to work well in Python 3, but please report any bugs that you find!

Unless you already have Numpy and Scipy installed, you need to install them:

$ sudo apt-get install python-numpy python-scipy 

or

$ sudo apt-get install python3-numpy python3-scipy 

In order to run the tests, you may also need to install Nose:

$ sudo apt-get install python-nose 

or

$ sudo apt-get install python3-nose 

In order to show plots, you may need to install Matplotlib:

$ sudo apt-get install python-matplotlib 

or

$ sudo apt-get install python3-matplotlib 

Downloading a stable release

Download the release of pylearn-parsimony from https://github.com/neurospin/pylearn-parsimony/releases. Unpack the file.

Downloading the latest development version

Clone the github repository

git clone https://github.com/neurospin/pylearn-parsimony.git 

Installing

To install on your system, go to the pylearn-parsimony directory and type:

$ sudo python setup.py install 

or

$ sudo python3 setup.py install 

Or, you can simply set your $PYTHONPATH variable to point parsimony:

$ export $PYTHONPATH=$PYTHONPATH:/directory/pylearn-parsimony 

You are now ready to use your fresh installation of pylearn-parsimony!

A simple example: We first build a simulated dataset X and y.

import numpy as np np.random.seed(42) shape = (1, 4, 4) # Three-dimension matrix num_samples = 10 # The number of samples num_ft = shape[0] * shape[1] * shape[2] # The number of features per sample # Randomly generate X X = np.random.rand(num_samples, num_ft) beta = np.random.rand(num_ft, 1) # Define beta # Add noise to y y = np.dot(X, beta) + 0.001 * np.random.rand(num_samples, 1) X_train = X[0:6, :] y_train = y[0:6] X_test = X[6:10, :] y_test = y[6:10]

We build a simple estimator using the OLS (ordinary least squares) loss function and minimize using Gradient descent.

import parsimony.estimators as estimators import parsimony.algorithms as algorithms ols = estimators.LinearRegression(algorithm_params=dict(max_iter=1000))

Then we fit the model, estimate beta, and predict on test set.

res = ols.fit(X_train, y_train) print("Estimated beta error = ", np.linalg.norm(ols.beta - beta)) print("Prediction error = ", np.linalg.norm(ols.predict(X_test) - y_test))

Now we build an estimator with the OLS loss function and a Total Variation penalty and minimize using FISTA.

import parsimony.estimators as estimators import parsimony.algorithms as algorithms import parsimony.functions.nesterov.tv as tv l = 0.0 # l1 lasso coefficient k = 0.0 # l2 ridge regression coefficient g = 1.0 # tv coefficient A = tv.linear_operator_from_shape(shape) # Memory allocation for TV olstv = estimators.LinearRegressionL1L2TV(l, k, g, A, mu=0.0001, algorithm=algorithms.proximal.FISTA(), algorithm_params=dict(max_iter=1000))

We fit the model, estimate beta, and predict on the test set.

res = olstv.fit(X_train, y_train) print("Estimated beta error = ", np.linalg.norm(olstv.beta - beta)) print("Prediction error = ", np.linalg.norm(olstv.predict(X_test) - y_test))

• Tutorials

• Documentation