Mdl = sparsefilt (X, Q) learns Q features from the $N\; \times \; P$ data matrix X (rows are observations, columns are predictors) and returns a SparseFiltering object Mdl. Apply the learned transformation to data with transform (Mdl, X).

The $N\; \times$Q features returned by transform are the soft-absolute activations sqrt ((X * W) .^ 2 + 1e-8), normalized first across observations (each feature) and then across features (each observation). The $P\; \times$Q weight matrix W minimizes the sum of those features plus an L2 penalty Lambda * ||W ||_F^2, driving the features to be sparse.

Name/Value pairs:

'IterationLimit'

Maximum number of iterations (default 1000).

'Lambda'

Weight of the L2 penalty on the transform weights (default 1).

'Standardize'

Logical; center and scale each predictor before fitting (default false).

'InitialTransformWeights'

A $P\; \times$Q initial value for the weights. The default is random.

'GradientTolerance', 'StepTolerance'

Stop once the gradient’s or the step’s infinity norm falls to or below the given value (default 1e-6 each). They govern the fit only under 'Solver', 'lbfgs'; the 'quasinewton' solver runs to its own tighter internal tolerances and records these without acting on them.

'Solver'

'quasinewton' (default) minimizes through Octave’s fminunc, which carries a full inverse Hessian. 'lbfgs' selects the limited-memory BFGS solver MATLAB uses, holding as many curvature pairs as the transform has parameters, and is several times faster here. It stops where 'GradientTolerance' and 'StepTolerance' say to, so a value tighter than the default carries it further.

Note on reproducibility

The sparse filtering objective is not convex and is minimized by a quasi-Newton solver, so the learned weights depend on the starting point and the solver. Different runs (or different software, including MATLAB) may return different weights that nonetheless describe an equally valid feature transformation. Fix 'InitialTransformWeights' for a reproducible result.

See also: SparseFiltering, rica, pca

Source Code: sparsefilt