L1Prior

Adds L1 regularization to a Linear layer.

This is implemented as a proximal operator during SGD.

class L1Prior : ProxyLinearPrior {
float _lambda;
float[][] W_prior;
size_t _ind_start;
}

Examples

1 // basic L1 regularization: loss = 0.03 * | W |
2 auto l1 = Linear(5).prior(L1Prior(0.03));
3 
4 // same, but centered around a non-zero matrix: loss = 0.03 * | W - W_p |
5 auto l2 = Linear(5).prior(L1Prior(0.03, W_p));

Meta