 Implemented algorithms

EThe standard VB method ('Sparse full-covariance method') calculates an inverse of regularized input covariance matrix. When the input dimension becomes over several thousands, computational time becomes very long because computational time for inverse calculation increases as cubic power of input dimension. 

E'Sparse space-covariance method' uses sparse constraints only on the space dimension and it does not impose sparse constraints on temporal dimension using time embedding (: temporal dimension is not pruned). This algorithm calculates inverse of spatial covariance matrix and much faster than the standard method for time embedding representation.

E'Sparse stepwise method' maximize the free energy coordinate by coordinate. This method does not use inverse covariance matrix. This method also introduce sparse condition only for spatial dimension but not for temporal dimension.

E'Sparse sequential method' is a incremental method proposed by Tipping and Faul (2003). It increases effective input dimension one by one. This method introduce sparse condition both for space & time dimension. However, this method becomes slow for more than 10,000 total dimension.

EFor vector output, there are three possibilities:

1.Train different models for each output component.
  In this case, selected input dimensions might be different for each output.

2.Train one model for vector output under the assumption that 
  all output component have the same observation noise variance.
  When input dimension is pruned, weights for all output components are pruned.
  Then, selected input dimensions are the same for all output components.

3.Train one model for vector output under the assumption that 
  each output component has different observation noise variance.
  When input dimension is pruned, weights for all output components are pruned.
  Then, selected input dimensions are the same for all output components.

  Please use functions such as 'linear_sparse_xxx_vec' for this case.

--------------------------------------------------
linear_sparse_space.m
linear_sparse_space_vec.m

linear_sparse_space_sw.m
linear_sparse_space_vec_sw.m

linear_sparse_stepwise.m
linear_sparse_stepwise_vec.m

linear_sparse_stepwise_sw.m

linear_map_sparse_cov.m
linear_map_sparse_cov_pinv.m
linear_sparse_cov.m

linear_sparse_seq.m

linear_stepwise_reg.m
linear_stepwise_reg_bayes.m

linear_fit.m
