ICA

Independent Component Analysis

ICA is linear subspace method with the aim to find independet vectors according to higher order statistics (in layman terms: principal component analysis is a subset of ICA since on PCA first order gaussian distributions are assumed).

It's application is e.g. blind source separation (it's a special case of it), artefact removal on e.g. ECG and many more.

Definition

Check out the wikipedia page here which provides a great overview on ICA.

Implementation details

This library provides two implementation of the ICA algorithm:

• InfoMax: Following the approach in the Face Recognition by ICA.
• Fast ICA: The algorithm from Hyvärinen et.al. that provide a fast iterative Newton based optimization scheme.

Interface Defintion

   type
     TFastICANonLinEstimator = (iePow3, ieTanh, ieGauss, ieSkew);
     TICAMethod = (imFastICA, imInfomax);
  
   type
     TICAProps = record
       method : TICAMethod;
       BlockSize : integer;
       NumIter : integer;
       Annealing : boolean;
       KeepAdditionalData : boolean;

       // fast ica params
       NumIC : integer;
       epsilon : double;
       symApproach : boolean;
       nonLin : TFastICANonLinEstimator;
       a1 : double;           // parameter for tuning tanh
       a2 : double;           // parameter for tuning gaus
       myy : double;          // step size algorithm
       maxFineTune : Integer; // max number of finetuning steps
       stabilization : boolean;
       fineTuning : boolean;
    end;

   procedure ICA(Examples : TDoubleMatrix; props : TICAProps);
   
   class function DefProps : TICAProps;  

ICA is the main routine to calculate the indepndent component vectors. It takes the property structure to determine which method and parameters to use. NOt the parameters after the comment fast ica params are only used if the method is imFastICA. Please refere to the ICA example to check out good parameters for the algorithm.

• Examples: height=number of examples, width=example dimension
• NumIC: is either 0: then the resulting projection matrix is examples.width - 1

DefProps returns a prefiled structure with some suitable parameters. width:

• method : imFastICA
• BlockSize : 10 - block size parameter used in the infomax algorithm.
• NumIter := 200- maximum of 200 iterations until convergence
• Annealing := False - simulated annealing in infomax
• epsilon : 0.0001 - minimum error meas to start finetuning in fastica
• maxFineTune : 100 - number of fine tune iterations in fastica
• a1 := 1 - scaling factor for nonlinear tanh matrix operation
• a2 := 1 - scaling factor for nonlinear exponetial matrix operation
• myy := 1 - initial learning factor that is adjusted during execution for fastica
• NumIC := 0 - number of independet components to calculate
• symApproach := True - deflation or symmetric approach in fastica
• nonLin := ieTanh - matrix nonlinear operator
• fineTuning := False - if set to true another (maxfinetune) iterations with different learning rates are performed
• stabilization := True - if true the myy factor is adjusted

Examples

From the testing suit: This examples calculates 10 independet components on the given image matrix.

   function LoadImages(out w, h : integer) : TDoubleMatrix;
   begin
        // return a matrix with the loaded images. one column represents one images
		// output is original width and height
   end;   
   
   procedure testica;
   var examples : TDoubleMatrix;
       ica : TMatrixICA;
       w, h : integer;
       counter: integer;
       projMtx : IMatrix;
       reconMtx : IMatrix;
       props : TICAProps;
   begin
        Examples := LoadImages(w, h);

        ica := TMatrixICA.Create;
        try
           props := TMatrixICA.DefProps;
           props.NumIC := 10;
           ica.ICA(examples, props);

           for counter := 0 to Examples.Width - 1 do
           begin
                Examples.SetSubMatrix(counter, 0, 1, Examples.Height);
                projMtx := ica.ProjectToFeatureSpace(Examples);
                reconMtx := ica.Reconstruct(projMtx.GetObjRef);

                ImageFromMatrix(reconMtx.GetObjRef, w, h, Format('%s\ica_%d.bmp', [ExtractFilePath(ParamStr(0)), counter]));
           end;
        finally
               ica.Free;
        end;

        Examples.Free;
   end;