本文转自：http://blog.csdn.net/abcjennifer/article/details/8603214
首先声明，MOD不是取模函数！MOD是字典学习和sparse coding的一种方法… 最近在看KSVD，其简化版就是MOD（method of directions）,这么说吧，KSVD和MOD的优化目标函数是相同的，MOD之所以可以称作KSVD的简化版是因为KSVD在MOD的基础上做了顺序更新列的优化。关于KSVD和MOD的理论知识请见下面我给出的一页note和referenc中的paper。本文主要给出其基本思想及我的代码，已经过测试，如有bug欢迎提出。
Reference
<<From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images>>,
Page 68~70
KSVD & MOD's principle & objective function
Principle:
简单来说，其优化就是一个OMP（orthogonal matching pursuit）与Regression的迭代过程，因此代码包括一个OMP.m, regression.m.
Objective Function & the variation from MOD to KSVD:
Code
CODE1. MOD
运行Main（Main中通过MOD）学习字典和稀疏表示，MOD迭代调用Regression学习字典，调用和OMP获得sparse representation.
Main.m
- %% Main.m
- clc;
- clear;
- P = 512;
- N = 256;
- M = 128;
- K = 100;
- %% Data Generator Method 1
- % sparsity_X = 0.4;
- % Y = randi(10,M,P);
- % X = floor(sprand(N,P,sparsity_X)*10);
- %% Data Generator Method 2
- Y = randn(M,P);%Notice that Y should be full rank, that is, rank(Y) = N
- X = randn(N,P);% initialization of X
- %% Main Iteration
- [D,X] = MOD(Y,X,K,1e-4);
MOD.m
- % @Function: Method Of Dirction of 2D signal
- % For dictionary and sparse representation learning
- % @CreateTime: 2013-2-22
- % @Author: Rachel Zhang @ http://blog.csdn.net/abcjennifer
- %
- % @Reference: From Sparse Solutions of Systems of Equations to
- % Sparse Modeling of Signals and Images
- function [ D , X ] = MOD( Y ,X ,K ,ErrorThreshold )
- %MOD Summary of this function goes here
- % Detailed explanation goes here
- % Sample_Data is Y
- % Coefficient is X
- % Dictionary is D
- % sparsity is K
- disp('Run Method of directions');
- iteration_time = 1;
- error = ErrorThreshold+1;
- while error>=ErrorThreshold;
- disp(['iteration time = ' num2str(iteration_time)]);
- D = Regression(Y,X);
- X = OMP(Y,D,K);
- iteration_time = iteration_time+1;
- error = sum(sum(abs(Y-D*X)))
- end
- end
OMP.m
- % @Function: Orthogonal Matching Pursuit of 2D signal
- % Learning Sparse Representation Given Dictionary
- % @CreateTime: 2013-2-21
- % @Author: Rachel Zhang @ http://blog.csdn.net/abcjennifer
- %
- % @Reference: http://www.eee.hku.hk/~wsha/Freecode/freecode.htm
- function [ X ] = OMP( Y,D,K )
- % Y is the sample data to be recovered M*P
- % D is the dictionary M*N
- % X is the sparse coefficient N*P
- % K is the sparsity
- if nargin==2
- K = size(D,2);
- end;
- M = size(D,1);
- P = size(Y,2);
- N = size(D,2);
- m = K*2; % execute iterations
- for idx = 1:P
- % recover the idx-th column sample
- y = Y(:,idx);
- residual = y;
- Aug_D = [];
- D1 = D;
- for times = 1:m;
- product = abs(D1'*residual);
- [~,pos] = max(product); % 最大投影系数对应的位置
- Aug_D = [Aug_D, D1(:,pos)];
- D1(:,pos) = zeros(M,1); %去掉选中的列
- indx(times) = pos;
- Aug_x = (Aug_D'*Aug_D)^-1*Aug_D'*y; % 最小二乘,使残差最小,i.e. x = pinv(Aug_D)*y
- residual = y - Aug_D*Aug_x;
- if sum(residual.^2)<1e-6
- break;
- end
- end
- temp = zeros(N,1);
- temp(indx(1:times)) = Aug_x;
- X(:,idx) = sparse(temp);
- end
- end
Regression.m
- % @Function: Dictionary learning & Regression
- % Learning Dictionary Given Sparse Representation
- % @CreateTime: 2013-2-21
- % @Author: Rachel Zhang @ http://blog.csdn.net/abcjennifer
- %
- function [ D ] = Regression( Y,X )
- % Y is the sample data to be recovered M*P
- % D is the dictionary M*N
- % X is the sparse coefficient N*P
- % P>N>M
- %由于X是扁矩阵,需要转置求D0 = min(D) ||Y^T-X^TD^T||
- %这样就是N个未知数，P个方程去求解；
- %每次解得D中的一列，共解M次
- Y = Y';
- X = X';
- P = size(Y,1);
- N = size(X,2);
- M = size(Y,2);
- D = zeros(N,M);
- for i = 1:M;
- y = Y(:,i);
- D(:,i) =