Target function for optimization in regression analysis

The target function for optimization in regression analysis usually requires additional parameters and must be placed apart from the main module. It is convenient to have several target functions and use them in turn.


Load data from file

fname = 'demo_target_function.csv';
DATA = dlmread(fname,',');
y = DATA(:,1);
x = DATA(:,2:end);

Tune a linear regression model using a common optimization algorithm

w0 = 0.1*ones(10,1); % parameters
w1 = 0.1*rand(10,1); % additional parameters
lambda =1; % regularization parameter

% run the optimization algorithm
w = fminunc(@target_function,w0,[],x,y);
Warning: Gradient must be provided for trust-region method;
  using line-search method instead.
Optimization terminated: relative infinity-norm of gradient less than options.TolFun.

Plot the results

h = figure; hold on
legend('Raw data','Regression');
% saveas(h,'demo_target_function_01','png')

Target function for optimization in regression analysis

The target function example

% function err = target(w,x,y,lambda,w1)
% Target function for a demo  optimization problem
% err = target(w,x,y,lambda,w1)
% w [1,W] parameter vector to be optimized
% x [N,1] independent variable of the regression model
% y [N,1] dependent variable
% lambda [scalar] regularization parameter (optional)
% num = nargin;
% if num == 5
%       regul = lambda* sumsqr(w-w1);
% else
%       regul = 0;
% end
% err = sumsqr(y-x*w) + regul;
% return

% this file: demo_target_function

Vadim Victor

Vadim V. Strijov, Data Analysis & Machine Learning professor at the FRCCSC of the RAS, Doctor of Physics and mathematics sciences

You may also like...