ROC analysis (Receiver Operating Characteristic example)

Contents

Receiver Operating Characteristic example

ROC analysis provides tools to select possibly optimal models and to discard suboptimal ones independently from (and prior to specifying) the cost context or the class distribution. ROC analysis is related in a direct and natural way to cost/benefit analysis of diagnostic decision making. See http://en.wikipedia.org/wiki/Receiver_Operating_Characteristic for more information.

% demo_ROC.m

Construct the sample set

P = 100; % Number of True,  P=TP+FN  (True posititive + False negative)
N = 100; % Number of False, N=FP+TN  (True posititive + False negative)
muP = -0.5;% Mean of positive
sgP = 1; % Variance of positive
muN = 0.5; % Mean of negative
sgN = 1; % Variance of negative

x=[sgP*randn(P,1)+muP; sgN*randn(N,1)+muN];
y=[ones(P,1);   zeros(N,1)];

Plot the histogram

figure
hold on
[hP, xP]= hist(x(y==1));  % Histogram vercor for the Positive, along with the histogram values
[hN, xN] = hist(x(y==0)); % The same for the Negative
bar([xP',xN'],[hP',hN']); % Plot both histograms
legend('Positive','Negative');
xlabel('Independent variable, x');
ylabel('Number of the objects');
title('Histograms of two classes');
axis tight
hold off

Construct the ROC

nSamples = 100; % Number of samples in the threshold
vecThresh = linspace(min(x), max(x), nSamples); % Vector of various thesholds

ROC = []; % ROC is {(FPR, TPR)}set
for t = vecThresh
    idxLeft = find(x<=t);
    idxRight = find(x>t);
    TP = length(find(y(idxLeft)==1));    % Number of the True positive items
    TN = length(find(y(idxRight)==0)); % Number of the True negative items

    TPR = TP/P;               % Fals positive rate
    SPC = TN/N;               % Sensitivity

    ROC = [ROC;[1-SPC, TPR]];% Add new element ti the ROC curve
end

Plot the ROC curve

figure
hold on
plot([0,1],[0,1],'k:'); % Plot the random decision line
plot(ROC(:,1), ROC(:,2)); % Plot the ROC itself
xlabel ('False positive rate, FPR=1-SPC');
ylabel ('True positive rate, TPR');
title('Receiver Operating Characteristic')
hold off