de.lmu.ifi.dbs.elki.evaluation.scores

Class ROCEvaluation

java.lang.Object

de.lmu.ifi.dbs.elki.evaluation.scores.ROCEvaluation

All Implemented Interfaces:

ScoreEvaluation

public class ROCEvaluation
extends java.lang.Object
implements ScoreEvaluation

Compute ROC (Receiver Operating Characteristics) curves. A ROC curve compares the true positive rate (y-axis) and false positive rate (x-axis). It was first used in radio signal detection, but has since found widespread use in information retrieval, in particular for evaluating binary classification problems. ROC curves are particularly useful to evaluate a ranking of objects with respect to a binary classification problem: a random sampling will approximately achieve a ROC value of 0.5, while a perfect separation will achieve 1.0 (all positives first) or 0.0 (all negatives first). In most use cases, a score significantly below 0.5 indicates that the algorithm result has been used the wrong way, and should be used backwards.

Since:

0.7.0

Author:

Erich Schubert

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ROCEvaluation.Parameterizer Parameterization class.

Nested classes/interfaces inherited from interface de.lmu.ifi.dbs.elki.evaluation.scores.ScoreEvaluation

ScoreEvaluation.Predicate<T>, ScoreEvaluation.ScoreIter

Field Summary

Fields

Modifier and Type	Field and Description
static ROCEvaluation	STATIC Static instance

Constructor Summary

Constructors

Constructor and Description
ROCEvaluation()

Method Summary

Modifier and Type	Method and Description
static <I extends ScoreEvaluation.ScoreIter> double	computeROCAUC(ScoreEvaluation.Predicate<? super I> predicate, I iter) Compute the area under the ROC curve given a set of positive IDs and a sorted list of (comparable, ID)s, where the comparable object is used to decided when two objects are interchangeable.
<I extends ScoreEvaluation.ScoreIter> double	evaluate(ScoreEvaluation.Predicate<? super I> predicate, I iter) Evaluate a given predicate and iterator.
double	expected(int pos, int all) Expected score for a random result.
static <I extends ScoreEvaluation.ScoreIter> XYCurve	materializeROC(ScoreEvaluation.Predicate<? super I> predicate, I iter) Compute a ROC curve given a set of positive IDs and a sorted list of (comparable, ID)s, where the comparable object is used to decided when two objects are interchangeable.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface de.lmu.ifi.dbs.elki.evaluation.scores.ScoreEvaluation

evaluate

Field Detail

STATIC

public static final ROCEvaluation STATIC

Static instance

Constructor Detail

ROCEvaluation

public ROCEvaluation()

Method Detail

evaluate

public <I extends ScoreEvaluation.ScoreIter> double evaluate(ScoreEvaluation.Predicate<? super I> predicate,
                                                             I iter)

Description copied from interface: ScoreEvaluation

Evaluate a given predicate and iterator.

Specified by:

evaluate in interface ScoreEvaluation

Type Parameters:

I - Iterator type

Parameters:

predicate - Predicate (for positives)

iter - Iterator

Returns:

Score

materializeROC

public static <I extends ScoreEvaluation.ScoreIter> XYCurve materializeROC(ScoreEvaluation.Predicate<? super I> predicate,
                                                                           I iter)

Compute a ROC curve given a set of positive IDs and a sorted list of (comparable, ID)s, where the comparable object is used to decided when two objects are interchangeable.

Type Parameters:

I - Iterator type

Parameters:

predicate - Predicate to test for positive objects

iter - Iterator over results, with ties.

Returns:

area under curve

expected

public double expected(int pos,
                       int all)

Description copied from interface: ScoreEvaluation

Expected score for a random result.

Specified by:

expected in interface ScoreEvaluation

Parameters:

pos - Number of positive elements

all - Total number of elements

Returns:

Expected score

computeROCAUC

public static <I extends ScoreEvaluation.ScoreIter> double computeROCAUC(ScoreEvaluation.Predicate<? super I> predicate,
                                                                         I iter)

Compute the area under the ROC curve given a set of positive IDs and a sorted list of (comparable, ID)s, where the comparable object is used to decided when two objects are interchangeable.

Type Parameters:

I - Iterator type

Parameters:

predicate - Predicate to test for positive objects

iter - Iterator over results, with ties.

Returns:

area under curve