dclassify

dclassify is a Naive Bayesian classifier for NodeJS that is optimized for certain use cases using a unique "probability of absence" feature, resulting in a ~10% improvement over conventional binary classifiers. It is designed for classifying items based on a limited set of characteristics (tokens) rather than for general language processing.

Optimized Classification

Most binary classifiers work by looking for the presence of certain tokens. For example, an email spam filter might categorize emails by looking for words that are spam-related. It cares about the words that are present, but not about words that are missing.

In other cases we care if important key ingredients are missing. For example, an animal without wings is probably not a bird. And a mobile app without internet connectivity is probably not malware. These qualities (sometimes referred to as "prevalent negatives") are very effective if the total set of tokens is limited, and the classification has certain "key ingredients" (ie. birds should have wings). Checking for prevalent negatives can significantly improve the classifier results in such cases.

This optimization can be enabled in dclassify with the applyInverse option; this will calculate probabilities based on the present tokens as well as the inverse - a probability of absence for tokens that are NOT present.

Installation

npm install dclassify

Usage

1. Require the classifier and reference its utilities.
2. Create Document instances with names and an array of tokens representing the document's characteristics.
3. Add document instances to a DataSet using appropriate categories.
4. Create and train a classifier using the DataSet.
5. Test the classifier using a test Document.

    // module dependencies
    var dclassify = require('dclassify');

    // Utilities provided by dclassify
    var Classifier = dclassify.Classifier;
    var DataSet    = dclassify.DataSet;
    var Document   = dclassify.Document;
    
    // create some 'bad' test items (name, array of characteristics)
    var item1 = new Document('item1', ['a','b','c']);
    var item2 = new Document('item2', ['a','b','c']);
    var item3 = new Document('item3', ['a','d','e']);

    // create some 'good' items (name, characteristics)
    var itemA = new Document('itemA', ['c', 'd']);
    var itemB = new Document('itemB', ['e']);
    var itemC = new Document('itemC', ['b','d','e']);

    // create a DataSet and add test items to appropriate categories
    // this is 'curated' data for training
    var data = new DataSet();
    data.add('bad',  [item1, item2, item3]);    
    data.add('good', [itemA, itemB, itemC]);
    
    // an optimisation for working with small vocabularies
    var options = {
        applyInverse: true
    };
    
    // create a classifier
    var classifier = new Classifier(options);
    
    // train the classifier
    classifier.train(data);
    console.log('Classifier trained.');
    console.log(JSON.stringify(classifier.probabilities, null, 4));
    
    // test the classifier on a new test item
    var testDoc = new Document('testDoc', ['b','d', 'e']);    
    var result1 = classifier.classify(testDoc);
    console.log(result1);

The output of the above will be:

{
    "category": "good",
    "probability": 1,
    "timesMoreLikely": "Infinity",
    "secondCategory": "bad",
    "probabilities": [
        { "category": "good", "probability": 0.09876543209876543 },
        { "category": "bad", "probability": 0 }
    ]
}

Probabilities

The probabilities get calculated like this.

    {
        "bad": {
            "a": 1,
            "b": 0.6666666666666666,
            "c": 0.6666666666666666,
            "d": 0.3333333333333333,
            "e": 0.3333333333333333
        },
        "good": {
            "a": 0,
            "b": 0.3333333333333333,
            "c": 0.3333333333333333,
            "d": 0.6666666666666666,
            "e": 0.6666666666666666
        }
    }

Output

Standard results (without the applyInverse: true option) would look like this:

    {
        "category": "good",
        "probability": 0.6666666666666666,
        "timesMoreLikely": 2,
        "secondCategory": "bad",
        "probabilities": [
            { "category": "good", "probability": 0.14814814814814814},
            { "category": "bad", "probability": 0.07407407407407407}
        ]
    }

If you use the applyInverse: true option, the results are much more emphatic, because training indicates bad items never lack the "a" token.

    {
        "category": "good",
        "probability": 1,
        "timesMoreLikely": "Infinity",
        "secondCategory": "bad",
        "probabilities": [
            { "category": "good", "probability": 0.09876543209876543 },
            { "category": "bad", "probability": 0 }
        ]
    }

Intro to Machine Learning with Node.JS

View slides from a talk presented at OttawaJS.