3
$\begingroup$

I'm attempting to build a classifier using complex strings as one of the variables.

Modified Titanic data as an example:

trainingset = Dataset[{<|"age" -> 32, "gender" ->"female","name"->"Anna Banana"|>, <|"age" -> 41, "gender" -> "female","name"->"Suzy Banana"|>, <|"age" -> 30, "gender" -> "female","name"->"Jane Apple"|>, <|"age" -> 21, "gender" -> "male","name"->"John Apple"|>, <|"age" -> 11, "gender" -> "male","name"->"David Orange"|>, <|"age" -> 52,"gender" -> "female","name"->"Anna Orange"|>}] classifier = Classify[trainingset -> "gender"]

How would you train the classifier to use each word in the "name" variable to influence the prediction?

Using variable "name" alone does not appear to be enough to make a prediction:

In= c[<|"name" -> "John Apple"|>] Out= female In= c[<|"name" -> "John Apple", "age" -> 20|>] Out= male

This feature in the documentation for Classify seems to suggest it is already capable of finding keywords:

enter image description here

Improve this question
$\endgroup$
8
  • $\begingroup$ Do you want to use Classify or any other classifier would be fine? $\endgroup$ Commented May 9, 2016 at 15:18
  • $\begingroup$ @AntonAntonov I'd like to stick to Classify if possible, but I'm open to suggestions. $\endgroup$ Commented May 9, 2016 at 15:25
  • $\begingroup$ I was going to suggest to get the conditional probabilities from a mosaic plot as shown in the gray rectangle in this image from the discussion "How can I determine the importance of variables from Classify?". Sticking to Classify, it might be easier to build several ones, one for the full dataset, the others for the partial records versions. (If the number of variables is small enough.) $\endgroup$ Commented May 9, 2016 at 15:49
  • $\begingroup$ @AntonAntonov I will read that post again, thank you. Your post here is inspiring: mathematicaforprediction.wordpress.com/2014/03/30/… Concerning Classify, you suggest attempting to build additional classifiers for those complex strings? Concerning the number of variables, that remains, part of my dilemma. I'm aiming to use 10 variables total, 2 of them being complex strings as I described above. I estimate the number of variables within those strings to be less than 10 on average. $\endgroup$ Commented May 9, 2016 at 16:31
  • 1
    $\begingroup$ May be it is better for you to use frequent sets mining algorithms like Apriori (for associations rules mining) applied in the blog post you linked, "Classification and association rules for census income data", and in "Movie genre associations". Association rules mining does not require full arrays of data, we can use general lists of lists. $\endgroup$ Commented May 9, 2016 at 16:39

2 Answers 2

Reset to default
2
$\begingroup$

Your fundamental problem is that toy training set is far too small to produce a reliable classification function. The following works a little better but is still prone to getting it wrong on occasion. 

trainingset11 = Dataset[{<|"age" -> 32, "gender" -> "female", "name" -> "Anna Banana"|>, <|"age" -> 41, "gender" -> "female", "name" -> "Suzy Banana"|>, <|"age" -> 30, "gender" -> "female", "name" -> "Jane Apple"|>, <|"age" -> 21, "gender" -> "male", "name" -> "John Apple"|>, <|"age" -> 11, "gender" -> "male", "name" -> "David Orange"|>, <|"age" -> 52, "gender" -> "female", "name" -> "Anna Orange"|>, <|"age" -> Missing, "gender" -> "male", "name" -> "James Apple"|>, <|"age" -> Missing, "gender" -> "female", "name" -> "Fiona Banana"|>}] classifier11 = Classify[trainingset11 -> "gender"] **male** classifier11[<|"age" -> Missing, "name" -> "Jane Apple"|>] **female**

Note that just 1 change of "Fiona Banana" in the above set is enough to flip the Age classifier from RandomForest to NaiveBayes. Also note that with a few more data points the "name" field is treated as a "text" field thus string processing can be applied.

ClassifierInformation[classifier11, FeatureTypes] **<|"age" -> "Nominal", "name" -> "Text"|> **

I think you might find it useful to set the Classifier methods manually

classifier12 = Classify[trainingset11 -> "gender", Method -> {"age" -> "NaiveBayes", "name" -> "Markov"}]

You might also find it useful to measure the accuracy as per the "Monks" example in the Classifier help page.

Improve this answer
$\endgroup$
2
  • $\begingroup$ classifier12 =Classify[trainingset11 -> "gender", Method {"age" -> "NaiveBayes", "name" -> "Markov"}] is more correct I believe. $\endgroup$ Commented May 12, 2016 at 14:37
  • $\begingroup$ Actually the Method-> format is the one used in the Help pages. Your example throws an error for me. $\endgroup$ Commented May 13, 2016 at 6:56
2
$\begingroup$

This answer has three parts: the first and second use nearest neighbors like voting, the third uses the Apriori algorithm for Association Rules Mining. All use linear vector space representation and categorization of the numerical variable(s).

The ideas of the first and third parts can be found in the document "Importance of variables investigation guide".

The second part uses Classify and it is a modification of the first.

Data

Let us get Titanic data and categorize the age variable.

titanicDataset = Flatten /@ Apply[List, ExampleData[{"MachineLearning", "Titanic"}, "Data"], {1}]; Dimensions[titanicDataset] (* {1309, 4} *) titanicVarNames = Flatten[List @@ ExampleData[{"MachineLearning", "Titanic"}, "VariableDescriptions"]] (* {"passenger class", "passenger age", "passenger sex", "passenger survival"} *) titanicDatasetCatAge = titanicDataset; ageQF = Piecewise[{{1, -\[Infinity] < #1 <= 5}, {2, 5 < #1 <= 14}, {3, 14 < #1 <= 21}, {4, 21 < #1 <= 28}, {5, 28 < #1 <= 35}, {6, 35 < #1 <= 50}, {7, 50 < #1 <= \[Infinity]}}, 0] &; titanicDatasetCatAge[[All, 2]] = Map[If[MissingQ[#], 0, ageQF[#]] &, titanicDatasetCatAge[[All, 2]]] /. {1 -> "1(0-6)", 2 -> "2(6-14)", 3 -> "3(15-21)", 4 -> "4(22-28)", 5 -> "5(29-35)", 6 -> "6(36-50)", 7 -> "7(50+)", 0 -> "0(missing)"};

This is how the data titanicDatasetCatAge looks like:

enter image description here

Class determination by tallying

The idea is

  1. to make a linear vector space representation of data, and

  2. for a qiven query list of attributes that do not make a full record, to find all records that have those attributes and use voting to determine the class for the query list.

Vector space representation

titanicDatasetCatAge = DeleteCases[titanicDatasetCatAge, {___, _Missing, ___}]; docs = Map[ToString, titanicDatasetCatAge, {-1}]; RandomSample[docs, 4] (* {{"2nd", "6(36-50)", "female", "survived"}, {"1st", "3(15-21)", "female", "survived"}, {"1st", "5(29-35)", "female", "survived"}, {"3rd", "1(0-6)", "male", "died"}} *) cTerms = Union[Flatten[docs]]; cTermToIndexRules = Thread[cTerms -> Range[Length[cTerms]]]; cMat = SparseArray[ Flatten@MapIndexed[ Thread[Thread[{#2[[1]], #[[All, 1]] /. cTermToIndexRules}] -> #[[All, 2]]] &, Tally /@ docs]];

Class label tally finding function

Clear[ClassLabelTally] ClassLabelTally[{cMat_SparseArray, termToIndexRules : {_Rule ..}}, classLabels : {_String ..}, query_: {_String ..}] := Block[{ds, nnFunc, nnInds, qvec, res}, qvec = SparseArray[Thread[(query /. termToIndexRules) -> 1], Dimensions[cMat][[2]]]; nnInds = Pick[Range[Dimensions[cMat][[1]]], # >= Total[qvec] & /@ Flatten[cMat.qvec]]; res = Transpose[{classLabels, Normal[Total[cMat[[nnInds, classLabels /. termToIndexRules]]]]}]; res[[All, 2]] = N[res[[All, 2]]/Total[res[[All, 2]]]]; res ];

Examples of use

ClassLabelTally[{cMat, cTermToIndexRules}, {"died", "survived"}, {"female"}] (* {{"died", 0.272532}, {"survived", 0.727468}} *) ClassLabelTally[{cMat, cTermToIndexRules}, {"died", "survived"}, {"male"}] (* {{"died", 0.809015}, {"survived", 0.190985}} *) ClassLabelTally[{cMat, cTermToIndexRules}, {"died", "survived"}, {"male", "3rd"}] (* {{"died", 0.84787}, {"survived", 0.15213}} *) ClassLabelTally[{cMat, cTermToIndexRules}, {"died", "survived"}, {"female", "2nd"}] (* {{"died", 0.113208}, {"survived", 0.886792}} *) ClassLabelTally[{cMat, cTermToIndexRules}, {"died", "survived"}, {"female", "3(15-21)"}] (* {{"died", 0.289855}, {"survived", 0.710145}} *) ClassLabelTally[{cMat, cTermToIndexRules}, {"died", "survived"}, {"female", "3(15-21)", "1st"}] (* {{"died", 0.}, {"survived", 1.}} *)

Modification with Classify

The approach above can be modified to use Classify.

First we make a classifier:

cf = Classify[ titanicDatasetCatAge[[All, {1, 2, 3}]] -> titanicDatasetCatAge[[All, -1]]]

We can define a function to run the classifier over instances containing the set of variable features:

Clear[VarFeaturesClassify] VarFeaturesClassify[{cf_, data_, n_}, {cMat_SparseArray, termToIndexRules : {_Rule ..}}, classLabels : {_String ..}, query_: {_String ..}] := Block[{ds, nnFunc, nnInds, qvec, res}, qvec = SparseArray[Thread[(query /. termToIndexRules) -> 1], Dimensions[cMat][[2]]]; nnInds = Pick[Range[Dimensions[cMat][[1]]], # >= Total[qvec] & /@ Flatten[cMat.qvec]]; res = cf[data[[#]], "TopProbabilities"] & /@ RandomSample[nnInds, If[IntegerQ[n], UpTo[n], All]]; Map[#[[1, 1]] -> Mean[#[[All, 2]]] &, GatherBy[Flatten[res], #[[1]] &]] ];

Here are examples of use:

VarFeaturesClassify[{cf, titanicDatasetCatAge[[All, {1, 2, 3}]], 20}, {cMat, cTermToIndexRules}, {"died", "survived"}, {"female", "2nd"}] (* {"survived" -> 0.828833, "died" -> 0.176815} *) VarFeaturesClassify[{cf, titanicDatasetCatAge[[All, {1, 2, 3}]], All}, {cMat, cTermToIndexRules}, {"died", "survived"}, {"female", "2nd"}] (* {"survived" -> 0.853315, "died" -> 0.152542} *) VarFeaturesClassify[{cf, titanicDatasetCatAge[[All, {1, 2, 3}]], 20}, {cMat, cTermToIndexRules}, {"died", "survived"}, {"female", "3(15-21)", "1st"}] (* {"survived" -> 0.984931} *)

Using Apriori

The code below closely follows the code in the document "Importance of variables investigation guide" pages 20-22.

Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/AprioriAlgorithm.m"] \[Mu] = 0.02; Print["Number of records corresponding to \[Mu]=", \[Mu], ": ", Length[titanicDatasetCatAge]*\[Mu]] Print["Computation time:", AbsoluteTiming[ {aprioriRes, itemToIDRules, idToItemRules} = AprioriApplication[titanicDatasetCatAge, \[Mu]]; ][[1]]]; Grid[Prepend[ Tally[Map[Length, Join @@ aprioriRes]], {"frequent set\nlength", "number of\nfrequent sets"}], Dividers -> {None, {True, True}}]

enter image description here

items = {"survived"}; itemRules = ItemRules[titanicDatasetCatAge, aprioriRes, itemToIDRules, idToItemRules, items, 0.7, 0.02];

The following command tabulates those of the rules that have "survived" as a consequent. The rules are sorted according to their confidence.

Magnify[#, 0.7] &@Grid[ Prepend[ SortBy[Select[Join @@ itemRules, MemberQ[items, #[[-1, 1]]] && #[[2]] > 0.7 && 2 <= Length[#[[-2]]] <= 10 &], -#[[2]] &], Map[Style[#, Blue, FontFamily -> "Times"] &, {"Support", "Confidence", "Lift", "Leverage", "Conviction", "Antecedent", "Consequent"}] ], Alignment -> Left]

enter image description here

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Thank you for the educational response. I am going to hold off on marking this as an accepted answer until I am able to directly apply this principles to my data. $\endgroup$ Commented May 10, 2016 at 19:39
  • $\begingroup$ This doesn't really answer the users question. It should be possible to do what they want with Classify. $\endgroup$ Commented May 12, 2016 at 5:48
  • $\begingroup$ @GordonCoale Read my first comment to the question and OP's answer -- non-Classify solutions are of interest. Your comment, though, reminded me that I did come up with a solution that uses Classify ... $\endgroup$ Commented May 12, 2016 at 8:09
  • $\begingroup$ @AntonAntonov - I would be interested in seeing a Classify example. I did a fair bit of playing around before posting my - unsatisfactory imo - answer. $\endgroup$ Commented May 12, 2016 at 10:25
  • 1
    $\begingroup$ @GordonCoale I did post an update about using Classify one hour ago -- see the section "Modification with Classify" in my answer. $\endgroup$ Commented May 12, 2016 at 10:35

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.