[DL-Learner discussion] Dl-Learner/CrossValidation

[Faranak S. <f.s...@qm...>]

Dear Professor;
 I am using the DL-learner and I did face with few problems when I use
 cross-validation  which you might  possibly help me over that;

 The following is what I did so far;
 We want to evaluate 7 classes based on Cross validation
 Considering the 7 following Target classes with the number of instances
 for each of them in our ontology;
  Target C = {C_1: A (57), C_2: B (11),  C_3: C (8),C_4: D(63),  C_5:  E
 (30),  C_6: F (83)C_7: G (21)}

   A = {all the instances of the Target Classes = 273}, {p_1…, p_273}

  a)     i:=1
  posExamplesClass ={Instance of C_2}
  negExamples = Sets.difference(lp.getReasoner().getIndividuals(),
 posExamplesClass);  - all the indivisuals -{posExamplesClass}
  b) consider p_i and learn set of  L  for all classes from O \cup (A
 \setminus \{ p_i\}) %learn by taking out p_i From A
  c)     now compute precision, recall, F1 wrt O \cup A \cup L %test by
 taking now also p_i into account wrt the whole ontology O, A, and
 learned GCIs L.
  d)    i:= i+1
  e)    repeat a)- d) until i  273
   Eventually, compute micro, macro precision, recall, F1 of all n trials.

 I did use the provided CrossValidation sample code  in DL-Learner
 package and my main problem is;
 As an example, I do have 11 instances for the Target C_2 as
 posExamplesClass and 262 (273-11} instance for negative class, I won't
 be able to get the proper learning solution and all I get is so general
 concepts. (Obviously, the number of positive is much less than negExamples).

 Also as I can have *n* different solutions, in order to select the best
 the one I use;
   List<OWLClassExpression list = la.getCurrentlyBestDescriptions();
   OWLClassExpression BestDescription = list.get(*n*);

 A couple of things;
 A) Does it make any difference in the solution if the LP considered only positive? I did define the LP as following;
  PosNegLPStandard lp = new PosNegLPStandard(rc, posExamples , negExamples);
  Can you please let me know how I can define LP with only positive classes?

  B) Obviously, the *n* (number of the best solution ) is changing for
 each Target class but another problem  is this*n*keepsp changing in
 every Fold as well.
 Imagine for the Target class C_1 we have the soulotuin
  Fold 1,   solutions  *2:* Father and has kids,  n=2
  Fold 2,  solutions   *5:* Father and has kids,  n=5

 Then I have to change *n* in the following for each fold?
  List<OWLClassExpression list = la.getCurrentlyBestDescriptions();
   OWLClassExpression BestDescription = list.get(*n*);

 How do you recommend me in this case and also how do you define the best
 solution generally?



--
Faranak Sobhani
PhD Electronic Engineering Student
MMV (Multimedia and Vision) Lab
School of Electronic Engineering and Computer Science
Queen Mary University of London
Mile End Road, London E1 4NS, UK