Menu
▾
▴
#7 calculation of joint probability
Hello Everybody,
is there any function/method in the UnBBayes system for calculating the joint probability distribution of a set of Bayes Network nodes given some evidence? In other words, given a Bayesian Network with nodes A, B, C and D, I would like to calculate the probability distribution p(A, B | C = c1, D = d1), where c1 and d1 are some evidence for nodes C and D.
Please note that I would like to integrate such a functionality in a Java application, i.e., I would like to include the UnBBayes as a Java library in my project.
Thanks,
Vincenzo Lagani
Sure we can help. I just, kindly, ask that you post your questions in our forum so that other users can benefit from your question and our answer. Nevertheless, here is a link to one of the post from our forum that answers your question @ http://sourceforge.net/p/unbbayes/discussion/156016/thread/10b1a45b/.
Cheers,
Rommel
Thanks a lot for your prompt reply!
The example is pretty much illuminating. Unfortunately, it is not exactly what I need: the method "getMarginalAt()" returns the marginal probability for a single status of a single node. I need the joint probability of two or more nodes, instead.
A little bit more formally:
Given the BN nodes A, B, C, D, and some evidence a1 and b1 for A and B, the "getMarginal()" method can retrieve probabilities like:
p(C = c1 | A = a1, B = b1)
p(D = d2 | A = a1, B = b1)
etc.
what I need is instead:
p(C = c1, D = d2 | A = a1, B = b1)
Unfortunately, this joint probability is not a simple product of marginals when C and D are not d-separated by A and B.
To cut a long story short, I need to know if UnBBayes has any method that, given 2 or more nodes, is able to retrieve their joint probability distribution (after the propagation of some belief)
Thanks again,
Vincenzo
In the source code in SVN repository, we can find the method "getJointProbability" in the class JunctionTreeAlgorithm (line 1466 of https://sourceforge.net/p/unbbayes/code/2714/tree/trunk/UnBBayes/src/main/java/unbbayes/prs/bn/JunctionTreeAlgorithm.java), which can be used to query P(C = c1, D = d2).
So, I believe that in order to calculate P(C = c1, D = d2 | A = a1, B = b1), you may need to first insert and propagate the evidences A = a1, B = b1, and then call "getJointProbability" for P(C = c1, D = d2), so that the result becomes numerically equivalent to P(C = c1, D = d2 | A = a1, B = b1).
I'm not entirely sure, but there is a chance (actually, risk) that "getJointProbability" resets previous findings (so, calling "getJointProbability" after propagating "A = a1, B = b1" may cancel "A = a1, B = b1"). In such case, you may also want to absorb the findings "A = a1, B = b1" (i.e. delete nodes A and B by calling the method "unbbayes.prs.bn.ProbabilisticNetwork#removeNode") before calling "getJointProbability".
Best regards.
Thanks again for your help.
I have run some simple tests on the method "getJointProbability", and
the problem about previous findings (evidences) getting reset has never
arose.
However, I have observed another behavior that is probably incorrect.
Let's say we have a Bayes Network formed by two sub network. The two
graphs are not connected to each other, i.e., there are not edges
connecting any node of the first sub network with any node of the other
sub network.
If I set some evidence for any node of the first sub network, I expect
any marginal and joint probability in the second sub network not to change.
Unfortunately, it seems that this is not the case, as shown in the small
attached example.
Can you please tell me whether I am making any mistake or if there is
any problem with the "getJointProbability" method?
Best regards,
Vincenzo
On 8/9/2013 1:56 PM, cardialfly wrote:
Related
Support Requests: #7
Hi Vincenzo,
Could you please attach the network you are making these tests here? We will take a look at it.
Cheers,
Rommel
Hello, Vincenzo. Thanks for your attention on UnBBayes.
I'm starting to suspect that your issue is actually a rounding/precision problem of Java's floating point numbers. If so, I'm not sure if the effort of fixing it would worth the result.
UnBBayes does not assume that findings (evidences) comes from only 1 source/node. Therefore, when you propagate an evidence, it will attempt to settle global consistency of all cliques anyway (so, when you insert an evidence on a node and propagate, UnBBayes will check global consistency of other nodes regardless of being connected or not).
At that point (settling global consistency), UnBBayes multiplies clique potentials and then divide by separator potentials. Virtually, it does X = X*Y/Z in lots of places.
We know that X*Y/Z is mathematically equal to X if Y=Z (in this case, the value of X will remain unchanged), but in floating point operations, X*Y/Z may give slightly different results because of precision, especially if values are not multiple of 2, or when the order of operations may change, or due to lots of other possible sources of imprecision - because the inputs are not precisely the same for the system as a whole, if you insert an evidence somewhere.
When I ran your example (which I got from Rommel), I realized that when you propagated the evidence of node 5, the state 0 of node 13 changed from 0.68746004 to 0.68746006, and the state 0 of node 11 changed from 1.0 to 1.0000001 (OK, this may give some problems for applications calling UnBBayes, because they may include assertions like x <= 1.0). This really sounds to me like a precision problem, especially in UnBBayes, which was originally created as a GUI-based application whose the GUI only used (and still uses) 4 significant digits.
So, I'm afraid I have to ask you to deal with it (perhaps by using Math.round(), or simply accepting the "floating" precision)... Most of UnBBayes applications ran OK with this level of precision actually (this level of difference did not make differences in decisions, because I suppose Bayes nets don't encode model credibility anyway).
My apologies for any inconvenience, and pardon-me in case this wasn't helpful at all.
My best regards.
Last edit: Shou Matsumoto 2013-09-05
Hello,
thanks a lot for your very detailed answer. You are right when you say
that the marginals of single nodes do not change when some evidence is
set for a node that is not connected to them (at least do not change up
to the machine numerical precision).
Thus, on my toy example, when the evidence of node 5 is propagated, the
marginals of the nodes 13 and 11 do not change (node 5 is not connected
to nodes 13 and 11).
Unfortunately, this is not true for joint probabilities. This means that
the joint probability of state 0 for node 13 and state 1 for node 15 is
originally 0.035700217, but it becomes 0.0043083364 (significantly
different) when I set the evidence for node 5 to state 0.
I attached both my toy network and a java example file for your perusal.
Thanks again,
Vincenzo
import java.io.File;
import java.util.HashMap;
import unbbayes.io.NetIO;
import unbbayes.prs.bn.JunctionTreeAlgorithm;
import unbbayes.prs.bn.ProbabilisticNetwork;
import unbbayes.prs.bn.ProbabilisticNode;
public class TestJointProbability {
public static void main(String[] args) throws Exception { // open a .net file ProbabilisticNetwork net = (ProbabilisticNetwork)newNetIO().load(new File("toyNetwork.net"));
// prepare the algorithm to compile network JunctionTreeAlgorithm alg = new JunctionTreeAlgorithm(); alg.setNetwork(net); alg.run(); //specifying the nodes and states for inquiring about the jointdistribution
ProbabilisticNode node13 = (ProbabilisticNode)net.getNode("Node13");
int node13State = 0;
ProbabilisticNode node15 = (ProbabilisticNode)net.getNode("Node15");
int node15State = 1;
HashMap<ProbabilisticNode, Integer=""> map = new
HashMap<ProbabilisticNode, Integer="">();
map.put(node13, node13State);
map.put(node15, node15State);
//calling the joint probability method System.out.println("No evidence specified"); System.out.println("Joint probability for state " + node13State" of node " + node13.getDescription() + " and state " + node15State +
" of node " + node15.getDescription());
float jointProb = alg.getJointProbability(map);
System.out.println(jointProb);
System.out.println("Probability for state " + node13State + " of
node " + node13.getDescription());
System.out.println(node13.getMarginalAt(node13State));
System.out.println("Probability for state " + node15State + " of
node " + node15.getDescription());
System.out.println(node13.getMarginalAt(node15State));
System.out.println();
and 15)
ProbabilisticNode nodeTest=
(ProbabilisticNode)net.getNode("Node10");
System.out.println("Setting evidence for node " +
nodeTest.getDescription());
System.out.println();
nodeTest.addFinding(1); // the 1st state is now 100%
// propagate evidence net.updateEvidences(); //calling the joint probaility method jointProb = alg.getJointProbability(map); System.out.println("After specifying evidence for node11"); System.out.println("Joint probability for state " + node13State" of node " + node13.getDescription() + " and state " + node15State +
" of node " + node15.getDescription());
System.out.println(jointProb);
System.out.println("Probability for state " + node13State + " of
node " + node13.getDescription());
System.out.println(node13.getMarginalAt(node13State));
System.out.println("Probability for state " + node15State + " of
node " + node15.getDescription());
System.out.println(node13.getMarginalAt(node15State));
System.out.println();
//resetting evidence
net.resetEvidences();
alg.run();
//calling the joint probability method
System.out.println("Evidence resetted");
System.out.println("Joint probability for state " + node13State
+ " of node " + node13.getDescription() + " and state " + node15State +
" of node " + node15.getDescription());
jointProb = alg.getJointProbability(map);
System.out.println(jointProb);
System.out.println("Probability for state " + node13State + " of
node " + node13.getDescription());
System.out.println(node13.getMarginalAt(node13State));
System.out.println("Probability for state " + node15State + " of
node " + node15.getDescription());
System.out.println(node13.getMarginalAt(node15State));
System.out.println();
// insert evidence (finding) for node 5 (not connected to nodes
13 and 15)
ProbabilisticNode node5 = (ProbabilisticNode)net.getNode("Node5");
System.out.println("Setting evidence for node " +
node5.getDescription());
System.out.println();
node5.addFinding(0); // the 1st state is now 100%
// propagate evidence
net.updateEvidences();
//calling the joint probaility method
System.out.println("After specifying evidence for node5. Node5
is not connected to node 11 and 13");
jointProb = alg.getJointProbability(map);
System.out.println("Joint probability for state " + node13State
+ " of node " + node13.getDescription() + " and state " + node15State +
" of node " + node15.getDescription());
System.out.println(jointProb);
System.out.println("Probability for state " + node13State + " of
node " + node13.getDescription());
System.out.println(node13.getMarginalAt(node13State));
System.out.println("Probability for state " + node15State + " of
node " + node15.getDescription());
System.out.println(node13.getMarginalAt(node15State));
System.out.println();
}
}
net
{
node_size = (179 50);
label = "toyNetwork";
name = "toyNetwork";
}
node Node1
{
label = "Node1";
position = (100 100);
states = ( "0" "1");
}
node Node2
{
label = "Node2";
position = (200 100);
states = ( "0" "1");
}
node Node3
{
label = "Node3";
position = (300 100);
states = ( "0" "1" "2");
}
node Node4
{
label = "Node4";
position = (400 100);
states = ( "0" "1" "2");
}
node Node5
{
label = "Node5";
position = (500 100);
states = ( "0" "1" "2");
}
node Node6
{
label = "Node6";
position = (100 200);
states = ( "0" "1");
}
node Node7
{
label = "Node7";
position = (200 200);
states = ( "0" "1");
}
node Node8
{
label = "Node8";
position = (300 200);
states = ( "0" "1");
}
node Node9
{
label = "Node9";
position = (400 200);
states = ( "0" "1");
}
node Node10
{
label = "Node10";
position = (500 200);
states = ( "0" "1");
}
node Node11
{
label = "Node11";
position = (100 300);
states = ( "0" "1" "2");
}
node Node12
{
label = "Node12";
position = (200 300);
states = ( "0" "1" "2");
}
node Node13
{
label = "Node13";
position = (300 300);
states = ( "0" "1");
}
node Node14
{
label = "Node14";
position = (400 300);
states = ( "0" "1");
}
node Node15
{
label = "Node15";
position = (500 300);
states = ( "0" "1");
}
potential (Node1)
{
data = ( 0.50658 0.49342 );
}
potential (Node2 | Node3 Node4)
{
data = ((( 0.074074 0.92593 )( 0.35897 0.64103 )( 0.93939 0.060606
))(( 0.081481 0.91852 )( 0.44643 0.55357 )( 0.99057 0.009434 ))((
0.15556 0.84444 )( 0.41848 0.58152 )( 0.97059 0.029412 )));
}
potential (Node3 | Node4)
{
data = (( 0.12745 0.65686 0.21569 )( 0.20726 0.59722 0.19551 )(
0.10323 0.68065 0.21613 ));
}
potential (Node4)
{
data = ( 0.13989 0.64681 0.2133 );
}
potential (Node5 | Node2 Node3)
{
data = ((( 0.12621 0.86408 0.0097087 )( 0.010616 0.52654 0.46285 )(
0.02 0.48 0.5 ))(( 0.22368 0.76974 0.0065789 )( 0.15826 0.83716
0.0045872 )( 0.095238 0.89116 0.013605 )));
}
potential (Node6 | Node7)
{
data = (( 0.11805 0.88195 )( 0.99859 0.0014124 ));
}
potential (Node7 | Node5)
{
data = (( 0.58955 0.41045 )( 0.5442 0.4558 )( 0.35593 0.64407 ));
}
potential (Node8 | Node9)
{
data = (( 0.98908 0.010925 )( 0.98611 0.013889 ));
}
potential (Node9 | Node11)
{
data = (( 0.5 0.5 )( 0.94982 0.050182 )( 0.95714 0.042857 ));
}
potential (Node10 | Node11 Node12 Node13 Node14)
{
data = ((((( 0.5 0.5 )( 0.5 0.5 ))(( 0.5 0.5 )( 0.5 0.5 )))((( 0.5
0.5 )( 0.5 0.5 ))(( 0.5 0.5 )( 0.5 0.5 )))((( 0.5 0.5 )( 0.5 0.5 ))((
0.5 0.5 )( 0.5 0.5 ))))(((( 0.5 0.5 )( 0.5 0.5 ))(( 0.5 0.5 )( 0.5 0.5
)))((( 0.99753 0.0024691 )( 0.99254 0.0074627 ))(( 0.9977 0.0022989 )(
0.5 0.5 )))((( 0.5 0.5 )( 0.5 0.5 ))(( 0.5 0.5 )( 0.5 0.5 ))))(((( 0.5
0.5 )( 0.5 0.5 ))(( 0.5 0.5 )( 0.5 0.5 )))((( 0.5 0.5 )( 0.5 0.5 ))((
0.5 0.5 )( 0.5 0.5 )))((( 0.98 0.02 )( 0.8 0.2 ))(( 0.89474 0.10526 )(
0.5 0.5 )))));
}
potential (Node11)
{
data = ( 0.00069252 0.95152 0.047784 );
}
potential (Node12)
{
data = ( 0.00069252 0.95152 0.047784 );
}
potential (Node13)
{
data = ( 0.68746 0.31254 );
}
potential (Node14)
{
data = ( 0.90575 0.094248 );
}
potential (Node15 | Node14)
{
data = (( 0.94343 0.056575 )( 0.9927 0.0072993 ));
}
On 9/5/2013 3:37 AM, cardialfly wrote:
Related
Support Requests: #7
My apologies for the late reply.
I guess the getJointProbability is returning the joint probability of the nodes you specify in the arguments plus the evidences you inserted, so the last joint probability, for example, seems to be the joint probability of nodes 5 (evidence), 13, and 15 together...
I'll see if I can fix this issue quickly.
While the problem is not fixed, you may try using a workaround instead, which is to absorb the node with evidence, by deleting it:
ProbabilisticNetwork#removeNode(Node, boolean);
In your code, it would be something like:
net.removeNode(node5, true);
One issue I foresee in absorbing a node is that the node won't exist anymore; so, you may find strange issues while resetting evidences, for example.
Apologies for all the inconvenience.
Sincerely.
Hi all,
I have a problem regarding the computation of the joint probability of nodes. Specifically, although the JunctionTreeAlgorithm provides the joint probabilities at a first step, at a second step after I change the CPTs (through the getProbabilityFunction method of the ProbabilisticNode class and using the setup provided in the above code), and recalcuate the joint probabilities, they remain the same.
I am not sure what I am doing wrong. Do I miss something crucial regarding the junction tree algorithm class.
Thank you for your help,
Giannis
Hi, Giannis.
Could you make sure you recompiled the network (i.e. call the method in JunctionTreeAlgorithm which you supposedly used to compile the junction tree the first time)? The joint probability is calculated from a junction tree, which may have been built/compiled from the old network (i.e. the one before your change).
In UnBBayes (not only in it, though), Bayes nets and junction trees are different entities (they are like different structures trying to represent equivalent prob. distribution), so updating the CPT (a component in Bayes net) won't update the content of the junction tree unless you manually synchronize them.
Recompiling the junction tree forces such synchronization, or should I say it simply creates a new junction tree from the updated Bayes net.
Regards,
Hi cardialfly,
Thank you for your prompt reply!
Can you please explain how can I recompile the junction tree?
What I am doing at the beggining of each CPT update iteration is (before the update) to create a new JunctionTreeAlgorithm object (.. = new JunctionTreeAlgorithm() ) and then use the SetNet method to the CPT updated network.
Is there a specific method that I have to also call?
Thank you.
Hello, Giannis;
If I'm not wrong, calling the method JunctionTreeAlgorithm#run() in the object you instantiated should re-build the junction tree -- of the network you've set with JunctionTreeAlgorithm#setNet(ProbabilisticNetwork) -- for you.
Of course, I'm assuming the JunctionTreeAlgorithm and all objects aggregated to/with it were correctly instantiated.
Normally some exception should be thrown if there is some requirement not properly settled at this point (I can't imagine in advance what kind of problems may happen to you, because honestly I'm not sure how you've instantiated the 1st junction tree -- to obtain the joint probability before changes in CPT -- at all). You can try and see what happens.
Best regards.