# Marginal probabilities for each n-tuple (Migrated from community.research.microsoft.com)

• ### Question

• Lato posted on 04-05-2011 7:28 AM

Hi, I tried to look on website and forum for an answer to my question, without success. I want to get the marginal probabilities for each n-tuple of observations that I set in the model. Is it possible?

Friday, June 3, 2011 6:44 PM

• Lato replied on 04-15-2011 2:32 AM

Thanks John,

I read the guide when I tried to use and modify the generated code.

now i'm trying to solve my OutOfMemory problems following the guides and extending your code to calculate P (O1 | S1, H1)

Friday, June 3, 2011 6:46 PM

### All replies

• John Guiver replied on 04-05-2011 9:22 AM

Can you give more details? What is the model? Which marginals do you want to get?

Thanks

John

Friday, June 3, 2011 6:44 PM
• Lato replied on 04-05-2011 9:59 AM

I suppose this is my model.

the node p is directly connected to the nodes s1 and s2 (children nodes), s1 and s2, respectively, are directly connected with the nodes o1 and o2 as in figures

I can only observe the variables colored (O)
o range = 4
h range = 3
p range = 2

I set my values ​​observed for o1 and o2  and want to know for each pair of o1 and o2 the marginal distribution of another variable if it's possible.

Friday, June 3, 2011 6:44 PM
• Lato replied on 04-06-2011 3:13 AM

This is the model image

Friday, June 3, 2011 6:44 PM
• John Guiver replied on 04-07-2011 3:32 AM

Hi Alessandro

I don't think I fully understand your question, because, from looking at the model code you have sent us, you already know how to observed variables, and how to make an inference query, namely:

O1.ObservedValue = O1Data;
O2.ObservedValue = O2Data;

where O1Data and O2Data are int[] observations.

Then Infer S1 by engine.Infer(S1);

Perhaps again it would be best to make your current code available, and put a comment in the code as to where you are having difficulties.

John

Friday, June 3, 2011 6:44 PM
• Lato replied on 04-07-2011 9:26 AM

I need to infer values ​​for singles observations.

For example, if O1 is = {1,2,3,4,0,2,4,2, ...} and O2 = {1,2,2,2,2,0,2,3,0 .. .}

I need P (S1 | O1 = 1), P (S2 | O2 = 1) etc. ...

The next value will be

P (S1 | O1 = 2), P (S2 | O2 = 2)

etc. ..

so for all pairs of O1 and O2

same for P

Because I need to study the changes in the values ​​to use for text analysis, where text is converted into numbers.

if you prefer I can fix the code and share it as previously.

Thanks

Alessandro

Friday, June 3, 2011 6:45 PM
• John Guiver replied on 04-08-2011 3:56 AM

So assuming 'O1' and 'O2' are of type VariableArray<int>, and the size of their Range is an observed Variable<int> 'NumO', you can do:

NumO.ObservedValue = 1;
int[] O1Data = new int[] {1,2,3,4,0,2,4,2};
int[] O2Data = new int[] {1,2,2,2,2,0,2,3};
for (int i=0; i < O1Data.Length; i++)
{
O1.ObservedValue =
new int[] {O1Data[ i ]};
O2.ObservedValue =
new int[] {O2Data[ i ]};
}

Then query as before.

Friday, June 3, 2011 6:45 PM
• Lato replied on 04-08-2011 7:00 AM

Dear john,
Sorry for the confusion..
This is the link to my code
http://dl.dropbox.com/u/20337898/Work.zip
It contains the drawings of the models developed.
I would like  to extract the following conditional probabilities from the chain:

P(O=i|S=j) where i=1..range(O) and j=1..range(S)
the same for other hidden variables P and H.

I assume that i should access some internal structure of the gibbs sampling engine, is it correct?
Using the code you suggested, I have to wait for iterations of the inference engine for each set of observations?
I understand you correctly?

Alessandro

Friday, June 3, 2011 6:45 PM
• Lato replied on 04-11-2011 9:21 AM

I'm trying to understand and edit the file in the folder "generatedsource".

i think there is in this file generated the structure  i need.

Friday, June 3, 2011 6:45 PM
• John Guiver replied on 04-11-2011 11:56 AM

No - you should not in general be looking at or editing the generated source. Here is how you get P(O1 | S1) assuming everything else is unobserved:

public Discrete[] QueryO1FromS1()

{

// Set the CPTs qnd PTs to their poserior values

cpt_o1_prior.ObservedValue = cpt_o1_posterior;

cpt_o2_prior.ObservedValue = cpt_o2_posterior;

cpt_o3_prior.ObservedValue = cpt_o3_posterior;

cpt_s2_prior.ObservedValue = cpt_s2_posterior;

cpt_h2_prior.ObservedValue = cpt_h2_posterior;

cpt_s3_prior.ObservedValue = cpt_s3_posterior;

cpt_h3_prior.ObservedValue = cpt_h3_posterior;

pt_h1_prior.ObservedValue = pt_h1_posterior;

pt_s1_prior.ObservedValue = pt_s1_posterior;

// Clear any values we are not observing

o1.ClearObservedValue();

o2.ClearObservedValue();

o3.ClearObservedValue();

h1.ClearObservedValue();

h2.ClearObservedValue();

h3.ClearObservedValue();

s2.ClearObservedValue();

s3.ClearObservedValue();

int num_s1 = s1.GetValueRange().SizeAsInt;

Discrete[] p_o1_given_s1 = new Discrete[num_s1];

for (int j = 0; j < num_s1; j++)

p_o1_given_s1[j] = InfEngine.Infer<Discrete>(o1);

return p_o1_given_s1;

}

If you need joint distributions, you can use the following post for guidance: http://community.research.microsoft.com/forums/t/6507.aspx

If this is too slow for your purposes, instead of using the full posterior for the CPTs you can

1. Use less iterations on the inference engine
2. Set the observed values on the CPTs and PTs to the mean of the posteriors rather than to the full posteriors.

John

Friday, June 3, 2011 6:45 PM
• Lato replied on 04-12-2011 4:55 AM

reading the post you suggested me,

I understand that;
1) I define the model
2) ask P (S1 | 01), P (H1 | S1) P (S2 | O2, S1), etc. .. from the model
3) observe values

then I should not use the method
InferAll()  to compute the entire model,

But I did not understand how to integrate your code with mine. when  can I  call that method?

Friday, June 3, 2011 6:45 PM
• John Guiver replied on 04-13-2011 11:41 AM

I assume that you first want to infer the CPTs and PTs from different observations on O1, O2, O3. This is what you already have in your code.

Then you want to fix CPTs and PTs, and use the same model to make inferences in the absence of data. For example P(S1|O1) asks the model what is the marginal distribution over S1 (marginalising out all other variables), given that we observe O1. This is what the routine I gave you does.

If you want to know the general logic you might have in your main program, please refer to the original post on this at http://community.research.microsoft.com/forums/t/6357.aspx

John.

Friday, June 3, 2011 6:46 PM
• Lato replied on 04-14-2011 5:15 AM

I used your code in this sequence.

2) Get o1, o2 ,o3 observed values

3) CreateModel

4)InferParameters

5)QueryO1FromS1 (this method is in the same class as the others)

the line: p_o1_given_s1[j] = InfEngine.Infer<Discrete>(o1);

generates a argumentException in generated code.

Friday, June 3, 2011 6:46 PM
• John Guiver replied on 04-14-2011 7:39 AM

Sorry - I missed a couple of things out. Replace the last few lines by:

int num_s1 = s1.GetValueRange().SizeAsInt;

Discrete[] p_o1_given_s1 = new Discrete[num_s1];

NumCases.ObservedValue = 1;

for (int j = 0; j < num_s1; j++)

{

s1.ObservedValue = new int[] { j };

p_o1_given_s1[j] = InfEngine.Infer<Discrete[]>(o1)[0];

}

return p_o1_given_s1;

John

Friday, June 3, 2011 6:46 PM
• John Guiver replied on 04-14-2011 7:45 AM

By the way, if you really want to look at and understand the generated source, please (a) read the section in the user guide on the structure of the generated code, and (b) name all your variables using the Name property or the inline Named() method on variables (these names will then show up in the generated code).

John

Friday, June 3, 2011 6:46 PM
• Lato replied on 04-15-2011 2:32 AM

Thanks John,

I read the guide when I tried to use and modify the generated code.

now i'm trying to solve my OutOfMemory problems following the guides and extending your code to calculate P (O1 | S1, H1)

Friday, June 3, 2011 6:46 PM