# Bayes network, gaussian algorithm infer.net with multiple nodes

• ### Question

• Hello all,

I have a similar problem with the usual <<Wet grass/sprinkler/rain>> example but I have multiple nodes that they are not True or False and we have many different discrete  variables.

If there is anyone who can help us we would be great full .

Thanks

The example with nodes and values is below:

Sunday, January 15, 2012 6:05 PM

• Hi Antrikos

What sort of help are you looking for? The WetGrassSprinklerRain example is deliberately set up to show the general case of discrete variables (ie not restriceted to true/false) and the general AddChildFromOneParent and AddChildFromTwoParents are general methods for adding children. In addition there are several posts about extending this to more parents.

As regards your specific CPTs above, whether you observe these CPTs or whether you want to learn them, you can use the same pattern as in the example. But I am a bit confused by your tables. For example, smoking is conditioned on nightlife; in the cpt, the probabilities for the different smoking events (1...5, etc) should add to 1. You can then set up the observed distributions for cptSmoking similarly to cptSprinkler except the vectors are of length 5 represent the 5 smoking bins.

Let me know if you need further clarification.

John

Monday, January 16, 2012 2:14 PM
• Hi Antrikos

What sort of help are you looking for? The WetGrassSprinklerRain example is deliberately set up to show the general case of discrete variables (ie not restriceted to true/false) and the general AddChildFromOneParent and AddChildFromTwoParents are general methods for adding children. In addition there are several posts about extending this to more parents.

As regards your specific CPTs above, whether you observe these CPTs or whether you want to learn them, you can use the same pattern as in the example. But I am a bit confused by your tables. For example, smoking is conditioned on nightlife; in the cpt, the probabilities for the different smoking events (1...5, etc) should add to 1. You can then set up the observed distributions for cptSmoking similarly to cptSprinkler except the vectors are of length 5 represent the 5 smoking bins.

Let me know if you need further clarification.

John

Hello,

First of all I would like to thank you for your time and help.

Firstly our problem has four nodes:

1<sup>st</sup> node: nightlife with variables <<TRUE, FALSE>>

2<sup>nd</sup> node: smoking with variables <<1,5,10,20,NO>>

3<sup>rd</sup> node: alcohol with variables <<YES, NO>>

4<sup>th</sup> node: HealthType with variables <<NORMAL, RISKY, CRITICAL>> where we have the final probabilities all together as you can see from the table above.

The difference from the sprinkler example is that our nodes are not of a <<bool>> type so I do not know how to type the commands correctly. This is my major problem.

For example smoking is not a <<bool>> type node and it takes five variables.

My question is for example  “What is the HealthType if the nightlife is (YES), smoking is (5) and alcohol is (YES) ? ”. I need the result for the HealthType if it is <<NORMAL, RISKY, CRITICAL>> and its corresponding probability.

Thank you in advance, let me know if you need further clarification.

Antrikos

Monday, January 23, 2012 11:38 AM
• Make sure you are looking at the example provided under the Start menu:  http://research.microsoft.com/en-us/um/cambridge/projects/infernet/docs/Discrete%20Bayesian%20network.aspx
Monday, January 23, 2012 4:46 PM
• Hi Antrikos

You can pretty much substitute your specification directly into the code. The sprinkler example uses ints rather than bools for that very purpose. Here is the sprinkler code with the variables renamed to your nodes, and with your given CPTs. Just copy and paste the code below - you can run it in any of the three modes, namely:

• Query the model when we know the parameters exactly
• Learn the posteriors of the CPTs
• Query the model taking into account parameter uncertainty

The only other thing I needed to change was your Smoking table - as I needed to normalise the columns to add to 1.

```using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using MicrosoftResearch.Infer.Models;
using MicrosoftResearch.Infer;
using MicrosoftResearch.Infer.Distributions;
using MicrosoftResearch.Infer.Maths;

namespace Nightlife
{
public class NightlifeModel
{
public static int NumNightlife = 2;
public static int NumSmoking = 5;
public static int NumAlcohol = 2;
public static int NumHealthType = 3;

// Primary random variables
public VariableArray<int> Nightlife;
public VariableArray<int> Smoking;
public VariableArray<int> Alcohol;
public VariableArray<int> HealthType;
public Variable<int> NumberOfExamples;

// Random variables representing the parameters of the distributions
// of the primary random variables. For child variables, these are
// in the form of conditional probability tables (CPTs)
public Variable<Vector> ProbNightlife;
public VariableArray<Vector> CPTSmoking;
public VariableArray<Vector> CPTAlcohol;
public VariableArray<VariableArray<Vector>, Vector[][]> CPTHealthType;

// Prior distributions for the probability and CPT variables.
// The prior distributions are formulated as Infer.NET variables
// so that they can be set at runtime without recompiling the model
public Variable<Dirichlet> ProbNightlifePrior;
public VariableArray<Dirichlet> CPTSmokingPrior;
public VariableArray<Dirichlet> CPTAlcoholPrior;
public VariableArray<VariableArray<Dirichlet>, Dirichlet[][]> CPTHealthTypePrior;

// Posterior distributions for the probability and CPT variables.
public Dirichlet ProbNightlifePosterior;
public Dirichlet[] CPTSmokingPosterior;
public Dirichlet[] CPTAlcoholPosterior;
public Dirichlet[][] CPTHealthTypePosterior;

// Inference engine
public InferenceEngine Engine = new InferenceEngine();

/// <summary>
/// Constructs a new model
/// </summary>
public NightlifeModel()
{
// Set up the ranges
NumberOfExamples = Variable.New<int>().Named("NofE");
Range N = new Range(NumberOfExamples).Named("N");

Range L = new Range(NumNightlife).Named("L");
Range S = new Range(NumSmoking).Named("S");
Range A = new Range(NumAlcohol).Named("A");
Range H = new Range(NumHealthType).Named("H");

// Define the priors and the parameters
ProbNightlifePrior = Variable.New<Dirichlet>().Named("ProbNightlifePrior");
ProbNightlife = Variable<Vector>.Random(ProbNightlifePrior).Named("ProbNightlife");
ProbNightlife.SetValueRange(L);
// Smoking probability table conditioned on night life
CPTSmokingPrior = Variable.Array<Dirichlet>(L).Named("CPTSmokingPrior");
CPTSmoking = Variable.Array<Vector>(L).Named("CPTSmoking");
CPTSmoking[L] = Variable<Vector>.Random(CPTSmokingPrior[L]);
CPTSmoking.SetValueRange(S);
// Alcohol probability table conditioned on night life
CPTAlcoholPrior = Variable.Array<Dirichlet>(L).Named("CPTAlcoholPrior");
CPTAlcohol = Variable.Array<Vector>(L).Named("CPTAlcohol"); ;
CPTAlcohol[L] = Variable<Vector>.Random(CPTAlcoholPrior[L]);
CPTAlcohol.SetValueRange(A);
// Health type probability table conditioned on Smoking and Alcohol
CPTHealthTypePrior = Variable.Array(Variable.Array<Dirichlet>(A), S).Named("CPTHealthTypePrior");
CPTHealthType = Variable.Array(Variable.Array<Vector>(A), S).Named("CPTHealthType");
CPTHealthType[S][A] = Variable<Vector>.Random(CPTHealthTypePrior[S][A]);
CPTHealthType.SetValueRange(H);

// Define the primary variables
Nightlife = Variable.Array<int>(N).Named("Nightlife");
Nightlife[N] = Variable.Discrete(ProbNightlife).ForEach(N);
}

/// <summary>
/// Learns the parameters of the cloud/Smoking/Alcohol example
/// </summary>
/// <param name="nightlife">Cloudiness data</param>
/// <param name="smoking">Smoking data</param>
/// <param name="alcohol">Alcohol data</param>
/// <param name="healthType">Health type data</param>
/// <param name="probNightlifePrior">Prior for cloudiness probability vector</param>
/// <param name="cptSmokingPrior">Prior for Smoking conditional probability table</param>
/// <param name="cptAlcoholPrior">Prior for Alcohol conditional probability table</param>
/// <param name="cptHealthTypePrior">Prior for health type conditional probability table</param>
public void LearnParameters(
int[] nightlife,
int[] smoking,
int[] alcohol,
int[] healthType,
Dirichlet probNightlifePrior,
Dirichlet[] cptSmokingPrior,
Dirichlet[] cptAlcoholPrior,
Dirichlet[][] cptHealthTypePrior
)
{
NumberOfExamples.ObservedValue = nightlife.Length;
Nightlife.ObservedValue = nightlife;
Smoking.ObservedValue = smoking;
Alcohol.ObservedValue = alcohol;
HealthType.ObservedValue = healthType;
ProbNightlifePrior.ObservedValue = probNightlifePrior;
CPTSmokingPrior.ObservedValue = cptSmokingPrior;
CPTAlcoholPrior.ObservedValue = cptAlcoholPrior;
CPTHealthTypePrior.ObservedValue = cptHealthTypePrior;
// Inference
ProbNightlifePosterior = Engine.Infer<Dirichlet>(ProbNightlife);
CPTSmokingPosterior = Engine.Infer<Dirichlet[]>(CPTSmoking);
CPTAlcoholPosterior = Engine.Infer<Dirichlet[]>(CPTAlcohol);
CPTHealthTypePosterior = Engine.Infer<Dirichlet[][]>(CPTHealthType);
}

/// <summary>
/// Learns the parameters of the cloud/Smoking/Alcohol example assuming uniform priors
/// </summary>
/// <param name="nightlife">Cloudiness data</param>
/// <param name="Smoking">Smoking data</param>
/// <param name="alcohol">Alcohol data</param>
/// <param name="healthType">Health type data</param>
public void LearnParameters(
int[] nightlife,
int[] smoking,
int[] alcohol,
int[] healthType
)
{
// Set all priors to uniform
Dirichlet probNightlifePrior = Dirichlet.Uniform(NumNightlife);
Dirichlet[] probSmokingPrior = Enumerable.Repeat(Dirichlet.Uniform(NumSmoking), NumNightlife).ToArray();
Dirichlet[] probAlcoholPrior = Enumerable.Repeat(Dirichlet.Uniform(NumAlcohol), NumNightlife).ToArray();
Dirichlet[][] probHealthTypePrior = Enumerable.Repeat(Enumerable.Repeat(Dirichlet.Uniform(NumHealthType), NumAlcohol).ToArray(), NumSmoking).ToArray();

LearnParameters(nightlife, smoking, alcohol, healthType, probNightlifePrior, probSmokingPrior, probAlcoholPrior, probHealthTypePrior);
}

/// <summary>
/// Returns the probability of Alcohol given optional readings on
/// cloudiness, smoking, and health type, and given prior distributions
/// over the parameters. Priors may be manually set, or may be the posteriors
/// from learning the parameters.
/// </summary>
/// <param name="nightlife">Optional observation of nightlife category</param>
/// <param name="smoking">Optional observation of smoking category</param>
/// <param name="healthType">Optional observation of health type</param>
/// <param name="probNightlifePrior">Prior distribution over night life probability vector</param>
/// <param name="cptSmokingPrior">Prior distribution over smoking conditional probability table</param>
/// <param name="cptAlcoholPrior">Prior distribution over alcohol conditional probability table</param>
/// <param name="cptHealthTypePrior">Prior distribution over health type conditional probability table</param>
/// <returns>Probability of alcohol category</returns>
public double ProbAlcohol(
int? nightlife,
int? smoking,
int? healthType,
Dirichlet probNightlifePrior,
Dirichlet[] cptSmokingPrior,
Dirichlet[] cptAlcoholPrior,
Dirichlet[][] cptHealthTypePrior)
{
NumberOfExamples.ObservedValue = 1;
if (nightlife.HasValue)
Nightlife.ObservedValue = new int[] { nightlife.Value };
else
Nightlife.ClearObservedValue();
if (smoking.HasValue)
Smoking.ObservedValue = new int[] { smoking.Value };
else
Smoking.ClearObservedValue();
if (healthType.HasValue)
this.HealthType.ObservedValue = new int[] { healthType.Value };
else
this.HealthType.ClearObservedValue();
this.Alcohol.ClearObservedValue();

ProbNightlifePrior.ObservedValue = probNightlifePrior;
CPTSmokingPrior.ObservedValue = cptSmokingPrior;
CPTAlcoholPrior.ObservedValue = cptAlcoholPrior;
CPTHealthTypePrior.ObservedValue = cptHealthTypePrior;

// Inference
var alcoholPosterior = Engine.Infer<Discrete[]>(Alcohol);
// In this example, index 0 is true and index 1 is false
return alcoholPosterior[0].GetProbs()[0];
}

/// <summary>
/// Returns the probability of Alcohol given optional readings on
/// cloudiness, Smoking, and Health type, and given known parameters.
/// </summary>
/// <param name="nightlife">Optional observation of nightlife</param>
/// <param name="smoking">Optional observation of smoking</param>
/// <param name="healthType">Optional observation or health type</param>
/// <param name="probNightlifePrior">Cloudiness probability vector</param>
/// <param name="cptSmokingPrior">Smoking conditional probability table</param>
/// <param name="cptAlcoholPrior">Alcohol conditional probability table</param>
/// <param name="cptHealthTypePrior">Health type conditional probability table</param>
/// <returns>Probability that it has Alcoholed</returns>
public double ProbAlcohol(
int? nightlife,
int? smoking,
int? healthType,
Vector probNightlife,
Vector[] cptSmoking,
Vector[] cptAlcohol,
Vector[][] cptHealthType)
{
var probNightlifePrior = Dirichlet.PointMass(probNightlife);
var cptSmokingPrior = cptSmoking.Select(v => Dirichlet.PointMass(v)).ToArray();
var cptAlcoholPrior = cptAlcohol.Select(v => Dirichlet.PointMass(v)).ToArray();
var cptHealthTypePrior = cptHealthType.Select(va => va.Select(v => Dirichlet.PointMass(v)).ToArray()).ToArray();
return ProbAlcohol(nightlife, smoking, healthType, probNightlifePrior, cptSmokingPrior, cptAlcoholPrior, cptHealthTypePrior);
}

/// <summary>
/// Sample the model
/// </summary>
/// <param name="numData">Number of data in sample</param>
/// <param name="probNightlifePrior">Cloudiness probability vector</param>
/// <param name="cptSmokingPrior">Smoking conditional probability table</param>
/// <param name="cptAlcoholPrior">Alcohol conditional probability table</param>
/// <param name="cptHealthTypePrior">Health type grass conditional probability table</param>
/// <returns></returns>
public static int[][] Sample(
int numData,
Vector probNightlife,
Vector[] cptSmoking,
Vector[] cptAlcohol,
Vector[][] cptHealthType)
{
int[][] sample = new int[4][];
for (int i = 0; i < 4; i++)
sample[i] = new int[numData];
for (int j = 0; j < numData; j++)
{
int nightlife = Discrete.Sample(probNightlife);
int smoking = Discrete.Sample(cptSmoking[nightlife]);
int alcohol = Discrete.Sample(cptAlcohol[nightlife]);
int healthType = Discrete.Sample(cptHealthType[smoking][alcohol]);
sample[0][j] = nightlife;
sample[1][j] = smoking;
sample[2][j] = alcohol;
sample[3][j] = healthType;
}
return sample;
}

/// <summary>
/// Helper method to add a child from one parent
/// </summary>
/// <param name="parent">Parent (a variable array over a range of examples)</param>
/// <param name="cpt">Conditional probability table</param>
/// <returns></returns>
VariableArray<int> parent,
VariableArray<Vector> cpt)
{
var n = parent.Range;
var child = Variable.Array<int>(n);
using (Variable.ForEach(n))
using (Variable.Switch(parent[n]))
child[n] = Variable.Discrete(cpt[parent[n]]);
return child;
}

/// <summary>
/// Helper method to add a child from two parents
/// </summary>
/// <param name="parent1">First parent (a variable array over a range of examples)</param>
/// <param name="parent2">Second parent (a variable array over the same range)</param>
/// <param name="cpt">Conditional probability table</param>
/// <returns></returns>
VariableArray<int> parent1,
VariableArray<int> parent2,
VariableArray<VariableArray<Vector>, Vector[][]> cpt)
{
var n = parent1.Range;
var child = Variable.Array<int>(n);
using (Variable.ForEach(n))
using (Variable.Switch(parent1[n]))
using (Variable.Switch(parent2[n]))
child[n] = Variable.Discrete(cpt[parent1[n]][parent2[n]]);
return child;
}
}

class Program
{
static void Main(string[] args)
{
// Set random seed for repeatable example
Rand.Restart(12347);

// Create a new model
NightlifeModel model = new NightlifeModel();
model.Engine.ShowProgress = false;

// In this example, each variable just takes two states - true (index 0)
// and false (index 1). The example is written so that extensions to
// problems of more than two states is straightforward.

// -------------------------------------------------------------
// Usage 1: Query the model when we know the parameters exactly
// -------------------------------------------------------------
Console.WriteLine("\n*********************************************");
Console.WriteLine("Querying the model with a known ground truth");
Console.WriteLine("*********************************************");

Vector probNightlife = Vector.FromArray(0.2, 0.35);
probNightlife.SetToProduct(probNightlife, 1.0 / probNightlife.Sum());

Vector probSmokingGivenNightlifeTrue = Vector.FromArray(0.8, 0.85, 0.9, 0.95, 0.15);
probSmokingGivenNightlifeTrue.SetToProduct(probSmokingGivenNightlifeTrue, 1.0 / probSmokingGivenNightlifeTrue.Sum());
Vector probSmokingGivenNightlifeFalse = Vector.FromArray(0.2, 0.15, 0.1, 0.05, 0.9);
probSmokingGivenNightlifeFalse.SetToProduct(probSmokingGivenNightlifeFalse, 1.0 / probSmokingGivenNightlifeFalse.Sum());

Vector probAlcoholGivenNightlifeTrue = Vector.FromArray(0.9, 0.1);
Vector probAlcoholGivenNightlifeFalse = Vector.FromArray(0.2, 0.8);

Vector[] cptSmoking = new Vector[] { probSmokingGivenNightlifeTrue, probSmokingGivenNightlifeFalse };
Vector[] cptAlcohol = new Vector[] { probAlcoholGivenNightlifeTrue, probAlcoholGivenNightlifeFalse };

Vector[][] cptHealthType = new Vector[][]
{
// Smoking 1..5
new Vector[] { Vector.FromArray(0.2, 0.5, 0.3) /* A yes */,  Vector.FromArray(0.85, 0.1, 0.05) /* A no */},
// Smoking 5..10
new Vector[] { Vector.FromArray(0.15, 0.55, 0.3) /* A yes */,  Vector.FromArray(0.8, 0.12, 0.08) /* A no */},
// Smoking 10..20
new Vector[] { Vector.FromArray(0.12, 0.57, 0.31) /* A yes */,  Vector.FromArray(0.7, 0.14, 0.16) /* A no */},
// Smoking 21...
new Vector[] { Vector.FromArray(0.09, 0.6, 0.31) /* A yes */,  Vector.FromArray(0.6, 0.18, 0.22) /* A no */},
// Smoking Nothing
new Vector[] { Vector.FromArray(0.09, 0.6, 0.31) /* A yes */,  Vector.FromArray(0.6, 0.18, 0.22) /* A no */},
};

double probAlcoholGivenHealthTypeNormal = model.ProbAlcohol(
null, null, 0,
probNightlife, cptSmoking, cptAlcohol, cptHealthType);
double probAlcoholGivenHealthTypeRisky = model.ProbAlcohol(
null, null, 1,
probNightlife, cptSmoking, cptAlcohol, cptHealthType);
double probAlcoholGivenHealthTypeCritical = model.ProbAlcohol(
null, null, 2,
probNightlife, cptSmoking, cptAlcohol, cptHealthType);

Console.WriteLine("P(Alcohol | Health type is normal) = {0:0.0000}", probAlcoholGivenHealthTypeNormal);
Console.WriteLine("P(Alcohol | Health type is risky) = {0:0.0000}", probAlcoholGivenHealthTypeRisky);
Console.WriteLine("P(Alcohol | Health type is critical) = {0:0.0000}", probAlcoholGivenHealthTypeCritical);

// -------------------------------------------------------------
// Usage 2: Learn posterior distributions for the parameters
// -------------------------------------------------------------
// To validate the learning, first sample from a known model
int[][] sample = NightlifeModel.Sample(1000, probNightlife, cptSmoking, cptAlcohol, cptHealthType);

Console.WriteLine("\n*********************************************");
Console.WriteLine("Learning parameters from data (uniform prior)");
Console.WriteLine("*********************************************");

// Now see if we can recover the parameters from the data - assume uniform priors
model.LearnParameters(sample[0], sample[1], sample[2], sample[3]);

// The posteriors are distributions over the probabilities and CPTs. Print out the means of these
// distributions, and compare with the ground truth
Console.WriteLine("Prob. Health Type | Smoking: 1..5,   Alcohol: Yes = {0}", model.CPTHealthTypePosterior[0][0].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 1..5,   Alcohol: No  = {0}", model.CPTHealthTypePosterior[0][1].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 5..10,  Alcohol: Yes = {0}", model.CPTHealthTypePosterior[1][0].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 5..10,  Alcohol: No  = {0}", model.CPTHealthTypePosterior[1][1].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 10..20, Alcohol: Yes = {0}", model.CPTHealthTypePosterior[2][0].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 10..20, Alcohol: No  = {0}", model.CPTHealthTypePosterior[2][1].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 21,     Alcohol: Yes = {0}", model.CPTHealthTypePosterior[3][0].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 21,     Alcohol: No  = {0}", model.CPTHealthTypePosterior[3][1].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: None,   Alcohol: Yes = {0}", model.CPTHealthTypePosterior[4][0].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: None,   Alcohol: No  = {0}", model.CPTHealthTypePosterior[4][1].GetMean());

// -------------------------------------------------------------
// Usage 3: Querying the model taking into account uncertainty
//          of the parameters
// -------------------------------------------------------------

// Use posteriors we have just learnt
Console.WriteLine("\n**********************************************");
Console.WriteLine("Querying the model with uncertain ground truth");
Console.WriteLine("**********************************************");
probAlcoholGivenHealthTypeNormal = model.ProbAlcohol(
null, null, 0,
model.ProbNightlifePosterior, model.CPTSmokingPosterior, model.CPTAlcoholPosterior, model.CPTHealthTypePosterior);
probAlcoholGivenHealthTypeRisky = model.ProbAlcohol(
null, null, 1,
model.ProbNightlifePosterior, model.CPTSmokingPosterior, model.CPTAlcoholPosterior, model.CPTHealthTypePosterior);
probAlcoholGivenHealthTypeCritical = model.ProbAlcohol(
null, null, 2,
model.ProbNightlifePosterior, model.CPTSmokingPosterior, model.CPTAlcoholPosterior, model.CPTHealthTypePosterior);

Console.WriteLine("P(Alcohol | Health type is normal) = {0:0.0000}", probAlcoholGivenHealthTypeNormal);
Console.WriteLine("P(Alcohol | Health type is risky) = {0:0.0000}", probAlcoholGivenHealthTypeRisky);
Console.WriteLine("P(Alcohol | Health type is critical) = {0:0.0000}", probAlcoholGivenHealthTypeCritical);
}
}
}

```

Monday, January 23, 2012 4:50 PM

### All replies

• Hi Antrikos

What sort of help are you looking for? The WetGrassSprinklerRain example is deliberately set up to show the general case of discrete variables (ie not restriceted to true/false) and the general AddChildFromOneParent and AddChildFromTwoParents are general methods for adding children. In addition there are several posts about extending this to more parents.

As regards your specific CPTs above, whether you observe these CPTs or whether you want to learn them, you can use the same pattern as in the example. But I am a bit confused by your tables. For example, smoking is conditioned on nightlife; in the cpt, the probabilities for the different smoking events (1...5, etc) should add to 1. You can then set up the observed distributions for cptSmoking similarly to cptSprinkler except the vectors are of length 5 represent the 5 smoking bins.

Let me know if you need further clarification.

John

Monday, January 16, 2012 2:14 PM
• Hello,

First of all I would like to thank you for your time and help.

Firstly our problem has four nodes:

1<sup>st</sup> node: nightlife with variables <<TRUE, FALSE>>

2<sup>nd</sup> node: smoking with variables <<1,5,10,20,NO>>

3<sup>rd</sup> node: alcohol with variables <<YES, NO>>

4<sup>th</sup> node: HealthType with variables <<NORMAL, RISKY, CRITICAL>> where we have the final probabilities all together as you can see from the table above.

The difference from the sprinkler example is that our nodes are not of a <<bool>> type so I do not know how to type the commands correctly. This is my major problem.

For example smoking is not a <<bool>> type node and it takes five variables.

My question is for example  “What is the HealthType if the nightlife is (YES), smoking is (5) and alcohol is (YES) ? ”. I need the result for the HealthType if it is <<NORMAL, RISKY, CRITICAL>> and its corresponding probability.

Thank you in advance, let me know if you need further clarification.

Antrikos

Saturday, January 21, 2012 10:33 PM
• Hi Antrikos

What sort of help are you looking for? The WetGrassSprinklerRain example is deliberately set up to show the general case of discrete variables (ie not restriceted to true/false) and the general AddChildFromOneParent and AddChildFromTwoParents are general methods for adding children. In addition there are several posts about extending this to more parents.

As regards your specific CPTs above, whether you observe these CPTs or whether you want to learn them, you can use the same pattern as in the example. But I am a bit confused by your tables. For example, smoking is conditioned on nightlife; in the cpt, the probabilities for the different smoking events (1...5, etc) should add to 1. You can then set up the observed distributions for cptSmoking similarly to cptSprinkler except the vectors are of length 5 represent the 5 smoking bins.

Let me know if you need further clarification.

John

Hello,

First of all I would like to thank you for your time and help.

Firstly our problem has four nodes:

1<sup>st</sup> node: nightlife with variables <<TRUE, FALSE>>

2<sup>nd</sup> node: smoking with variables <<1,5,10,20,NO>>

3<sup>rd</sup> node: alcohol with variables <<YES, NO>>

4<sup>th</sup> node: HealthType with variables <<NORMAL, RISKY, CRITICAL>> where we have the final probabilities all together as you can see from the table above.

The difference from the sprinkler example is that our nodes are not of a <<bool>> type so I do not know how to type the commands correctly. This is my major problem.

For example smoking is not a <<bool>> type node and it takes five variables.

My question is for example  “What is the HealthType if the nightlife is (YES), smoking is (5) and alcohol is (YES) ? ”. I need the result for the HealthType if it is <<NORMAL, RISKY, CRITICAL>> and its corresponding probability.

Thank you in advance, let me know if you need further clarification.

Antrikos

Monday, January 23, 2012 11:38 AM
• Make sure you are looking at the example provided under the Start menu:  http://research.microsoft.com/en-us/um/cambridge/projects/infernet/docs/Discrete%20Bayesian%20network.aspx
Monday, January 23, 2012 4:46 PM
• Hi Antrikos

You can pretty much substitute your specification directly into the code. The sprinkler example uses ints rather than bools for that very purpose. Here is the sprinkler code with the variables renamed to your nodes, and with your given CPTs. Just copy and paste the code below - you can run it in any of the three modes, namely:

• Query the model when we know the parameters exactly
• Learn the posteriors of the CPTs
• Query the model taking into account parameter uncertainty

The only other thing I needed to change was your Smoking table - as I needed to normalise the columns to add to 1.

```using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using MicrosoftResearch.Infer.Models;
using MicrosoftResearch.Infer;
using MicrosoftResearch.Infer.Distributions;
using MicrosoftResearch.Infer.Maths;

namespace Nightlife
{
public class NightlifeModel
{
public static int NumNightlife = 2;
public static int NumSmoking = 5;
public static int NumAlcohol = 2;
public static int NumHealthType = 3;

// Primary random variables
public VariableArray<int> Nightlife;
public VariableArray<int> Smoking;
public VariableArray<int> Alcohol;
public VariableArray<int> HealthType;
public Variable<int> NumberOfExamples;

// Random variables representing the parameters of the distributions
// of the primary random variables. For child variables, these are
// in the form of conditional probability tables (CPTs)
public Variable<Vector> ProbNightlife;
public VariableArray<Vector> CPTSmoking;
public VariableArray<Vector> CPTAlcohol;
public VariableArray<VariableArray<Vector>, Vector[][]> CPTHealthType;

// Prior distributions for the probability and CPT variables.
// The prior distributions are formulated as Infer.NET variables
// so that they can be set at runtime without recompiling the model
public Variable<Dirichlet> ProbNightlifePrior;
public VariableArray<Dirichlet> CPTSmokingPrior;
public VariableArray<Dirichlet> CPTAlcoholPrior;
public VariableArray<VariableArray<Dirichlet>, Dirichlet[][]> CPTHealthTypePrior;

// Posterior distributions for the probability and CPT variables.
public Dirichlet ProbNightlifePosterior;
public Dirichlet[] CPTSmokingPosterior;
public Dirichlet[] CPTAlcoholPosterior;
public Dirichlet[][] CPTHealthTypePosterior;

// Inference engine
public InferenceEngine Engine = new InferenceEngine();

/// <summary>
/// Constructs a new model
/// </summary>
public NightlifeModel()
{
// Set up the ranges
NumberOfExamples = Variable.New<int>().Named("NofE");
Range N = new Range(NumberOfExamples).Named("N");

Range L = new Range(NumNightlife).Named("L");
Range S = new Range(NumSmoking).Named("S");
Range A = new Range(NumAlcohol).Named("A");
Range H = new Range(NumHealthType).Named("H");

// Define the priors and the parameters
ProbNightlifePrior = Variable.New<Dirichlet>().Named("ProbNightlifePrior");
ProbNightlife = Variable<Vector>.Random(ProbNightlifePrior).Named("ProbNightlife");
ProbNightlife.SetValueRange(L);
// Smoking probability table conditioned on night life
CPTSmokingPrior = Variable.Array<Dirichlet>(L).Named("CPTSmokingPrior");
CPTSmoking = Variable.Array<Vector>(L).Named("CPTSmoking");
CPTSmoking[L] = Variable<Vector>.Random(CPTSmokingPrior[L]);
CPTSmoking.SetValueRange(S);
// Alcohol probability table conditioned on night life
CPTAlcoholPrior = Variable.Array<Dirichlet>(L).Named("CPTAlcoholPrior");
CPTAlcohol = Variable.Array<Vector>(L).Named("CPTAlcohol"); ;
CPTAlcohol[L] = Variable<Vector>.Random(CPTAlcoholPrior[L]);
CPTAlcohol.SetValueRange(A);
// Health type probability table conditioned on Smoking and Alcohol
CPTHealthTypePrior = Variable.Array(Variable.Array<Dirichlet>(A), S).Named("CPTHealthTypePrior");
CPTHealthType = Variable.Array(Variable.Array<Vector>(A), S).Named("CPTHealthType");
CPTHealthType[S][A] = Variable<Vector>.Random(CPTHealthTypePrior[S][A]);
CPTHealthType.SetValueRange(H);

// Define the primary variables
Nightlife = Variable.Array<int>(N).Named("Nightlife");
Nightlife[N] = Variable.Discrete(ProbNightlife).ForEach(N);
}

/// <summary>
/// Learns the parameters of the cloud/Smoking/Alcohol example
/// </summary>
/// <param name="nightlife">Cloudiness data</param>
/// <param name="smoking">Smoking data</param>
/// <param name="alcohol">Alcohol data</param>
/// <param name="healthType">Health type data</param>
/// <param name="probNightlifePrior">Prior for cloudiness probability vector</param>
/// <param name="cptSmokingPrior">Prior for Smoking conditional probability table</param>
/// <param name="cptAlcoholPrior">Prior for Alcohol conditional probability table</param>
/// <param name="cptHealthTypePrior">Prior for health type conditional probability table</param>
public void LearnParameters(
int[] nightlife,
int[] smoking,
int[] alcohol,
int[] healthType,
Dirichlet probNightlifePrior,
Dirichlet[] cptSmokingPrior,
Dirichlet[] cptAlcoholPrior,
Dirichlet[][] cptHealthTypePrior
)
{
NumberOfExamples.ObservedValue = nightlife.Length;
Nightlife.ObservedValue = nightlife;
Smoking.ObservedValue = smoking;
Alcohol.ObservedValue = alcohol;
HealthType.ObservedValue = healthType;
ProbNightlifePrior.ObservedValue = probNightlifePrior;
CPTSmokingPrior.ObservedValue = cptSmokingPrior;
CPTAlcoholPrior.ObservedValue = cptAlcoholPrior;
CPTHealthTypePrior.ObservedValue = cptHealthTypePrior;
// Inference
ProbNightlifePosterior = Engine.Infer<Dirichlet>(ProbNightlife);
CPTSmokingPosterior = Engine.Infer<Dirichlet[]>(CPTSmoking);
CPTAlcoholPosterior = Engine.Infer<Dirichlet[]>(CPTAlcohol);
CPTHealthTypePosterior = Engine.Infer<Dirichlet[][]>(CPTHealthType);
}

/// <summary>
/// Learns the parameters of the cloud/Smoking/Alcohol example assuming uniform priors
/// </summary>
/// <param name="nightlife">Cloudiness data</param>
/// <param name="Smoking">Smoking data</param>
/// <param name="alcohol">Alcohol data</param>
/// <param name="healthType">Health type data</param>
public void LearnParameters(
int[] nightlife,
int[] smoking,
int[] alcohol,
int[] healthType
)
{
// Set all priors to uniform
Dirichlet probNightlifePrior = Dirichlet.Uniform(NumNightlife);
Dirichlet[] probSmokingPrior = Enumerable.Repeat(Dirichlet.Uniform(NumSmoking), NumNightlife).ToArray();
Dirichlet[] probAlcoholPrior = Enumerable.Repeat(Dirichlet.Uniform(NumAlcohol), NumNightlife).ToArray();
Dirichlet[][] probHealthTypePrior = Enumerable.Repeat(Enumerable.Repeat(Dirichlet.Uniform(NumHealthType), NumAlcohol).ToArray(), NumSmoking).ToArray();

LearnParameters(nightlife, smoking, alcohol, healthType, probNightlifePrior, probSmokingPrior, probAlcoholPrior, probHealthTypePrior);
}

/// <summary>
/// Returns the probability of Alcohol given optional readings on
/// cloudiness, smoking, and health type, and given prior distributions
/// over the parameters. Priors may be manually set, or may be the posteriors
/// from learning the parameters.
/// </summary>
/// <param name="nightlife">Optional observation of nightlife category</param>
/// <param name="smoking">Optional observation of smoking category</param>
/// <param name="healthType">Optional observation of health type</param>
/// <param name="probNightlifePrior">Prior distribution over night life probability vector</param>
/// <param name="cptSmokingPrior">Prior distribution over smoking conditional probability table</param>
/// <param name="cptAlcoholPrior">Prior distribution over alcohol conditional probability table</param>
/// <param name="cptHealthTypePrior">Prior distribution over health type conditional probability table</param>
/// <returns>Probability of alcohol category</returns>
public double ProbAlcohol(
int? nightlife,
int? smoking,
int? healthType,
Dirichlet probNightlifePrior,
Dirichlet[] cptSmokingPrior,
Dirichlet[] cptAlcoholPrior,
Dirichlet[][] cptHealthTypePrior)
{
NumberOfExamples.ObservedValue = 1;
if (nightlife.HasValue)
Nightlife.ObservedValue = new int[] { nightlife.Value };
else
Nightlife.ClearObservedValue();
if (smoking.HasValue)
Smoking.ObservedValue = new int[] { smoking.Value };
else
Smoking.ClearObservedValue();
if (healthType.HasValue)
this.HealthType.ObservedValue = new int[] { healthType.Value };
else
this.HealthType.ClearObservedValue();
this.Alcohol.ClearObservedValue();

ProbNightlifePrior.ObservedValue = probNightlifePrior;
CPTSmokingPrior.ObservedValue = cptSmokingPrior;
CPTAlcoholPrior.ObservedValue = cptAlcoholPrior;
CPTHealthTypePrior.ObservedValue = cptHealthTypePrior;

// Inference
var alcoholPosterior = Engine.Infer<Discrete[]>(Alcohol);
// In this example, index 0 is true and index 1 is false
return alcoholPosterior[0].GetProbs()[0];
}

/// <summary>
/// Returns the probability of Alcohol given optional readings on
/// cloudiness, Smoking, and Health type, and given known parameters.
/// </summary>
/// <param name="nightlife">Optional observation of nightlife</param>
/// <param name="smoking">Optional observation of smoking</param>
/// <param name="healthType">Optional observation or health type</param>
/// <param name="probNightlifePrior">Cloudiness probability vector</param>
/// <param name="cptSmokingPrior">Smoking conditional probability table</param>
/// <param name="cptAlcoholPrior">Alcohol conditional probability table</param>
/// <param name="cptHealthTypePrior">Health type conditional probability table</param>
/// <returns>Probability that it has Alcoholed</returns>
public double ProbAlcohol(
int? nightlife,
int? smoking,
int? healthType,
Vector probNightlife,
Vector[] cptSmoking,
Vector[] cptAlcohol,
Vector[][] cptHealthType)
{
var probNightlifePrior = Dirichlet.PointMass(probNightlife);
var cptSmokingPrior = cptSmoking.Select(v => Dirichlet.PointMass(v)).ToArray();
var cptAlcoholPrior = cptAlcohol.Select(v => Dirichlet.PointMass(v)).ToArray();
var cptHealthTypePrior = cptHealthType.Select(va => va.Select(v => Dirichlet.PointMass(v)).ToArray()).ToArray();
return ProbAlcohol(nightlife, smoking, healthType, probNightlifePrior, cptSmokingPrior, cptAlcoholPrior, cptHealthTypePrior);
}

/// <summary>
/// Sample the model
/// </summary>
/// <param name="numData">Number of data in sample</param>
/// <param name="probNightlifePrior">Cloudiness probability vector</param>
/// <param name="cptSmokingPrior">Smoking conditional probability table</param>
/// <param name="cptAlcoholPrior">Alcohol conditional probability table</param>
/// <param name="cptHealthTypePrior">Health type grass conditional probability table</param>
/// <returns></returns>
public static int[][] Sample(
int numData,
Vector probNightlife,
Vector[] cptSmoking,
Vector[] cptAlcohol,
Vector[][] cptHealthType)
{
int[][] sample = new int[4][];
for (int i = 0; i < 4; i++)
sample[i] = new int[numData];
for (int j = 0; j < numData; j++)
{
int nightlife = Discrete.Sample(probNightlife);
int smoking = Discrete.Sample(cptSmoking[nightlife]);
int alcohol = Discrete.Sample(cptAlcohol[nightlife]);
int healthType = Discrete.Sample(cptHealthType[smoking][alcohol]);
sample[0][j] = nightlife;
sample[1][j] = smoking;
sample[2][j] = alcohol;
sample[3][j] = healthType;
}
return sample;
}

/// <summary>
/// Helper method to add a child from one parent
/// </summary>
/// <param name="parent">Parent (a variable array over a range of examples)</param>
/// <param name="cpt">Conditional probability table</param>
/// <returns></returns>
VariableArray<int> parent,
VariableArray<Vector> cpt)
{
var n = parent.Range;
var child = Variable.Array<int>(n);
using (Variable.ForEach(n))
using (Variable.Switch(parent[n]))
child[n] = Variable.Discrete(cpt[parent[n]]);
return child;
}

/// <summary>
/// Helper method to add a child from two parents
/// </summary>
/// <param name="parent1">First parent (a variable array over a range of examples)</param>
/// <param name="parent2">Second parent (a variable array over the same range)</param>
/// <param name="cpt">Conditional probability table</param>
/// <returns></returns>
VariableArray<int> parent1,
VariableArray<int> parent2,
VariableArray<VariableArray<Vector>, Vector[][]> cpt)
{
var n = parent1.Range;
var child = Variable.Array<int>(n);
using (Variable.ForEach(n))
using (Variable.Switch(parent1[n]))
using (Variable.Switch(parent2[n]))
child[n] = Variable.Discrete(cpt[parent1[n]][parent2[n]]);
return child;
}
}

class Program
{
static void Main(string[] args)
{
// Set random seed for repeatable example
Rand.Restart(12347);

// Create a new model
NightlifeModel model = new NightlifeModel();
model.Engine.ShowProgress = false;

// In this example, each variable just takes two states - true (index 0)
// and false (index 1). The example is written so that extensions to
// problems of more than two states is straightforward.

// -------------------------------------------------------------
// Usage 1: Query the model when we know the parameters exactly
// -------------------------------------------------------------
Console.WriteLine("\n*********************************************");
Console.WriteLine("Querying the model with a known ground truth");
Console.WriteLine("*********************************************");

Vector probNightlife = Vector.FromArray(0.2, 0.35);
probNightlife.SetToProduct(probNightlife, 1.0 / probNightlife.Sum());

Vector probSmokingGivenNightlifeTrue = Vector.FromArray(0.8, 0.85, 0.9, 0.95, 0.15);
probSmokingGivenNightlifeTrue.SetToProduct(probSmokingGivenNightlifeTrue, 1.0 / probSmokingGivenNightlifeTrue.Sum());
Vector probSmokingGivenNightlifeFalse = Vector.FromArray(0.2, 0.15, 0.1, 0.05, 0.9);
probSmokingGivenNightlifeFalse.SetToProduct(probSmokingGivenNightlifeFalse, 1.0 / probSmokingGivenNightlifeFalse.Sum());

Vector probAlcoholGivenNightlifeTrue = Vector.FromArray(0.9, 0.1);
Vector probAlcoholGivenNightlifeFalse = Vector.FromArray(0.2, 0.8);

Vector[] cptSmoking = new Vector[] { probSmokingGivenNightlifeTrue, probSmokingGivenNightlifeFalse };
Vector[] cptAlcohol = new Vector[] { probAlcoholGivenNightlifeTrue, probAlcoholGivenNightlifeFalse };

Vector[][] cptHealthType = new Vector[][]
{
// Smoking 1..5
new Vector[] { Vector.FromArray(0.2, 0.5, 0.3) /* A yes */,  Vector.FromArray(0.85, 0.1, 0.05) /* A no */},
// Smoking 5..10
new Vector[] { Vector.FromArray(0.15, 0.55, 0.3) /* A yes */,  Vector.FromArray(0.8, 0.12, 0.08) /* A no */},
// Smoking 10..20
new Vector[] { Vector.FromArray(0.12, 0.57, 0.31) /* A yes */,  Vector.FromArray(0.7, 0.14, 0.16) /* A no */},
// Smoking 21...
new Vector[] { Vector.FromArray(0.09, 0.6, 0.31) /* A yes */,  Vector.FromArray(0.6, 0.18, 0.22) /* A no */},
// Smoking Nothing
new Vector[] { Vector.FromArray(0.09, 0.6, 0.31) /* A yes */,  Vector.FromArray(0.6, 0.18, 0.22) /* A no */},
};

double probAlcoholGivenHealthTypeNormal = model.ProbAlcohol(
null, null, 0,
probNightlife, cptSmoking, cptAlcohol, cptHealthType);
double probAlcoholGivenHealthTypeRisky = model.ProbAlcohol(
null, null, 1,
probNightlife, cptSmoking, cptAlcohol, cptHealthType);
double probAlcoholGivenHealthTypeCritical = model.ProbAlcohol(
null, null, 2,
probNightlife, cptSmoking, cptAlcohol, cptHealthType);

Console.WriteLine("P(Alcohol | Health type is normal) = {0:0.0000}", probAlcoholGivenHealthTypeNormal);
Console.WriteLine("P(Alcohol | Health type is risky) = {0:0.0000}", probAlcoholGivenHealthTypeRisky);
Console.WriteLine("P(Alcohol | Health type is critical) = {0:0.0000}", probAlcoholGivenHealthTypeCritical);

// -------------------------------------------------------------
// Usage 2: Learn posterior distributions for the parameters
// -------------------------------------------------------------
// To validate the learning, first sample from a known model
int[][] sample = NightlifeModel.Sample(1000, probNightlife, cptSmoking, cptAlcohol, cptHealthType);

Console.WriteLine("\n*********************************************");
Console.WriteLine("Learning parameters from data (uniform prior)");
Console.WriteLine("*********************************************");

// Now see if we can recover the parameters from the data - assume uniform priors
model.LearnParameters(sample[0], sample[1], sample[2], sample[3]);

// The posteriors are distributions over the probabilities and CPTs. Print out the means of these
// distributions, and compare with the ground truth
Console.WriteLine("Prob. Health Type | Smoking: 1..5,   Alcohol: Yes = {0}", model.CPTHealthTypePosterior[0][0].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 1..5,   Alcohol: No  = {0}", model.CPTHealthTypePosterior[0][1].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 5..10,  Alcohol: Yes = {0}", model.CPTHealthTypePosterior[1][0].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 5..10,  Alcohol: No  = {0}", model.CPTHealthTypePosterior[1][1].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 10..20, Alcohol: Yes = {0}", model.CPTHealthTypePosterior[2][0].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 10..20, Alcohol: No  = {0}", model.CPTHealthTypePosterior[2][1].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 21,     Alcohol: Yes = {0}", model.CPTHealthTypePosterior[3][0].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: 21,     Alcohol: No  = {0}", model.CPTHealthTypePosterior[3][1].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: None,   Alcohol: Yes = {0}", model.CPTHealthTypePosterior[4][0].GetMean());
Console.WriteLine("Prob. Health Type | Smoking: None,   Alcohol: No  = {0}", model.CPTHealthTypePosterior[4][1].GetMean());

// -------------------------------------------------------------
// Usage 3: Querying the model taking into account uncertainty
//          of the parameters
// -------------------------------------------------------------

// Use posteriors we have just learnt
Console.WriteLine("\n**********************************************");
Console.WriteLine("Querying the model with uncertain ground truth");
Console.WriteLine("**********************************************");
probAlcoholGivenHealthTypeNormal = model.ProbAlcohol(
null, null, 0,
model.ProbNightlifePosterior, model.CPTSmokingPosterior, model.CPTAlcoholPosterior, model.CPTHealthTypePosterior);
probAlcoholGivenHealthTypeRisky = model.ProbAlcohol(
null, null, 1,
model.ProbNightlifePosterior, model.CPTSmokingPosterior, model.CPTAlcoholPosterior, model.CPTHealthTypePosterior);
probAlcoholGivenHealthTypeCritical = model.ProbAlcohol(
null, null, 2,
model.ProbNightlifePosterior, model.CPTSmokingPosterior, model.CPTAlcoholPosterior, model.CPTHealthTypePosterior);

Console.WriteLine("P(Alcohol | Health type is normal) = {0:0.0000}", probAlcoholGivenHealthTypeNormal);
Console.WriteLine("P(Alcohol | Health type is risky) = {0:0.0000}", probAlcoholGivenHealthTypeRisky);
Console.WriteLine("P(Alcohol | Health type is critical) = {0:0.0000}", probAlcoholGivenHealthTypeCritical);
}
}
}

```

Monday, January 23, 2012 4:50 PM
• Hi John,

Slightly off-topic.  What would be equivalent in F# of:

`CPTHealthType[S][A] = Variable<Vector>.Random(CPTHealthTypePrior[S][A]);`

`Many thanks.`

Saturday, January 28, 2012 10:51 PM
• ```let CPTHealthTypePrior = Variable.Array<Dirichlet>(Variable.Array(A), S).Named("CPTHealthTypePrior")
let CPTHealthType = Variable.Array<Vector>(Variable.Array(A), S).Named("CPTHealthType");
CPTHealthType.[S].[A] <- Variable.Random<Vector, Dirichlet>(CPTHealthTypePrior.[S].[A]);
```

You can look at the FSharpWrapper.fs source file in the Infer.NET installation for other examples of F# sytax for InferNET (go to Start Menu/Infer.NET 2.4/Sources).

John

• Proposed as answer by Wednesday, February 1, 2012 5:23 AM
Monday, January 30, 2012 5:52 PM