How define a model with a variable dependent more than two. (Migrated from community.research.microsoft.com)

# How define a model with a variable dependent more than two. (Migrated from community.research.microsoft.com)

• Friday, June 03, 2011 6:51 PM
Owner

Lato posted on 05-19-2011 5:41 AM

Hi,

I have a problem to define a model with the above features, in particular how define CPT and its variables.

(In my case I have to define a dependent variable from five other.)

Thanks and regards

Alessandro

### All Replies

• Friday, June 03, 2011 6:51 PM
Owner

John Guiver replied on 05-23-2011 10:55 AM

Hi Alessandro

You can use arrays of arrays to nest things as deep as you want. As the syntax gets a bit messy, you can create an alias within you namespace as follows (example shown for 4 parents - i.e. the type of the 4-deep CPT random variable

using VarVectArr4 = VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>;

The code for constructing a 4-parent child is:

// Model code for adding a child from four parents

VariableArray<int> parent1,

VariableArray<int> parent2,

VariableArray<int> parent3,

VariableArray<int> parent4,

VarVectArr4 cpt)

{

var d = parent1.Range;

// data range

var child = Variable.Array<int>(d);

using (Variable.ForEach(d))

using (Variable.Switch(parent1[d]))

using (Variable.Switch(parent2[d]))

using (Variable.Switch(parent3[d]))

using (Variable.Switch(parent4[d]))

child[d] = Variable.Discrete(

cpt[parent1[d]][parent2[d]][parent3[d]][parent4[d]]);

return child;

}

See how far you can get with that, and let me know if/when you run into problems.

John

• Friday, June 03, 2011 6:51 PM
Owner

Lato replied on 05-24-2011 9:51 AM

Hi John,

I'm block on a problem, maybe I did a bit of confusion with alias

I started using the Dirichlet vectors then move on to the structure that you have indicated.

using System;

using System.Collections.Generic;

using System.Linq;

using System.Text;

using MicrosoftResearch.Infer.Models;

using MicrosoftResearch.Infer.Maths;

using MicrosoftResearch.Infer.Distributions;

using MicrosoftResearch.Infer;

using MicrosoftResearch.Infer.Collections;

namespace Project1

{

// Aliases for distributions over 1-D, 2-D, 3-D, 4-D, 5-D, 6-D probability tables

using VarVectArr =  VariableArray<Vector>;

//using DirichletArray = VariableArray<VariableArray<Vector>, Vector>;

using VarVectArr2 =     VariableArray<VariableArray<Vector>, Vector[][]>;

// using DirichletArray2 = VariableArray<VariableArray<VariableArray<Vector>, Vector>, Vector[]>;

//using DirichletArray3 = DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[][]>;

using VarVectArr3 = VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>;

using VarVectArr4 = VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>;

//using DirichletArray4 = DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>, Vector[][]>, Vector[][][]>;

using VarVectArr5 = VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>, Vector[][][][][]>;

//using DirichletArray5 = DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>, Vector[][]>, Vector[][][]>, Vector[][][][]>;

using VarVectArr6 = VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>, Vector[][][][][]>, Vector[][][][][][]>;

//using DirichletArray6 = DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>, Vector[][]>, Vector[][][]>, Vector[][][][]>, Vector[][][][][]>;

class ModelW_POS_P

{

public Variable<int> NumCases = Variable.New<int>();

public VariableArray<int> o1;

public VariableArray<int> o2;

public VariableArray<int> o3;

public VariableArray<int> o4;

public VariableArray<int> o5;

public VariableArray<int> w1;

public VariableArray<int> w2;

public VariableArray<int> w3;

public VariableArray<int> p;

// result model inference

public Discrete[] res_p;

// PT and CPT

public Variable<Vector> pt_p;

public Variable<Vector> pt_o1;

public Variable<Vector> pt_o2;

public Variable<Vector> pt_o3;

public Variable<Vector> pt_o4;

public Variable<Vector> pt_w1;

public Variable<Vector> pt_w2;

public VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]> cpt_w3;

public VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>, Vector[][][][][]>, Vector[][][][][][]> cpt_o5;

// Prior distributions for the probability tables - make these variable so we can

// change at run-time without model recompilation

public Variable<Dirichlet> pt_p_prior = null;

public Variable<Dirichlet> pt_o1_prior = null;

public Variable<Dirichlet> pt_o2_prior = null;

public Variable<Dirichlet> pt_o3_prior = null;

public Variable<Dirichlet> pt_o4_prior = null;

public Variable<Dirichlet> pt_w1_prior = null;

public Variable<Dirichlet> pt_w2_prior = null;

public Variable<VarVectArr3> cpt_w3_prior;

public Variable<VarVectArr6> cpt_o5_prior;

// public Variable<DirichletArrayArrayArrayArrayArray> cpt_w5_prior = null;

public InferenceEngine InfEngine = new InferenceEngine();

public void CreateModel(int range_o, int range_w, int range_p)

{

NumCases = Variable.New<int>();

Range n = new Range(NumCases);

Range r_o1 = new Range(range_o);

Range r_o2 = new Range(range_o);

Range r_o3 = new Range(range_o);

Range r_o4 = new Range(range_o);

Range r_o5 = new Range(range_o);

Range r_w1 = new Range(range_w);

Range r_w2 = new Range(range_w);

Range r_w3 = new Range(range_w);

Range r_w4 = new Range(range_w);

Range r_w5 = new Range(range_w);

Range r_p = new Range(range_p);

// Define the priors as variables - these will be set (observed) at run time

pt_p_prior = Variable.New<Dirichlet>();

pt_o1_prior = Variable.New<Dirichlet>();

pt_o2_prior = Variable.New<Dirichlet>();

pt_o3_prior = Variable.New<Dirichlet>();

pt_o4_prior = Variable.New<Dirichlet>();

pt_w1_prior = Variable.New<Dirichlet>();

pt_w2_prior = Variable.New<Dirichlet>();

cpt_o5_prior = Variable.New<VarVectArr6>();

cpt_w3_prior = Variable.New<VarVectArr3>();

// Probability table and conditional probability table variables (parameters)

pt_w1 = Variable<Vector>.Random(pt_w1_prior);

pt_o1 = Variable<Vector>.Random(pt_o1_prior);

pt_w2 = Variable<Vector>.Random(pt_w2_prior);

pt_o2 = Variable<Vector>.Random(pt_o2_prior);

pt_o3 = Variable<Vector>.Random(pt_o3_prior);

pt_o4 = Variable<Vector>.Random(pt_o4_prior);

// ===========Problem ==============================================================================

cpt_o5 = Variable.Array(Variable.Array(Variable.Array(Variable.Array(Variable.Array(Variable.Array<Vector>(r_o1), r_o2), r_o3), r_o4), r_o5), p);

}

// Model code for adding a child from a single parent

public static VariableArray<int> AddChildFromOneParent(VariableArray<int> parent, VariableArray<Vector> cpt)

{

var d = parent.Range;

// data range

var child = Variable.Array<int>(d);

using (Variable.ForEach(d))

using (Variable.Switch(parent[d]))

child[d] = Variable.Discrete(cpt[parent[d]]);

return child;

}

// Model code for adding a child from two parents

VariableArray<int> parent1, VariableArray<int> parent2,

VariableArray<VariableArray<Vector>, Vector[][]> cpt)

{

var d = parent1.Range;

// data range

var child = Variable.Array<int>(d);

using (Variable.ForEach(d))

using (Variable.Switch(parent1[d]))

using (Variable.Switch(parent2[d]))

child[d] = Variable.Discrete(cpt[parent1[d]][parent2[d]]);

return child;

}

// Model code for adding a child from three parents

VariableArray<int> parent1, VariableArray<int> parent2,VariableArray<int> parent3,

VarVectArr3 cpt)

{

var d = parent1.Range;

// data range

var child = Variable.Array<int>(d);

using (Variable.ForEach(d))

using (Variable.Switch(parent1[d]))

using (Variable.Switch(parent2[d]))

using (Variable.Switch(parent3[d]))

child[d] = Variable.Discrete(cpt[parent1[d]][parent2[d]][parent3[d]]);

return child;

}

// Model code for adding a child from four parents

VariableArray<int> parent1,

VariableArray<int> parent2,

VariableArray<int> parent3,

VariableArray<int> parent4,

VarVectArr4 cpt)

{

var d = parent1.Range;

// data range

var child = Variable.Array<int>(d);

using (Variable.ForEach(d))

using (Variable.Switch(parent1[d]))

using (Variable.Switch(parent2[d]))

using (Variable.Switch(parent3[d]))

using (Variable.Switch(parent4[d]))

child[d] = Variable.Discrete(

cpt[parent1[d]][parent2[d]][parent3[d]][parent4[d]]);

return child;

}

// Model code for adding a child from five parents

VariableArray<int> parent1,

VariableArray<int> parent2,

VariableArray<int> parent3,

VariableArray<int> parent4,

VariableArray<int> parent5,

VarVectArr5 cpt)

{

var d = parent1.Range;

// data range

var child = Variable.Array<int>(d);

using (Variable.ForEach(d))

using (Variable.Switch(parent1[d]))

using (Variable.Switch(parent2[d]))

using (Variable.Switch(parent3[d]))

using (Variable.Switch(parent4[d]))

child[d] = Variable.Discrete(

cpt[parent1[d]][parent2[d]][parent3[d]][parent4[d]][parent5[d]]);

return child;

}

// Model code for adding a child from six parents

VariableArray<int> parent1,

VariableArray<int> parent2,

VariableArray<int> parent3,

VariableArray<int> parent4,

VariableArray<int> parent5,

VariableArray<int> parent6,

VarVectArr6 cpt)

{

var d = parent1.Range;

// data range

var child = Variable.Array<int>(d);

using (Variable.ForEach(d))

using (Variable.Switch(parent1[d]))

using (Variable.Switch(parent2[d]))

using (Variable.Switch(parent3[d]))

using (Variable.Switch(parent4[d]))

child[d] = Variable.Discrete(

cpt[parent1[d]][parent2[d]][parent3[d]][parent4[d]][parent5[d]][parent6[d]]);

return child;

}

}

}

this is the model's image. It can help you to understand the code.

thanks

Alessandro

• Friday, June 03, 2011 6:51 PM
Owner

John Guiver replied on 05-27-2011 12:10 PM

Hi Alessandro

When you get above depth 3, you have to use the generic Array factor method on the Variable class - this takes two type parameters. I show an example below of how to build up a variable - or you could try to do it in one statement.

Apologies for not mentioning this in my previous post. You are the first ever (that I know of) to go to this depth!

var a = Variable.Array<Vector>(r_o1);

var b = Variable.Array<VarVectArr, Vector[][]>(a, r_o2);

var c = Variable.Array<VarVectArr2, Vector[][][]>(b, r_o3);

var d = Variable.Array<VarVectArr3, Vector[][][][]>(c, r_o4);

var e = Variable.Array<VarVectArr4, Vector[][][][][]>(d, r_o5);

var f = Variable.Array<VarVectArr5, Vector[][][][][][]>(e, r_p);

John

• Monday, June 06, 2011 8:32 AM

Thanks you John,

i realized why i am the first to go so deep, because it's not easy!

i have another problem in this step, because no - one of my solution is correct.

cpt_o5.SetTo(Variable<Vector[][][][][]>.Random(cpt_o5_prior));

cpt_o5.SetTo(Variable<VarVectArr5>.Random(cpt_o5_prior));

cpt_o5 is so definited:

cpt_o5 = Variable.Array<VarVectArr4Vector[][][][][]>(Variable.Array<VarVectArr3Vector[][][][]>(Variable.Array<VarVectArr2Vector[][][]>(Variable.Array<VarVectArrVector[][]>(Variable.Array<Vector>(r_o1), r_o2), r_o3), r_o4), r_p);

Regards

Alessandro

• Tuesday, June 07, 2011 9:07 AM
Owner

Alessandro

There are two ways to deal with this. You can either manage the priors as .NET arrays, or as Infer.NET distribution arrays (look back to the original discussions for this). The latter is more efficient (and is the one you are trying to use) if you are directly using the training posteriors to set the test-time priors because this is the natural way posteriors are stored; the first is more efficient, but the posteriors need to be converted/cast (via the Infer method type parameter). Anyway, let's first review the different aliases we need:

```  using DirArr = DistributionRefArray<Dirichlet, Vector>;
using DirArr2= DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>;
using DirArr3 = DistributionRefArray<DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>, Vector[][]>;
using DirArr4 = DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>, Vector[][]>, Vector[][][]>;
using DirArr5 = DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>, Vector[][]>, Vector[][][]>, Vector[][][][]>;

using VarVectArr = VariableArray<Vector>;
using VarVectArr2 = VariableArray<VariableArray<Vector>, Vector[][]>;
using VarVectArr3 = VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>;
using VarVectArr4 = VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>;
using VarVectArr5 = VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>, Vector[][][][][]>;

using VarDirArr = VariableArray<Dirichlet>;
using VarDirArr2 = VariableArray<VariableArray<Dirichlet>, Dirichlet[][]>;
using VarDirArr3 = VariableArray<VariableArray<VariableArray<Dirichlet>, Dirichlet[][]>, Dirichlet[][][]>;
using VarDirArr4 = VariableArray<VariableArray<VariableArray<VariableArray<Dirichlet>, Dirichlet[][]>, Dirichlet[][][]>, Dirichlet[][][][]>;
using VarDirArr5 = VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<Dirichlet>, Dirichlet[][]>, Dirichlet[][][]>, Dirichlet[][][][]>, Dirichlet[][][][][]>;

```

Now for the first solution, we use:

```VarDirArr5 cpt_o5_prior = Variable.Array<VarDirArr4, Dirichlet[][][][][]>(Variable.Array<VarDirArr3, Dirichlet[][][][]>(Variable.Array<VarDirArr2, Dirichlet[][][]>(Variable.Array<VarDirArr, Dirichlet[][]>(Variable.Array<Dirichlet>(r_o1), r_o2), r_o3), r_o4), r_p);
VarVectArr5 cpt_o5 = Variable.Array<VarVectArr4, Vector[][][][][]>(Variable.Array<VarVectArr3, Vector[][][][]>(Variable.Array<VarVectArr2, Vector[][][]>(Variable.Array<VarVectArr, Vector[][]>(Variable.Array<Vector>(r_o1), r_o2), r_o3), r_o4), r_p);
cpt_o5[r_p][r_o4][r_o3][r_o2][r_o1] = Variable<Vector>.Random(cpt_o5_prior[r_p][r_o4][r_o3][r_o2][r_o1]);

```

For the second solution, we use:

```Variable<DirArr5> cpt_prior = Variable.New<DirArr5>();
VarVectArr5 cpt_o5 = Variable.Array<VarVectArr4, Vector[][][][][]>(Variable.Array<VarVectArr3, Vector[][][][]>(Variable.Array<VarVectArr2, Vector[][][]>(Variable.Array<VarVectArr, Vector[][]>(Variable.Array<Vector>(r_o1), r_o2), r_o3), r_o4), r_p);
cpt_o5.SetTo(Variable<Vector[][][][][]>.Random(cpt_prior));

```

You will, of course need to set the observed values for the priors appropriately. Hope this helps

John

• Tuesday, June 07, 2011 3:23 PM

Yes John,

In spite of all I think the model is not currently implementable.

When I run the code I get the following exception: "arrays of rank higher than 3 are not supported yet" as NotSupportedException at line:

cpt_o5[r_p][r_o4][r_o3][r_o2][r_o1] = Variable<Vector>.Random(cpt_o5_prior[r_p][r_o4][r_o3][r_o2][r_o1]);

• Tuesday, June 07, 2011 5:39 PM
Owner

Hi Alessandro

The second method should work fine, but it looks like the first method needs explicit for blocks (as I said, you are the first to go to this depth...) :

```using (Variable.ForEach(r_p))
using (Variable.ForEach(r_o4))
using (Variable.ForEach(r_o3))
using (Variable.ForEach(r_o2))
using (Variable.ForEach(r_o1))
cpt_o5[r_p][r_o4][r_o3][r_o2][r_o1] = Variable<Vector>.Random(cpt_o5_prior_[r_p][r_o4][r_o3][r_o2][r_o1]);

```

John