locked
How define a model with a variable dependent more than two. (Migrated from community.research.microsoft.com) RRS feed

  • Question

  • Lato posted on 05-19-2011 5:41 AM

    Hi,

    I have a problem to define a model with the above features, in particular how define CPT and its variables.

    (In my case I have to define a dependent variable from five other.)

    Thanks and regards

    Alessandro

     


    Friday, June 3, 2011 6:51 PM

Answers

  • John Guiver replied on 05-27-2011 12:10 PM

    Hi Alessandro

    When you get above depth 3, you have to use the generic Array factor method on the Variable class - this takes two type parameters. I show an example below of how to build up a variable - or you could try to do it in one statement.

    Apologies for not mentioning this in my previous post. You are the first ever (that I know of) to go to this depth!

    var a = Variable.Array<Vector>(r_o1);

    var b = Variable.Array<VarVectArr, Vector[][]>(a, r_o2);

    var c = Variable.Array<VarVectArr2, Vector[][][]>(b, r_o3);

    var d = Variable.Array<VarVectArr3, Vector[][][][]>(c, r_o4);

    var e = Variable.Array<VarVectArr4, Vector[][][][][]>(d, r_o5);

    var f = Variable.Array<VarVectArr5, Vector[][][][][][]>(e, r_p);

    John

    Friday, June 3, 2011 6:51 PM

All replies

  • John Guiver replied on 05-23-2011 10:55 AM

    Hi Alessandro

    You can use arrays of arrays to nest things as deep as you want. As the syntax gets a bit messy, you can create an alias within you namespace as follows (example shown for 4 parents - i.e. the type of the 4-deep CPT random variable

    using VarVectArr4 = VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>;

    The code for constructing a 4-parent child is:

    // Model code for adding a child from four parents

     public static VariableArray<int> AddChildFromFourParents(

         VariableArray<int> parent1,

         VariableArray<int> parent2,

         VariableArray<int> parent3,

         VariableArray<int> parent4,

         VarVectArr4 cpt)

     {

         var d = parent1.Range;

         // data range

         var child = Variable.Array<int>(d);

         using (Variable.ForEach(d))

         using (Variable.Switch(parent1[d]))

         using (Variable.Switch(parent2[d]))

         using (Variable.Switch(parent3[d]))

         using (Variable.Switch(parent4[d]))

         child[d] = Variable.Discrete(

             cpt[parent1[d]][parent2[d]][parent3[d]][parent4[d]]);

         return child;

     }

    See how far you can get with that, and let me know if/when you run into problems.

    John

    Friday, June 3, 2011 6:51 PM
  • Lato replied on 05-24-2011 9:51 AM

    Hi John,

    I'm block on a problem, maybe I did a bit of confusion with alias

    I started using the Dirichlet vectors then move on to the structure that you have indicated.

    using System;

    using System.Collections.Generic;

    using System.Linq;

    using System.Text;

    using MicrosoftResearch.Infer.Models;

    using MicrosoftResearch.Infer.Maths;

    using MicrosoftResearch.Infer.Distributions;

    using MicrosoftResearch.Infer;

    using MicrosoftResearch.Infer.Collections;

     

    namespace Project1

    {

        // Aliases for distributions over 1-D, 2-D, 3-D, 4-D, 5-D, 6-D probability tables

        using VarVectArr =  VariableArray<Vector>;

        //using DirichletArray = VariableArray<VariableArray<Vector>, Vector>;

        using VarVectArr2 =     VariableArray<VariableArray<Vector>, Vector[][]>;

       // using DirichletArray2 = VariableArray<VariableArray<VariableArray<Vector>, Vector>, Vector[]>;

        //using DirichletArray3 = DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[][]>;

        using VarVectArr3 = VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>;

        using VarVectArr4 = VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>;

        //using DirichletArray4 = DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>, Vector[][]>, Vector[][][]>;

        using VarVectArr5 = VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>, Vector[][][][][]>;

        //using DirichletArray5 = DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>, Vector[][]>, Vector[][][]>, Vector[][][][]>;

        using VarVectArr6 = VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>, Vector[][][][][]>, Vector[][][][][][]>;

        //using DirichletArray6 = DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>, Vector[][]>, Vector[][][]>, Vector[][][][]>, Vector[][][][][]>;

     

        class ModelW_POS_P

        {

            public Variable<int> NumCases = Variable.New<int>();

            public VariableArray<int> o1;

            public VariableArray<int> o2;

            public VariableArray<int> o3;

            public VariableArray<int> o4;

            public VariableArray<int> o5;

            public VariableArray<int> w1;

            public VariableArray<int> w2;

            public VariableArray<int> w3;

            public VariableArray<int> p;

     

            // result model inference

            public Discrete[] res_p;

     

     

            // PT and CPT

            public Variable<Vector> pt_p;

            public Variable<Vector> pt_o1;

            public Variable<Vector> pt_o2;

            public Variable<Vector> pt_o3;

            public Variable<Vector> pt_o4;

     

            public Variable<Vector> pt_w1;

            public Variable<Vector> pt_w2;

     

            public VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]> cpt_w3;

            public VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>, Vector[][][][][]>, Vector[][][][][][]> cpt_o5;

     

            // Prior distributions for the probability tables - make these variable so we can

            // change at run-time without model recompilation

     

            public Variable<Dirichlet> pt_p_prior = null;

            public Variable<Dirichlet> pt_o1_prior = null;

            public Variable<Dirichlet> pt_o2_prior = null;

            public Variable<Dirichlet> pt_o3_prior = null;

            public Variable<Dirichlet> pt_o4_prior = null;

            public Variable<Dirichlet> pt_w1_prior = null;

            public Variable<Dirichlet> pt_w2_prior = null;

     

     

            public Variable<VarVectArr3> cpt_w3_prior;

            public Variable<VarVectArr6> cpt_o5_prior;

           // public Variable<DirichletArrayArrayArrayArrayArray> cpt_w5_prior = null;

     

            public InferenceEngine InfEngine = new InferenceEngine();

     

            public void CreateModel(int range_o, int range_w, int range_p)

            {

     

                NumCases = Variable.New<int>();

                Range n = new Range(NumCases);

                Range r_o1 = new Range(range_o);

                Range r_o2 = new Range(range_o);

                Range r_o3 = new Range(range_o);

                Range r_o4 = new Range(range_o);

                Range r_o5 = new Range(range_o);

     

                Range r_w1 = new Range(range_w);

                Range r_w2 = new Range(range_w);

                Range r_w3 = new Range(range_w);

                Range r_w4 = new Range(range_w);

                Range r_w5 = new Range(range_w);

     

                Range r_p = new Range(range_p);

     

                // Define the priors as variables - these will be set (observed) at run time

                pt_p_prior = Variable.New<Dirichlet>();

                pt_o1_prior = Variable.New<Dirichlet>();

                pt_o2_prior = Variable.New<Dirichlet>();

                pt_o3_prior = Variable.New<Dirichlet>();

                pt_o4_prior = Variable.New<Dirichlet>();

                pt_w1_prior = Variable.New<Dirichlet>();

                pt_w2_prior = Variable.New<Dirichlet>();

                cpt_o5_prior = Variable.New<VarVectArr6>();

                cpt_w3_prior = Variable.New<VarVectArr3>();

     

                // Probability table and conditional probability table variables (parameters)

                pt_w1 = Variable<Vector>.Random(pt_w1_prior);

                pt_o1 = Variable<Vector>.Random(pt_o1_prior);

                pt_w2 = Variable<Vector>.Random(pt_w2_prior);

                pt_o2 = Variable<Vector>.Random(pt_o2_prior);

                pt_o3 = Variable<Vector>.Random(pt_o3_prior);

                pt_o4 = Variable<Vector>.Random(pt_o4_prior);

     

                // ===========Problem ==============================================================================

                cpt_o5 = Variable.Array(Variable.Array(Variable.Array(Variable.Array(Variable.Array(Variable.Array<Vector>(r_o1), r_o2), r_o3), r_o4), r_o5), p);

     

            }

     

            // Model code for adding a child from a single parent

            public static VariableArray<int> AddChildFromOneParent(VariableArray<int> parent, VariableArray<Vector> cpt)

            {

                var d = parent.Range;

                // data range

                var child = Variable.Array<int>(d);

                using (Variable.ForEach(d))

                using (Variable.Switch(parent[d]))

                    child[d] = Variable.Discrete(cpt[parent[d]]);

                return child;

            }

     

            // Model code for adding a child from two parents

            public static VariableArray<int> AddChildFromTwoParents(

                VariableArray<int> parent1, VariableArray<int> parent2,

                VariableArray<VariableArray<Vector>, Vector[][]> cpt)

            {

                var d = parent1.Range;

                // data range

                var child = Variable.Array<int>(d);

                using (Variable.ForEach(d))

                using (Variable.Switch(parent1[d]))

                using (Variable.Switch(parent2[d]))

                    child[d] = Variable.Discrete(cpt[parent1[d]][parent2[d]]);

                return child;

            }

     

            // Model code for adding a child from three parents

            public static VariableArray<int> AddChildFromThreeParents(

                VariableArray<int> parent1, VariableArray<int> parent2,VariableArray<int> parent3,

                VarVectArr3 cpt)

            {

                var d = parent1.Range;

                // data range

                var child = Variable.Array<int>(d);

                using (Variable.ForEach(d))

                using (Variable.Switch(parent1[d]))

                using (Variable.Switch(parent2[d]))

                using (Variable.Switch(parent3[d]))

                    child[d] = Variable.Discrete(cpt[parent1[d]][parent2[d]][parent3[d]]);

                return child;

            }

     

            // Model code for adding a child from four parents

            public static VariableArray<int> AddChildFromFourParents(

                VariableArray<int> parent1,

                VariableArray<int> parent2,

                VariableArray<int> parent3,

                VariableArray<int> parent4,

                VarVectArr4 cpt)

            {

                var d = parent1.Range;

                // data range

                var child = Variable.Array<int>(d);

                using (Variable.ForEach(d))

                using (Variable.Switch(parent1[d]))

                using (Variable.Switch(parent2[d]))

                using (Variable.Switch(parent3[d]))

                using (Variable.Switch(parent4[d]))

                    child[d] = Variable.Discrete(

                        cpt[parent1[d]][parent2[d]][parent3[d]][parent4[d]]);

                return child;

            }

            // Model code for adding a child from five parents

            public static VariableArray<int> AddChildFromFiveParents(

                VariableArray<int> parent1,

                VariableArray<int> parent2,

                VariableArray<int> parent3,

                VariableArray<int> parent4,

                VariableArray<int> parent5,

                VarVectArr5 cpt)

            {

                var d = parent1.Range;

                // data range

                var child = Variable.Array<int>(d);

                using (Variable.ForEach(d))

                using (Variable.Switch(parent1[d]))

                using (Variable.Switch(parent2[d]))

                using (Variable.Switch(parent3[d]))

                using (Variable.Switch(parent4[d]))

                    child[d] = Variable.Discrete(

                        cpt[parent1[d]][parent2[d]][parent3[d]][parent4[d]][parent5[d]]);

                return child;

            }

     

            // Model code for adding a child from six parents

            public static VariableArray<int> AddChildFromSixParents(

                VariableArray<int> parent1,

                VariableArray<int> parent2,

                VariableArray<int> parent3,

                VariableArray<int> parent4,

                VariableArray<int> parent5,

                VariableArray<int> parent6,

                VarVectArr6 cpt)

            {

                var d = parent1.Range;

                // data range

                var child = Variable.Array<int>(d);

                using (Variable.ForEach(d))

                using (Variable.Switch(parent1[d]))

                using (Variable.Switch(parent2[d]))

                using (Variable.Switch(parent3[d]))

                using (Variable.Switch(parent4[d]))

                    child[d] = Variable.Discrete(

                        cpt[parent1[d]][parent2[d]][parent3[d]][parent4[d]][parent5[d]][parent6[d]]);

                return child;

            }

        }

    }

    this is the model's image. It can help you to understand the code.

     

    thanks

     

    Alessandro

     

    Friday, June 3, 2011 6:51 PM
  • John Guiver replied on 05-27-2011 12:10 PM

    Hi Alessandro

    When you get above depth 3, you have to use the generic Array factor method on the Variable class - this takes two type parameters. I show an example below of how to build up a variable - or you could try to do it in one statement.

    Apologies for not mentioning this in my previous post. You are the first ever (that I know of) to go to this depth!

    var a = Variable.Array<Vector>(r_o1);

    var b = Variable.Array<VarVectArr, Vector[][]>(a, r_o2);

    var c = Variable.Array<VarVectArr2, Vector[][][]>(b, r_o3);

    var d = Variable.Array<VarVectArr3, Vector[][][][]>(c, r_o4);

    var e = Variable.Array<VarVectArr4, Vector[][][][][]>(d, r_o5);

    var f = Variable.Array<VarVectArr5, Vector[][][][][][]>(e, r_p);

    John

    Friday, June 3, 2011 6:51 PM
  • Thanks you John,

    i realized why i am the first to go so deep, because it's not easy!

    i have another problem in this step, because no - one of my solution is correct.

                cpt_o5.SetTo(Variable<Vector[][][][][]>.Random(cpt_o5_prior));

                cpt_o5.SetTo(Variable<VarVectArr5>.Random(cpt_o5_prior));

    cpt_o5 is so definited:

    cpt_o5 = Variable.Array<VarVectArr4Vector[][][][][]>(Variable.Array<VarVectArr3Vector[][][][]>(Variable.Array<VarVectArr2Vector[][][]>(Variable.Array<VarVectArrVector[][]>(Variable.Array<Vector>(r_o1), r_o2), r_o3), r_o4), r_p);

    Regards

    Alessandro

     


    Monday, June 6, 2011 8:32 AM
  • Alessandro

    There are two ways to deal with this. You can either manage the priors as .NET arrays, or as Infer.NET distribution arrays (look back to the original discussions for this). The latter is more efficient (and is the one you are trying to use) if you are directly using the training posteriors to set the test-time priors because this is the natural way posteriors are stored; the first is more efficient, but the posteriors need to be converted/cast (via the Infer method type parameter). Anyway, let's first review the different aliases we need:

      using DirArr = DistributionRefArray<Dirichlet, Vector>;
      using DirArr2= DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>;
      using DirArr3 = DistributionRefArray<DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>, Vector[][]>;
      using DirArr4 = DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>, Vector[][]>, Vector[][][]>;
      using DirArr5 = DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<DistributionRefArray<Dirichlet, Vector>, Vector[]>, Vector[][]>, Vector[][][]>, Vector[][][][]>;
    
      using VarVectArr = VariableArray<Vector>;
      using VarVectArr2 = VariableArray<VariableArray<Vector>, Vector[][]>;
      using VarVectArr3 = VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>;
      using VarVectArr4 = VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>;
      using VarVectArr5 = VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<Vector>, Vector[][]>, Vector[][][]>, Vector[][][][]>, Vector[][][][][]>;
    
      using VarDirArr = VariableArray<Dirichlet>;
      using VarDirArr2 = VariableArray<VariableArray<Dirichlet>, Dirichlet[][]>;
      using VarDirArr3 = VariableArray<VariableArray<VariableArray<Dirichlet>, Dirichlet[][]>, Dirichlet[][][]>;
      using VarDirArr4 = VariableArray<VariableArray<VariableArray<VariableArray<Dirichlet>, Dirichlet[][]>, Dirichlet[][][]>, Dirichlet[][][][]>;
      using VarDirArr5 = VariableArray<VariableArray<VariableArray<VariableArray<VariableArray<Dirichlet>, Dirichlet[][]>, Dirichlet[][][]>, Dirichlet[][][][]>, Dirichlet[][][][][]>;
    
    

    Now for the first solution, we use:

    VarDirArr5 cpt_o5_prior = Variable.Array<VarDirArr4, Dirichlet[][][][][]>(Variable.Array<VarDirArr3, Dirichlet[][][][]>(Variable.Array<VarDirArr2, Dirichlet[][][]>(Variable.Array<VarDirArr, Dirichlet[][]>(Variable.Array<Dirichlet>(r_o1), r_o2), r_o3), r_o4), r_p);
    VarVectArr5 cpt_o5 = Variable.Array<VarVectArr4, Vector[][][][][]>(Variable.Array<VarVectArr3, Vector[][][][]>(Variable.Array<VarVectArr2, Vector[][][]>(Variable.Array<VarVectArr, Vector[][]>(Variable.Array<Vector>(r_o1), r_o2), r_o3), r_o4), r_p);
     cpt_o5[r_p][r_o4][r_o3][r_o2][r_o1] = Variable<Vector>.Random(cpt_o5_prior[r_p][r_o4][r_o3][r_o2][r_o1]);
    
    

    For the second solution, we use:

    Variable<DirArr5> cpt_prior = Variable.New<DirArr5>();
    VarVectArr5 cpt_o5 = Variable.Array<VarVectArr4, Vector[][][][][]>(Variable.Array<VarVectArr3, Vector[][][][]>(Variable.Array<VarVectArr2, Vector[][][]>(Variable.Array<VarVectArr, Vector[][]>(Variable.Array<Vector>(r_o1), r_o2), r_o3), r_o4), r_p);
    cpt_o5.SetTo(Variable<Vector[][][][][]>.Random(cpt_prior));
    
    

    You will, of course need to set the observed values for the priors appropriately. Hope this helps

    John 

    Tuesday, June 7, 2011 9:07 AM
    Owner
  • Yes John, 

     

    Your code is helpful,

    In spite of all I think the model is not currently implementable.

    When I run the code I get the following exception: "arrays of rank higher than 3 are not supported yet" as NotSupportedException at line:

    cpt_o5[r_p][r_o4][r_o3][r_o2][r_o1] = Variable<Vector>.Random(cpt_o5_prior[r_p][r_o4][r_o3][r_o2][r_o1]);

     


    Tuesday, June 7, 2011 3:23 PM
  • Hi Alessandro

    The second method should work fine, but it looks like the first method needs explicit for blocks (as I said, you are the first to go to this depth...) :

    using (Variable.ForEach(r_p))
    using (Variable.ForEach(r_o4))
    using (Variable.ForEach(r_o3))
    using (Variable.ForEach(r_o2))
    using (Variable.ForEach(r_o1))
    cpt_o5[r_p][r_o4][r_o3][r_o2][r_o1] = Variable<Vector>.Random(cpt_o5_prior_[r_p][r_o4][r_o3][r_o2][r_o1]);
    
    

    John

    Tuesday, June 7, 2011 5:39 PM
    Owner