locked
This model will consume a lot of memory due to the indexing expression RRS feed

  • Question

  • I am trying to build a Markov Chain mixture model for sequence clustering. As the first step, I am trying to recover the model parameters for one cluster: (1) init state probabilities , (2) state transition probabilities.

    I start with defining model ranges:

    N = new Range(NumChains);
    T = new Range(ChainLength);
    K = new Range(NumStates);

    Then I create init state and transition probability table:

    ProbInitPrior = Variable.New<Dirichlet>();
    ProbInit = Variable<Vector>.Random(ProbInitPrior).Named("ProbInit");
    ProbInit.SetValueRange(K);
    
    CPTTransPrior = Variable.Array<Dirichlet>(K);
    CPTTrans = Variable.Array<Vector>(K).Named("CPTTrans");
    CPTTrans[K] = Variable<Vector>.Random(CPTTransPrior[K]);
                CPTTrans.SetValueRange(K);

    I create the primary model variables:

    ZeroStates = Variable.Array<int>(N).Named("ZeroStates");
    ZeroStates[N] = Variable.Discrete(ProbInit).ForEach(N);
    
    States = Variable.Array(Variable.Array<int>(T), N).Named("States");

    Then comes the model definition:

    using (Variable.ForEach(N))
    {
      using (var block = Variable.ForEach(T))
      {
        var t = block.Index;
        var previousState = States[N][t - 1];
    
        using (Variable.If(t == 0))
        {
          using (Variable.Switch(ZeroStates[N]))
            States[N][T] = Variable.Discrete(CPTTrans[ZeroStates[N]]);
        }
    
        using (Variable.If(t > 0))
        {
           using (Variable.Switch(previousState))
             States[N][T] = Variable.Discrete(CPTTrans[previousState]);
        }
      }
    }

    The posteriors that I am trying to get are:

    CPTTransPosterior = engine.Infer<Dirichlet[]>(CPTTrans);
    ProbInitPosterior = engine.Infer<Dirichlet>(ProbInit);

    The engine is instantiated with the default parameters. The inference appears to be running just fine and it seems that I am able to recover the model parameters (there are some discrepancies but that perhaps is another question). What bothers me is this warning message:

    Compiling model...compilation had 2 warning(s).
      [1] This model will consume a lot of memory due to the indexing expression CPTTrans[States[N][T-1]] inside of a loop over N. Try simplifying this expression in your model, perhaps by creating auxiliary index arrays.
      [2] This model will consume a lot of memory due to the indexing expression CPTTrans[States[N][T-1]] inside of a loop over T. Try simplifying this expression in your model, perhaps by creating auxiliary index arrays.

    How can I get rid of this message? Are there any other issues with my model that I overlooked?

    Here are ground truth parameters:

    Init:
    [0.0394060758037617 0.0241456599948052 0.936448264201433]
    
    TRUE CPTTrans:
    [0.0531041028562771 0.705921889845134 0.240974007298589]
    [0.394798251714796 0.396364976539721 0.208836771745483]
    [0.110165777745994 0.874464160055694 0.0153700621983119]

    And here are the inferred ones from the data that has 100 chains, each chain is 10 symbols long and there are 3 states:

    ESTIMATED: ProbInit Posterior:
    0.9556 0.03241 0.01199
    
    ESTIMATED: CPTTrans Posterior:
    0.052 0.4748 0.4732
    0.4014 0.3759 0.2226
    0.1426 0.8236 0.03379

    Ignoring the order difference, the estimates seem to be ok almost everywhere, except for few parameters in the transition matrix. The difference is quite big for some parameters. I wonder if this is linked to that model warning or something else.


    • Edited by usptact Monday, March 5, 2018 9:53 PM
    Monday, March 5, 2018 9:49 PM

Answers

  • You should ignore that message in this case.
    • Marked as answer by usptact Tuesday, March 6, 2018 1:19 AM
    Monday, March 5, 2018 11:25 PM
    Owner

All replies

  • You should ignore that message in this case.
    • Marked as answer by usptact Tuesday, March 6, 2018 1:19 AM
    Monday, March 5, 2018 11:25 PM
    Owner
  • Thanks, Tom! There's still some issue with the model. I wrap the whole section with variable definitions and the model in a section to find the model evidence

    Variable<bool> ModelEvidence = Variable.Bernoulli(0.5);
    
    using (Variable.If(ModelEvidence))
    {
       ...
    }
    
    ...
    
    Bernoulli ModelEvidencePosterior = engine.Infer<Bernoulli>(ModelEvidence);
    
    

    I get 0 model probability while the other two model posterior estimates look ok.

    Model evidence: Bernoulli(0)

    Am I missing something obvious?

    Tuesday, March 6, 2018 4:50 AM
  • You should be looking at ModelEvidencePosterior.LogOdds.
    Tuesday, March 6, 2018 12:23 PM
    Owner