locked
Best practice for complicated transform of RVs. RRS feed

  • Question

  • I need a complicated transform of several random variables. Are there some things to keep in mind for making sure the function of these random variables is calculated as efficiently as possible? 

    Specifically I have 2 random variables. A vector (say a) of length k, and a vector (say b) of length k(k-1)/2.  I need to iterate over the upper triangle of a * a' and sum up the upper triangle value multiplied by the corresponding value in b. 

    This is similar to coding up an interaction term in a regression. 
    • Edited by cyentist Tuesday, September 12, 2017 7:14 PM
    Tuesday, September 12, 2017 4:26 PM

Answers

  • If efficiency is the only concern then the most efficient approach is to construct an array of the products and then use Variable.Sum on the array.  The fastest way to compute the products is with two index arrays giving the corresponding elements of a to multiply with each element of b.  So the code is something like:

    products[j] = b[j]*a[index1[j]]*a[index2[j]];
    sum = Variable.Sum(products);

    To make it even faster, you want to exploit the fact that index1 and index2 are permutations.  See Indexing arrays by observed variables.

    • Marked as answer by cyentist Wednesday, September 13, 2017 3:46 PM
    Tuesday, September 12, 2017 8:53 PM
    Owner

All replies

  • This is equivalent to obtaining the quadratic form a'Ba where B is a matrix with the upper triangle equal to my vector b from original question. Can I construct B so that upper triangle is equal to B and constrain the rest to be zero and then form the quadratic form a'Ba. 

    Will that work? 

    Tuesday, September 12, 2017 8:36 PM
  • If efficiency is the only concern then the most efficient approach is to construct an array of the products and then use Variable.Sum on the array.  The fastest way to compute the products is with two index arrays giving the corresponding elements of a to multiply with each element of b.  So the code is something like:

    products[j] = b[j]*a[index1[j]]*a[index2[j]];
    sum = Variable.Sum(products);

    To make it even faster, you want to exploit the fact that index1 and index2 are permutations.  See Indexing arrays by observed variables.

    • Marked as answer by cyentist Wednesday, September 13, 2017 3:46 PM
    Tuesday, September 12, 2017 8:53 PM
    Owner
  • What if a is actually a vector of 0/1s and could be represented as vector of bool? 
    Tuesday, September 12, 2017 9:34 PM
  • For the Boolean case, you can use Variable.SumWhere.
    Wednesday, September 13, 2017 5:05 PM
    Owner