locked
how can i setup simple 2d or 1d vector model RRS feed

  • Question

  • I am in the middle of a poker research project...

    I need to utilize an inference engine that will extrapolate/interpolate what cards someone might be holding given a simple first action on the first betting round based on what cards they were observed holding for correlating first action on first betting round.

    Base example:

    Someone has been observed being the first to raise for their first action on the first betting round , 4 times with AA, 7 times with KK, and 3 times with JTs.  There are 169 different starting hands.  So i have observed data on only 3 of these 169 starting hands.  I need to make an inference that would tell me when this person raises the next time first action on first betting round...what is the likelihood of them holding each of the 169 starting hands.

    so for my model do i just create 169 variables for each of the players i have data on?

    I have been looking at the examples and stuff And i was thinking i would use VMP on this, but mostly an educated guess here on that.

    Any help would be greatly appreciated.

    edit--

    I think i will try out the simplegaussian example.  But i still gotta find out how to output results to a .txt


    karlkruse



    ,
    • Edited by karlkruse Saturday, February 11, 2012 11:55 PM
    Saturday, February 11, 2012 12:01 AM

All replies

  • ok i have messed around with some examples, but i need to figure out how to do this.

    Say i have an array of like 10 values {0,3,12,6,20,3,2,2,1.0}

    key things i need to use in this are the fact that numbers can be from 0 to infiniti.

    Also numbers cannot be double type value.

    How i want to do this is taking the sum of all indexes above and then inferring a new array with say a sum of 10,000.

    so the sum above would be the total of each index ( each of these numbers represent the times a certain hole card pair went to showdown ) so the sum represents the total number of showdowns seen.

    So i do i infer the array as if there were actually 10,000 total showdowns seen.  I would need values from 0 to totalshowdowns seen for each index.

    Is this possible?

    edit--

    I just found out it might be how i define my model variables.

    edit--

    Is this a situation that requires multivariate variables?

    is a vector one where each index is considered sort of dependent on others so that they are related in some way?

    i think i need to use bpm example but looks complicated in vs2010.


    karlkruse





    • Edited by karlkruse Saturday, February 11, 2012 10:16 PM
    Saturday, February 11, 2012 7:23 PM
  • From your initial description, it sounds like you want something like this:

    			Range game = new Range(3);
    			Range possibleAction = new Range(3);
    			Range possibleHand = new Range(169);
    			VariableArray<int> hand = Variable.Array<int>(game);
    			VariableArray<Vector> actionProbs = Variable.Array<Vector>(possibleHand);
    			actionProbs[possibleHand] = Variable.DirichletUniform(possibleAction).ForEach(possibleHand);
    			VariableArray<int> action = Variable.Array<int>(game);
    			using (Variable.ForEach(game)) {
    				hand[game] = Variable.DiscreteUniform(possibleHand);
    				using (Variable.Switch(hand[game])) {
    					action[game] = Variable.Discrete(actionProbs[hand[game]]);
    				}
    			}
    
    			InferenceEngine engine = new InferenceEngine();
    			hand.ObservedValue = new int[] { 4, 5, 6 };
    			action.ObservedValue = new int[] { 2, 1, 0 };
    			Console.WriteLine(engine.Infer(actionProbs));
    
    Here I have assumed you have observed the same player across three "games" (a game being a single poker hand) and they have chosen one of three possible actions each time. Note that if you do it this way, you will need quite a lot of training data to learn the full actionProbs table. A smarter approach is to compute various features of the hand such as strength of cards, whether they match, etc. and compute actionProbs from those. This computation would involve learnable parameters, similar to a multi-class Bayes Point Machine or softmax regression.
    Sunday, February 12, 2012 6:04 PM
    Owner
  • sorry i was a little unclear.  I am new to probability/inference.

    Your code is similar to what i want...

    however full scenario is like this.

    I need a seperate model and seperate inference for each action someone takes the first time they act on the first betting round.

    like if some raises first action preflop, i want to look at all the times they did this very same action and see the probabilites of them holding each specific card pair based on an inferred vector of actual observed hands they showed at showdown for this particular action.

    So if i start with just when they raise for the first action on the first betting round i have this.

    Observed:

    Player1_shownhands_whenraisedfirstaction(thisshould probably be a vector from array){3,5,0,3,3,0,1,etc. all the way to n(number of indexes or Range you might say)=169}

    Player2_shownhands_whenraisedfirstaction(thisshould probably be a vector from array){{2,3,0,1,0,4etc. all the way to n=169}

    etc. for all players in my hand history database

    For my prior i was thinking an actual covariance matrix of all the player(x) vectors

    Then i use the prior which is some sort of mean vector based on the covariance matrix of all players, set the observed vector variables for each player as shown above, and then make the final inference for each player based on the observed and prior just mentioned.

    i have done a lot of testing and i am having trouble with these few areas.

    1. How to create a two dimensional array, first dimension is just indexes for each player, 2nd dimension is vectors for each players 169 starting hands.

    2. How to possibly create a prior based on some sort of mean of all of the actual observed hands for all players.

    3. When doing inference do i have to do an inference one player at a time, or can i infer for all players i have listed for observed

    4. finally, not a big deal just need to output to .txt

    Thanks for any help/advice.

    edit-- oh yea, and as far as the "learning goes"  I only have one learning phase.  I don't want to update during actual gamplay.  I want to take one huge chuck of obeserved hands and use that to make an inference that will be used for next session of gameplay.  then every so often redo the whole process over.  Reason for this is I have about 1 million observed hands, but will only be playing sessions of about 200-300 hands so it isn't really worth learning in a manner other than in one huge chunk of data.


    karlkruse




    • Edited by karlkruse Sunday, February 12, 2012 7:19 PM
    Sunday, February 12, 2012 7:02 PM
  • The code I sent already does essentially what you are saying.  If you want to predict the person's hand given a new action, just add the following code to the end:

    			Variable<int> newHand = Variable.DiscreteUniform(possibleHand);
    			Variable<int> newAction = Variable.New<int>();
    			using (Variable.Switch(newHand)) {
    				 newAction.SetTo(Variable.Discrete(actionProbs[newHand]));
    			}
    			newAction.ObservedValue = 2;
    			Console.WriteLine("predicted hand given action = {0}:", newAction.ObservedValue);
    			Console.WriteLine(engine.Infer(newHand));
    

    This outputs a big probability distribution over hands.  In this example, the highest probability is on hand 4.

    Sunday, February 12, 2012 8:06 PM
    Owner
  •             Range Possiblehands = new Range(169);
                Range NumberplayersObserved = new Range(3);
                VariableArray<vector> hands = Variable.Array<vector>(Possiblehands);
                hands[Possiblehands] = Variable.DirichletUniform(NumberplayersObserved).ForEach(Possiblehands);
        
    			InferenceEngine engine = new InferenceEngine();
                hands[0].ObservedValue = new int[] { 2, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 2, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 };
                hands[1].ObservedValue = new int[] { 1, 1, 0, 0, 2, 0, 2, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0 };
                hands[2].ObservedValue = new int[] { 5, 1, 6, 0, 2, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 3, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0 };
                
    			Console.WriteLine(engine.Infer(hands[0]));
                Console.WriteLine(engine.Infer(hands[1]));
                Console.WriteLine(engine.Infer(hands[2]));
                Console.WriteLine("test");
                while (true) ;</vector></vector>

    This is what i am working on now based on your previous code.

    The above is real observed data for 3 players.  I need to infer what each of their hands[] array would be as if they played like 1 million hands each.

    hands[0] corresponds to observed hands for player1, hands[1]  corresponds to obsserved hands for player 2 etc.

    obviously this won't work as i am getting an error saying i can't assign type double to a vector with my .observed lines at bottom.

    edit--

    i guess this line:

    VariableArray<Vector> hands = Variable.Array<Vector>(Possiblehands);
    cannot hold more than one vector i guess.

    this code might be helpful for me online


    double[] incomes = { 63, 16, 28, 55, 22, 20 };
    double[] ages = { 38, 23, 40, 27, 18, 40 };
    bool[] willBuy = { true, false, true, true, false, false };

    Vector[] xdata = new Vector[incomes.Length];
    for (int i = 0; i < xdata.Length; i++) xdata[i] = Vector.FromArray(incomes[i], ages[i], 1);
    VariableArray<Vector> x = Variable.Observed(xdata);
    VariableArray<bool> y = Variable.Observed(willBuy, x.Range);

    karlkruse



    • Edited by karlkruse Sunday, February 12, 2012 9:46 PM
    Sunday, February 12, 2012 8:27 PM
  • Why did you change hands to be a Dirichlet vector?  I think you are getting stuck on the idea that the observations must be in the form of counts such as "4 times with AA".  It is much easier and more flexible to model individual occurrences, so that your data is { AA, AA, AA, AA } instead of "4 times with AA".  That is what my code does.
    Sunday, February 12, 2012 11:31 PM
    Owner
  • sorry but i don't get what you mean.  your second code above return 169 values of the same exact value.  I need something that will infer based on hand histories.  I can format the data any way that is not a problem.  but i don't get what you mean by {AA, etc. }  do you mean seperate all of the single distinct hands apart from each other? did you get same values for your code above like i just mentioned.

    okay i ran your code again basically i get 169 diffent dirichlets(with like( 1,21)(1,1,1)etc) and then i get i believe about a 169 discrete uniform of all .005197 with your code.  Is this what you get?


    karlkruse



    • Edited by karlkruse Monday, February 13, 2012 12:35 AM
    Monday, February 13, 2012 12:06 AM
  • The inferred distribution of newHand is not uniform, because it is based on the actionProbs that were learned from the training data.  If you have trouble sifting through the output, try making the number of possible hands smaller (like 10).  In my code, the training data hand={ 4, 5, 6 } action={2,1,0} means that hand number 4 was observed with action 2, followed by hand number 5 with action 1, etc.  If hand 4 was observed again with action 2, then this would simply be repeated, i.e. hand = { 4, 5, 6, 4 } action = { 2, 1, 0, 2 }. 

    Monday, February 13, 2012 10:16 AM
    Owner
  • i think your code won't change the probabilities of them holding each hand.  I need a range of all hands returned from the inference engine that is a probability vector.  Each element is the chance of them holding one of the 169 starting hands.  Your code from what i have tried to understnad thus far would just work with one hand( by one hand i don't mean on hand history, i mean a specific hole card pair like AA).  I need to have an intial probability vector as a prior which would have inference done on it to return another probability vector as a posterior.  Thing is though i am not even sure with infer.net if you can infer a probability vector as a whole.

    karlkruse


    • Edited by karlkruse Monday, February 13, 2012 3:12 PM
    Monday, February 13, 2012 3:11 PM
  • The code I sent does infer an entire probability vector.  Here is the output from Console.WriteLine(engine.Infer(newHand)):

    Discrete(0.005917 0.005917 0.005917 0.005917 0.008876 0.004438 0.004438 0.005917 0.005917 0.005917 0.005917 0.005917 0.005917 0.005917 0.005917 0.005917 0.00591 ...)

    It is a probability vector over all 169 starting hands.  All hands except 4,5,6 have the same probability.  As you can see, the starting hand numbered 4 is the most likely since the observed action was 2 and in the training data this happened with hand 4.  So it is using the training data as you requested, and inferring an unknown starting hand from the observed action.

    Monday, February 13, 2012 11:35 PM
    Owner
  • oh, i guess i thought they were all .005917.

    What i don't understand is how i can't just train the engine with vectors of 169 elements one at a time and THEN get a probability vector.

    Reason i need this is i am using the 169 element probability vector to be inputted into a winning percentage calculator.  So i don't need the greatest probablity of one specific hand i need the probablity they could be holding any of the hands at one particular time.

    i think it will still be easier if i structure my code like i was trying to do above.

    Can you help me complete my code so it works like yours. 

    Sorry, i tried this stuff for like 10 hours over the the past 2 days and i sort of give up.

    What do you think about possible helping me train the machine with 169 element arrays rather than the way you did it?

    or i could do this as a weighting idea.

    use all players 169 element vector to get one "inferred" vector that would be sort of like a mean of all players.

    Then for each player i can do this:

    For observed data i use the average vector repeated the average number of each times each player has acted in the present manner.

    Then for more observed for the specific player in context i use their vector the number of times they acted in the present manner.

    So say i have 10 players.

    the average cards seen at showdown for each might be 12

    lets call the inferred "average" vector for all the AverageVectorForAllPlayers

    so before i make an inference for player one i input the observed data as follows, if for instance player 10 has only had 10 cards seen at showdown.

    i do twelve observed vectors of the AverageVectorForAllPlayers

    then i do

    10 vectors for the player i am going to get inference for.

    This is so if a player has less hands seen than the average player they wont diverge from the average to much.  But if they have more lets say a lot more hand seen than the average player the inference will converge on the real players observed data.

    Completely sorry if i am not fully explaining myself properly.


    karlkruse




    • Edited by karlkruse Tuesday, February 14, 2012 1:35 AM
    Tuesday, February 14, 2012 1:18 AM
  • Are you saying that you don't know how to convert from a vector of 169 counts into an array of numbers describing the individual hands (that my code uses)?  Is this basically a C# question?
    Tuesday, February 14, 2012 8:00 AM
    Owner
  • i give up, this is just too underdeveloped with too little support i think it would be best if i waited a while till their is some official support for infernet. 

    NO, i am not wondering how to convert a vector to an array.  Just how to return an probability vector directly from another probability vector of 169 elements through inference.


    karlkruse

    Tuesday, February 14, 2012 11:52 AM