locked
Precision of Vector type (Migrated from community.research.microsoft.com) RRS feed

  • Question

  • sungchul kim posted on 01-11-2011 9:40 PM

    Hi all,

    To checked the data set, I recorded data before (double[]) and after changing it into Vector type (using Vector.ToString()), and I found that the second one shows that the precision is only 0.0001 (the precision of raw data is at least 0.000001). The followings are some examples.

    0.166667 0.196279 0.163982 0.236024 0.847046 0.025027 0.399966

    -> 0.1667 0.1963 0.164 0.236 0.847 0.02503 0.4

    0.083333 0.196279 0.081991 0.191127 0.847046 0.023165 0.200046

    -> 0.08333 0.1963 0.08199 0.1911 0.847 0.02316 0.2 

    Does Vector type have this much of precision? It so, can I increase the precision? I think it makes the results incorrect.

    Friday, June 3, 2011 6:11 PM

Answers

  • John Guiver replied on 01-12-2011 4:39 AM

    Hi SungChul

    You are not losing any precision for the in-memory vector. Vectors have double precision. The ToSring method on Vector (as on any object) is just a way to give a reasonable default printout. We provide a ToString override to allow you to change the on-screen precision and also the delimiter (alternatively you can iterate through the Vector and print out the elements individually):

    Vector v = Vector.FromArray(0.123456789, 0.987654321);
    Console.WriteLine(v.ToString("0.00000000", ", "));

    John

    Friday, June 3, 2011 6:11 PM