locked
Infer.net on linux (and other platforms) (Migrated from community.research.microsoft.com) RRS feed

  • Question

  • varungulshan posted on 01-22-2009 6:27 PM

    Hi,

    I tried to run Infer.net on linux (the .net framework is supported on linux through the mono project).

    The problem i ran into was that some of the assembled code in Infer.Compiler.dll and Infer.Runtime.dll is actually platform specific.(i.e it makes calls to unmanaged code in user32.dll, kernel32.dll, lapack.dll and atlas_p4.dll ). So this would mean that it only runs on windows.

    Some of the function calls to unmanaged code that the mono analyser could find were:

    Calling function                                                Invoked Method                               Library

    void SetToProduct (Matrix, Matrix, Matrix)        void Lapack.dgemm                         atlas_P4.dll

    bool CholeskyInPlace (Matrix)                       int Lapack.dpotrf                                lapack.dll

    (---- some more calls to functions in atlas_P4.dll --------- )

    There are calls to functions in user32.dll and kernel32.dll which it could not find the name for.

     

    It would be good if you could fix this. Though i dont have a particular solution to fixing this, i imagine something like this will work:

    1. For the user32.dll and kernel32.dll function calls, one should be able to find equivalent functions in managed code

    2. for the lapack calls, as lapack also exists for linux, there could be a way of writing code so that it can call the right function depending on platform.

     

    It would be useful for the research community, i imagine linux is used around a lot there, especially in computing clusters. Also if I am right, the source code for the entire library is not in the package for download? if that could be provided, i could try to fix it myself.

     

    Thanks,
    Varun

    PhD student,

    VGG, University of Oxford

    Friday, June 3, 2011 5:07 PM

Answers

  • John Guiver replied on 07-24-2009 5:13 AM

    Good news on the Mono front. We have put a fix in our code which works around https://bugzilla.novell.com/show_bug.cgi?id=522303, and we now have a version of Infer.NET that works (at least for the tutorial examples) on Mono. Our plan is to do a release this coming week that will include this fix.

    John G.

    Friday, June 3, 2011 5:09 PM

All replies

  • John Guiver replied on 01-23-2009 10:02 AM

    Hi Varun

    When running inference in Infer.NET, you should not see any calls into the lapack or atlas dlls. The modeling and inference methods are entirely dotNET. The reasons you see references to LAPACK (and the user32 and kernel32 dlls is that the core maths library does in fact include methods which support fast matrix operations, but these are not called by the factors or distributions. The typical application of Infer.NET uses large-scale factorised graphical models and there is not a lot to be gained from using LAPACK unless you are working with large vector Gaussian distributions.

    As regards making source avaliable, we do currently submit source for distributions and factors. It is possible we could at some time ship source for the core Maths libraries, but as discussed above, this will not help. However, we do not have plans to ship source for the compiler library - this is intellectual property which we would like to protect for now. In fact the whole compiler library is obsfuscated, and it is possible that the obfuscation itself might cause issues with running on mono.

     Have you just run the mono analyser, or have you tried running inference. Let us know what issues you come across.

     Thanks

     John G.

    Friday, June 3, 2011 5:07 PM
  • varungulshan replied on 01-24-2009 9:19 AM

    Hi John,

    thanks for this information. I did try to run the actual code also after running the mono-analyser. To be precise, I ran FirstExample.cs (in the directory Examples and Tutorials) . I just wrote a main function to create an instance of FirstExample and called FirstExample.Run().

    The code runs into error on the first line of FirstExample.Run() while instantiating a Bernoulli variable, here is the output that i get when I run my compiled program using mono:

    -------------- output begin ----------------------

     [varun@rugby Debug]$ mono FirstExample.exe

    Unhandled Exception: System.InvalidProgramException: Invalid IL code in MicrosoftResearch.Infer.Models.Variable:Bernoulli (double): IL_0007: call      0x2b000032


      at MicrosoftResearch.Infer.Tutorials.FirstExample.Run () [0x00000]
      at try2.MainClass.Main (System.String[ args) [0x00000]
    ------------------ output end ------------

     

    As you say, it could be the fact that the obfuscation itself is causing issues with mono. I thought that this error might be due to a call into user32.dll/kernel32.dll . I'll try to do some more research into it , and get back if I can find more precise information about this error.

     

    Thanks,

    Varun

    Friday, June 3, 2011 5:07 PM
  • kuangc replied on 02-22-2009 8:12 PM

    Hi Varun,

    I am also trying to see if I can get Infer.NET running on mono.

    > it could be the fact that the obfuscation itself is causing issues with mono. I thought that this error might be due to a call into user32.dll/kernel32.dll . I'll try to do some more research into it , and get back if I can find more precise information about this error.

    Curious where you found any additional info or a solution?

    Cheers,

    kuang

     

    Friday, June 3, 2011 5:07 PM
  • kuangc replied on 02-23-2009 12:17 AM

    By the way, I get the same error:

    Unhandled Exception: System.InvalidProgramException: Invalid IL code in MicrosoftResearch.Infer.Models.Variable:Beta (double,double): IL_0007: call      0x2b000026

    This is the offending line of code:

    lambda = Variable.Beta(1, 2).Named("lambda");

    I'm running Mono 2.0 on OS 10.5.

    Friday, June 3, 2011 5:07 PM
  • varungulshan replied on 02-23-2009 8:49 AM

    Hi Kuang,

    No, i have not tried to dig any deeper since I had this error, so i cant really help you.

    good luck in your search,
    Varun

    Friday, June 3, 2011 5:07 PM
  • laura replied on 02-24-2009 10:43 AM

    Hi Kuangc,

    I think you can develop infer.net models unter windows, but run them on mono. Have you tried running a model as a "compiled algorithm"?

    http://research.microsoft.com/en-us/um/cambridge/projects/infernet/docs/Using%20a%20precompiled%20inference%20algorithm.aspx

    If I remember correctly, the compiled algorithm does not involve calls such as Variable.Beta()

     

    By the way, if the generated code is not written to disk use

    InferenEngine inf = ...;

    inf.Compiler.GenerateInMemory = false;

     

    Laura

    Friday, June 3, 2011 5:07 PM
  • John Guiver replied on 02-24-2009 12:42 PM

    I think you will need call the generated model class directly rather than use the compiled algorithm class, as the compiled algorithm class is in the obfuscated DLL. To call the model class directly, see the section entitled 'Calling the compiled model class directly'  in http://research.microsoft.com/en-us/um/cambridge/projects/infernet/docs/Using%20a%20precompiled%20inference%20algorithm.aspx.

    John

    Friday, June 3, 2011 5:07 PM
  • kuangc replied on 02-25-2009 2:20 AM

    Hi John -- Thank you for the hint. I am hopeful this will work, but ran into the following issue.

    In trying to compile Model_EP.cs on Mono/OS X, I get (several of) the following error:

    error CS0309: The type `MicrosoftResearch.Infer.Distributions.Discrete' must be convertible to `MicrosoftResearch.Infer.Distributions.CanGetLogProb<MicrosoftResearch.Infer.Distributions.Discrete>' in order to use it as parameter `T' in the generic type or method `MicrosoftResearch.Infer.Factors.UnaryOp<DomainType>.LogEvidenceRatio<T>(DomainType, T)'

    The offending line(s) each make this particular call:

    Bernoulli.FromLogOdds(UnaryOp<int>.LogEvidenceRatio<Discrete>(this.spend_welfare_entry, this.vDiscrete1));

    I don't quite understand this error message. From the documentation, I see that the Discrete class implements CanGetLogProb<int>, but not CanGetLogProb<Discrete> as far as I can see. Yet, the error seems to say that inside LogEvidenceRatio(), the code is casting a Discrete into a CanGetLogProb<Discrete>...?

    Guessing: could it be possible that this is also an obfuscation issue?

    There are no other compilation errors. I double checked that Model_EP.cs compiles and runs on Windows.

    Thanks very much.

    Friday, June 3, 2011 5:08 PM
  • John Guiver replied on 02-25-2009 4:59 AM

    Hi Kuang

    This error should not be from obfuscation - only the Infer.Runtime dll need be referenced for the generated model class.

    Can you confirm that the first argument (this.spend_welfare_entry) is of type int  in the call

    Bernoulli.FromLogOdds(UnaryOp<int>.LogEvidenceRatio<Discrete>(this.spend_welfare_entry, this.vDiscrete1))

    If so, it seems that Mono may be getting confused by the two versions of LogEvidenceRatio on UnaryOp<int>. The one it should infer is the one with signature:

    public static double LogAverageFactor<T>(DomainType random, T dist)
    where T : CanGetLogProb<DomainType>

    where the distribution type T (Discrete in your case) needs to implement CanGetLogProb over the DomainType (int  in your case), which it does!

    John

    Friday, June 3, 2011 5:08 PM
  • kuangc replied on 02-25-2009 3:39 PM

    John Guiver:

    only the Infer.Runtime dll need be referenced for the generated model class.

    I think generated models still depend on IIterativeProcess in the Compiler dll; Specifically this error:

    > gmcs Model_EP.cs /r:Infer.Runtime.dll
    Model_EP.cs(25,33): error CS0246: The type or namespace name `IIterativeProcess' could not be found. Are you missing a using directive or an assembly reference?

    John Guiver:

    Can you confirm that the first argument (this.spend_welfare_entry) is of type int

    Yes, it is an int; furthermore, the generated model works in Visual Studio / Windows.

    John Guiver:

    If so, it seems that Mono may be getting confused by the two versions of LogEvidenceRatio on UnaryOp<int>. The one it should infer is the one with signature:

    public static double LogAverageFactor<T>(DomainType random, T dist)
    where T : CanGetLogProb<DomainType>

    where the distribution type T (Discrete in your case) needs to implement CanGetLogProb over the DomainType (int  in your case), which it does!

    Makes sense to me. Perhaps it's time to stop avoiding Windows :\

    Friday, June 3, 2011 5:08 PM
  • laura replied on 02-26-2009 4:52 AM

    John Guiver:

    If so, it seems that Mono may be getting confused by the two versions of LogEvidenceRatio on UnaryOp<int>. The one it should infer is the one with signature:

    public static double LogAverageFactor<T>(DomainType random, T dist)
    where T : CanGetLogProb<DomainType>

    where the distribution type T (Discrete in your case) needs to implement CanGetLogProb over the DomainType (int  in your case), which it does!

     

    Since the infer.net bundle includes the necessary source code, how about removing the wrong method and recompiling? 

    Is this prohibited by the infer.net distribution license?

     

    Kuangc, I am wondering if the "compiled algorithm approach" works under windows. Might be that this problem does not depend on the OS, but on how polymorphism is resolved. I had similar issues with Java after they introduced Generics.

     

     

    kuangc:

    Makes sense to me. Perhaps it's time to stop avoiding Windows :\

     

    Unfortunately there is no way to talk our administrators into setting up windows servers.

    Laura

     

    Friday, June 3, 2011 5:08 PM
  • John Guiver replied on 02-26-2009 5:33 AM

    laura:

    Since the infer.net bundle includes the necessary source code, how about removing the wrong method and recompiling? 

    Is this prohibited by the infer.net distribution license?

    I think it would be unwise for me to make any comment about what is permitted or not by the license - just read the license .

    Technically you will not be able to compile everything because we just distribute Distribution and Factor source, and not some needed dependencies. It is likely in the next release that we will move the IIterativeProcess interface into the run-time library, as it is an oversight that it is in the Compiler library.

    John

    Friday, June 3, 2011 5:08 PM
  • laura replied on 02-26-2009 5:47 AM

    John Guiver:
    Technically you will not be able to compile everything because we just distribute Distribution and Factor source, and not some needed dependencies.

    Ok, makes sense...

    John Guiver:
    It is likely in the next release that we will move the IIterativeProcess interface into the run-time library, as it is an oversight that it is in the Compiler library.

    How is this issue related to the IterativeProcess interface?

     

    Friday, June 3, 2011 5:08 PM
  • John Guiver replied on 02-26-2009 5:57 AM

    laura:

    How is this issue related to the IterativeProcess interface?

     

    Well, there are a couple of different issues being discussed here. This one relates to Kuang's note that IIterativeProcess was in the Compiler library, so that you have to reference the Infer.Compiler library even if you were only using the generated model class. This will still work, but is unnecessary, and it makes much more sense for IIterativeProcess to be in the Infer.Runtime dll.

    John

    Friday, June 3, 2011 5:08 PM
  • jwinn replied on 02-26-2009 5:46 PM

    If you wish to call the generated code directly, a workaround is to manually delete the reference to IIterativeProcess from the generated code - it is only needed when the code is to be wrapped in a CompiledAlgorithm.

    As John G. says, we will move IIterativeProcess into the Runtime DLL in a forthcoming release.

    Best, John W.

    Friday, June 3, 2011 5:08 PM
  • kuangc replied on 02-26-2009 9:15 PM

    Thanks for the suggestions to remove IterativeProcess. It does remove the dependency on Infer.Compiler.dll.

    That said, I still run into the issue of Mono not handling LogEvidenceRatio, etc -- correctly.

    Friday, June 3, 2011 5:08 PM
  • PierreM replied on 07-19-2009 1:35 PM

    I am also trying to use infer.NET with mono. I started with the "FirstExample" about coins. I tried to call the compiled model class directly, and deleted the dependency on interative process in it.

    The code I use to call the compiled model is the following :

    public static void Main()
            {
                Variable<bool> firstCoin = Variable.Bernoulli(0.5).Named("firstCoin");
                Variable<bool> secondCoin = Variable.Bernoulli(0.5).Named("secondCoin");
                Variable<bool> bothHeads  = (firstCoin & secondCoin).Named("bothHeads");
               
                Model_EP model = new Model_EP();
               
                model.Reset() ;
                model.Initialise() ;
                for (int i=0; i<20; i++)
                    model.Update() ;
                   
               
                Bernoulli bothHeadsMarginal = model.BothHeadsMarginal() ;
                Console.WriteLine("Probability both coins are heads: " + bothHeadsMarginal);
            }

    this code works fine, but mono throws the following error message :

    Unhandled Exception: System.InvalidProgramException: Invalid IL code in MicrosoftResearch.Infer.Models.Variable:Bernoulli (double): IL_0007: call      0x2b000032

    I read the previous posts, but I couldn't find any way to correct it by myself.

    Thanks for your help !

     

    Friday, June 3, 2011 5:09 PM
  • John Guiver replied on 07-20-2009 3:16 AM

    Hi Pierre

    When you say that Mono throws the error, is this the analyser that throws the error? If so, I am guessing that this is to do with the LAPACK references. However, as discussed in a previous post, there are no calls into unmanaged code.

    On a more general topic, we have actively been looking into why there are issues with running Infer.NET under Mono. We have found that the problems have nothing to do with obfuscation in the Infer.NET compiler library (as I had previously hypothesised), but are due to a bug or bugs in the Mono runtime support for generics - see https://bugzilla.novell.com/show_bug.cgi?id=522303. There are no immediate plans to fix this bug, though even if it were fixed, this might just be the tip of the iceberg, as Infer.NET makes very extensive use of generics. We are still wrapping up our investigation of Mono/Infer.NET, and will post any further information as it becomes available.

    John

    Friday, June 3, 2011 5:09 PM
  • PierreM replied on 07-20-2009 5:20 AM

    Thank you for the quick reply, John !

    Friday, June 3, 2011 5:09 PM
  • John Guiver replied on 07-24-2009 5:13 AM

    Good news on the Mono front. We have put a fix in our code which works around https://bugzilla.novell.com/show_bug.cgi?id=522303, and we now have a version of Infer.NET that works (at least for the tutorial examples) on Mono. Our plan is to do a release this coming week that will include this fix.

    John G.

    Friday, June 3, 2011 5:09 PM