locked
Java vs Python for quotes updates? RRS feed

  • Question

  • Hello,

     

    Could anybody who has tried both available quotes updates solutions (the Java app, or Pocketsense scripts) please give some comparison advice?

    Ease of setup?

    Automation of the daily import process?

    Can either support import from multiple sources (as I have never found a server which provides quotes for ALL funds, including Swiss domiciled funds and some UK funds)?

     

    Sunday, January 30, 2011 8:06 AM

All replies

  • I have a test build for the java version that has support for additional quote sources:

      . ft.com: which has good coverage for UK funds

      . Google portfolio

      . An alternate Yahoo quote source that can look up option prices (as well as some symbols that the main Yahoo quote source has problem retrieving prices)

     

    Readme:

      http://www.le.com/~hle/out/hleOfxQuotes/test/latest/readme.txt

      http://www.le.com/~hle/out/hleOfxQuotes/test/Build_20110125_001/readme.txt

      http://www.le.com/~hle/out/hleOfxQuotes/test/Build_20110124_001/readme.txt

    Jar:

      http://www.le.com/~hle/out/hleOfxQuotes/test/latest/hleOfxQuotes.jar

     

    Sunday, January 30, 2011 3:50 PM
  • I have tried both, and until recently i had preferred the Python scripts (PocketSense) because the author was very responsive to revising code to comply with users feedback, the quotes would import directly into Money and because it was so easy to review the code for that personal satisfaction of data security and integrity.   But Hung Le's Java app (the only one to-date that I am aware of) has been updated several times recently (1/25/2010 is the most recent alternative version) in response to International user's requests and may in-fact be the more versatile at this time.   He needs some more users to try it out and provide feedback (here if Microsoft keeps this forum up, or ) on my Blog, where details on his Java apps can also be found: http://microsoftmoneyoffline.wordpress.com/2010/02/12/java-app-to-update-quotes/   Be sure to mention what Build you are using  when providing feedback / comments.

    Another advantage to Java is that it already exists in most computers already (unlike Python), but on the flip side, if you want to "decompile" the code to review it, you'll need to unzip the .jar package with WinZip first.   And of course the Java app only provides quotes, whereas PocketSense also accesses your statements at your financial institutions and feeds them to Money as well.

    • Edited by ameridan Sunday, January 30, 2011 7:13 PM
    Sunday, January 30, 2011 3:55 PM
  • For those fetching all quotes  in USD (US$), this quote fetching is a pretty easy process. The US folks don't need to read further. Non-US users who get their quotes with the OFX files from their brokers would not see the issues discussed below. Most other users would not run into the things being brought up below. Things are going to get more complex for some:

    For those non-US users and developers of the fetching software, there are some things I would like to add. When you bring in quotes from a server, they will be in some currency. So far, none of the software converts the currency on its own. The quotes would have to be fed into an account in Money who's currency matches that of the quotes. This would be done by having the OFX file fed to Money to have an account number that Money would remember goes to the account for that currency. The user would have to keep the exchange rates being used close enough.

    The user could take care of this issue by having a separate folders for the hleOfxQuotes or PocketSense installs for each download currency. Each instance would handle the downloads for a given currency. The things that have to be changed for each non-USD currency for PocketSense is to change the USD string to the appropriate currency symbol (e.g., EUR), and to change the pretend account number "0123456789" to something else (e.g., "EUR3456789") in the quotes.py file. I don't know how hleOfxQuotes handles currencies. One interesting alternative is to have PocketSense handle one currency and hleOfxQuotes to handle another. That might be a good solution for some user.

    Another issue is that Money limits (truncates) symbols that you provide to some length. For Money Plus Deluxe, that is 12 characters. Some method of handling that is needed. A sophisticated method would be to have the download software to use a translation table to a unique shorter symbol. An easier method that will work in most cases is to just truncate the symbol. That would work except in the rare case where a user has two securities whose symbols differ only in the lost characters. (Actually, I just thought of a potential alternative that maybe could be used by developers after experimentation, but that would only confuse the issue.) Actually this symbol-length issue is shared by US users of listed option. The new symbols for those are now huge -- 16 to 20 characters. I have an experimental version of Robert's quotes.py that truncates the symbols to 12 characters. I don't actually use it for myself, because I don't have anything with long symbols.

    Developers may address these issues more transparently in future releases.

    Sunday, January 30, 2011 6:25 PM
    Moderator
  •  

    Cal,

    For the symbol length issue, I use a mapper.csv fie to let user map

    MMSymbol: QuoteSourceSymbol

    User specify QuoteSourceSymbol, when I write the *.ofx file, I use the corresponding MMSymbol

    For currency issue: currently I  do write out the OFX currency tag per security but I have no clue if MM will honor it

          [SECID]
        [UNIQUEID]EZJ:LSE[/UNIQUEID]
        [UNIQUEIDTYPE]TICKER[/UNIQUEIDTYPE]
          [/SECID]
          ...
          [CURRENCY]
        [CURRATE]1.00[/CURRATE]
        [CURSYM]GBP[/CURSYM]
          [/CURRENCY]

    I set CURRATE to 1.00 right now but I have code to look up the exchange rate (using the pseudo-symbol USDGBP) but I have not put the code in yet because I did not want to muddle the water right now for the UK folks who are still having problem getting MM2005UK to update the price. Obviously, we can also convert from GBP -> USD before writing to *.ofx.

    I want to give Google Porfolio a plug here regarding how it handles currency. For Google Portfolio, you can set the porfolio base currency and Google will report the position market value in BOTH: "native" currency and portfolio based currency. For example, let say I set my base currency to GBP and have a position of AAPL, Google will report market value in both USD and GBP. So that will take of currency conversion outside OFX generation tool. Currently hleOfxQuotes.jar can fetch data from Google Portfolio and will:

      1. preserve shared count (instead of using 0 count)  // this is mostly to see if it will help the UK folks

      2. using "native" currency // since I don't know how multi-currency actually works in MM

    But it is fairly easy to add a couple check boxes to let user choose what he/she wants.

    FYI, I am back to using MM on daily basis now (hence more active in updating hleOfxQuotes.jar). I gave Quicken a college try but things are not working out:

      . I am used how the UI works and could not really overcome that

      . In Quicken, if I hide the reconciled transaction, the balance is displayed certain way that is just too confusing.

    I don't mean to bash Quicken here but just a note to say that I am back to the "right" side ;-).

     

     

     

    Sunday, January 30, 2011 7:12 PM
  • For currency issue: currently I  do write out the OFX currency tag per security but I have no clue if MM will honor it

          [SECID]
        [UNIQUEID]EZJ:LSE[/UNIQUEID]
        [UNIQUEIDTYPE]TICKER[/UNIQUEIDTYPE]
          [/SECID]
          ...
          [CURRENCY]
        [CURRATE]1.00[/CURRATE]
        [CURSYM]GBP[/CURSYM]
          [/CURRENCY]

    I set CURRATE to 1.00 right now but I have code to look up the exchange rate (using the pseudo-symbol USDGBP) but I have not put the code in yet because I did not want to muddle the water right now for the UK folks who are still having problem getting MM2005UK to update the price. Obviously, we can also convert from GBP -> USD before writing to *.ofx. 

    Interesting. So the hope would be that a currency and a looked-up conversion rate could be associated with a given quote, and Money would do the math.

    I would be surprised if it works the way we would like it to work. If I understand, you have already made such OFX files. If you can generate such an OFX file and upload it to www.speedyshare.com and post the URL, I can run some experiments with a test *.mny file in Money Plus Sunset. Note the delete key to allow you to delete after the need has gone away. There is no need to modify the actual file to remove certain characters. I would try, for example, changing the 1.00 to another value, and try other variations, and to observe the effects.

    Sunday, January 30, 2011 9:34 PM
    Moderator
  • Hi Cal,

      A sample: http://www.le.com/~hle/out/Cal.Learner/quotes.ofx

     

    Monday, January 31, 2011 12:41 AM
  • Got it. Experimenting. No clue yet that

    <CURRENCY>
      <CURRATE>1.35751</CURRATE>
      <CURSYM>EUR</CURSYM>
    </CURRENCY>

    has no effect. Still experimenting. I am getting some info.

     

    Monday, January 31, 2011 2:35 AM
    Moderator
  • I have a multi-currency portfolio - USD, Yen, SFR, EUR and GBP - I think I am going to wait for developers to work this out...
    Monday, January 31, 2011 12:10 PM
  • Having an issue importing quotes using pocket sense.  I am not prompted for an account as noted in this step.

    "Choose Yes and, when prompted by Money, select your dummy account to receive the stock/fund quotes."  I created a dummy Quotes account but not prompted for it nor prompted to created an import account.

    I also received an error that states "the file you attempted to import appears to be invaild or contains corrupt data."

    I noticed during pocketsense execution that ^DJI appears as an invalid stock symbol but in Yahoo finance it displays the Dow Jones Indust price just fine.

    Thank you

     

     


    Andrew
    Tuesday, February 1, 2011 7:46 AM
  • I also received an error that states "the file you attempted to import appears to be invaild or contains corrupt data." 

    The xfr folder contains the *.ofx files that have been sent to Money. You could re-send it by running the OFX file. If the file is incorrrect, you would get a failure again. You did not say if the error was a one-time event or a recurring thing.

    Is the quotes*.ofx the only *.ofx file there? If there is more than one, you may not know which file caused your error message.  Note that the *.ofx files in that folder are are deleted the next time Getdata.py is run. 

    You could try analyzing the file, or you could post it somewhere for somebody else to analyze. Only do that for quotes*.ofx file or sites.dat, because the other files would contain your real account number and holdings. quotes*.ofx  only has quote info. So if you zipped up your sites.dat and quotes*.ofx, and made the zip file available, then somebody could take a look to see what is happening.

    Tuesday, February 1, 2011 8:12 AM
    Moderator
  • Thanks for the pointers Cal.  I fixed my issue.  It appears that a fund symbol is the culprit.  The fund was found by Yahoo Finance but when I manually looked at the price (1.00) I knew that something was wrong so I removed the fund symbol and all is right with quotes.

    The fund in question:  SSgA Money Market Instl  (MUTF:SSMXX)

    Maybe I shouldn't have listed that in the Funds section, I don't know, but it's not absolutely necessary since Fidelity will download the right quote. 

    I am second guessing doing these quote updates since as stated before the PocketSense script will download the the statement and update - I figured I could have the most recent price by adding quotes.

     


    Andrew
    Wednesday, February 2, 2011 2:55 AM
  • Re: "^DJI appears as an invalid stock symbol". See: http://code.google.com/p/hle-ofx-quotes/wiki/Index_Symbols

     

    Wednesday, July 6, 2011 3:09 PM
  • Having an issue importing quotes using pocket sense.  I am not prompted for an account as noted in this step.

    "Choose Yes and, when prompted by Money, select your dummy account to receive the stock/fund quotes."  I created a dummy Quotes account but not prompted for it nor prompted to created an import account.

    I also received an error that states "the file you attempted to import appears to be invaild or contains corrupt data."

    I noticed during pocketsense execution that ^DJI appears as an invalid stock symbol but in Yahoo finance it displays the Dow Jones Indust price just fine.


    Andrew

    I see that in the newest release or Pocketsense, the initial sites.dat uses INDU rather than ^DJI. I understand that either works in the regular interface. I suggest you switch to that symbol. I also see that is what hleOfxQuotes suggested.

    I had originally made a diffrent assumption as to the cause. In that I suggested using the "s:" switch in the sites.dat.  To describe that that I referenced http://sites.google.com/site/pocketsense/home/msmoneyfixp1/revisionhistory , which may be of interest anyway.

    Regarding your Quotes folder, ignore that for now unless bogus transactions show up elsewhere.





    Wednesday, July 6, 2011 3:45 PM
    Moderator
  • Cal,

    I aimed Bryan and hleOfxQuotes on the right path yesterday: With Yahoo, simply use the symbol INDU for Dow Jones quotes now.

    Please fix your note (it's not INTU) so others don't try the wrong symbol again :)

    -Dan

    • Proposed as answer by ameridan Thursday, July 7, 2011 5:28 PM
    Thursday, July 7, 2011 5:27 PM
  • Is there a way to download option prices? I trade many options, and I can't figure out a way to keep my positions current. Thanks.
    Saturday, July 7, 2012 5:50 PM
  • hleofxquotes (https://code.google.com/p/hle-ofx-quotes/) has a quote
    source that will let you fetch options price from Yahoo. Issues that
    will face:

    1. Where to get update prices (https://code.google.com/p/hle-ofx-quotes/)
    2. How
      to name the symbols since MsMoney has internal limit on long name: you
      will have to manage the name yourself and use mapper.csv file to map
      between quote source symbols and MsMoney symbols

    See also:

      http://finance.groups.yahoo.com/group/OptionClub/message/22045

    Monday, July 9, 2012 2:37 PM