Making a PocketSense scrubber remove certain transactions. (specialized topic) RRS feed

  • General discussion

  • PocketSense will read transactions as an OFX file for accounts held at some financial institutions, and will feed them to Microsoft Money. This is mainly for some US brokers and credit cards.

    I made a function that can be used in scrubber.py that removes certain  transactions that you can specify. There is a data structure that contains pairs of tags and qualifying content.

    For example, you could list BUYMF, 1234567898 as an item. Then all  <BUYMF>..... </BUYMF> transactions that contain 1234567898 (typically a CUSIP) would be removed. Where could this come up? Normally you expect your sweep money market not have the buys and sells in the OFX file. If they start showing up sometimes, you might want them to be ignored. There could be other cases. The content test is pretty flexible. The important thing is that only the specified types of transactions are tested and possibly removed.

    The function could be in a different file, so the scrubber would have an appropriate include, or the function could be added to scrubber.py.

    This function also contains a line to add some newlines if your ofx file is run together. That improves human readability.

    I know this is of limited interest, but if you are interested, I could post the code.

    Wednesday, August 7, 2019 7:08 PM

All replies

  • Just so you don't think anybody is interested Cal, I have added scrubber snippets that have been created by others over the years on an as-needed basis, but they get overwritten as Pocketsense is updated, usually with Robert's version of the fix.   It is probably best to simply share your code with Robert, so it officially makes it's way into the scripts.
    Friday, August 9, 2019 4:00 PM
  • If the released scrubber.py could run a function from a file if a particular filename exists, that would let you have customized scrubber routines that would not require modifying scrubber.py when a new release.

    I just run my own scrubber that has things that may be of limited interest, and in the case of the dates, I think works better than the time offset. I keep a copy in case I inadvertently overwrote mine when I download a new version of PocketSense. For one thing, it knows that new quotes don't happen on Saturday or Sunday. But it is more complex and harder to follow probably.

    The symptom for a date problem is that the closing prices from the OFX file can get stored with the wrong date.

    This particular routine that I just wrote has some (easy to modify in a table declaration) CUSIPs that correspond to my sweep fund. Somebody else may have a different sweep fund. However if that code were incorporated into scrubber.py, a new version would overwrite that info. I guess it is possible that there could be some new entries in sites.dat descriptors that would accept replacement CUSIPS list, or in this case, lists of criteria for transactions that should be removed. However the demand for these functions is clearly very limited, so I don't see that happening.

    Friday, August 9, 2019 8:15 PM