Support assertions which include subaccounts

[mathstuf]

I have the current accounts for each credit card:

credit:bank:card
credit:bank:card:bill

I would like to assert on the outstanding balance on each card, but there is no way to take the amount paid in bills to the card when asserting on its balance.

Is there a better way to structure the accounts? I'd really like a separate one for the bills to go to so I can see how much I've paid over time, but if there's some other way to do so with a merged account, that'd be fine too.

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[simonmichael]

I agree we should have some way to assert on the subaccount-inclusive balance. (#195 is another balance assertions wish.)

I'm not familiar with how you're using the subaccount here. Can't you easily see what you've paid over time with a report ?

[simonmichael]

PS for example, I'm thinking of something like:

hledger register credit:bank:card amt:'<0'
hledger register credit:bank:card amt:'<0' -Q
hledger balance  credit:bank:card amt:'<0' -MA

[mathstuf]

On Sun, Oct 18, 2015 at 16:53:14 -0700, Simon Michael wrote:

I agree we should have some way to assert on the subaccount-inclusive
balance. (#195 is another balance assertions wish.)

I'm not familiar with how you're using the subaccount here. Can't you
easily see what you've paid over time with a report ?

As for how the accounts are used:

credit line -> card
card -> purchase
card -> purchase
card -> purchase
card -> purchase
credit line increase -> card

which all works fine. I also need an account to pay off bills. My
thought was to use a subaccount so that I can see it individually, but I
guess I could just use queries to get this information instead. Playing
around with it, it seems to be sufficient.

Setting the card == bill account and instead splitting the credit line
into a subaccount lets me assert on values matching the bills.

[mathstuf]

On Sun, Oct 18, 2015 at 20:56:43 -0400, Ben Boeckel wrote:

Setting the card == bill account and instead splitting the credit line
into a subaccount lets me assert on values matching the bills.

Maybe adding such a layout should be put into some kind of
documentation? I'd also be interested in recommended layouts for things
like refunds (where I use a :refund subaccount), 401k tracking with
shares, and other common patterns. For example, I'm now thinking that
refunds shouls use the same account, but managed using queries.

For example, here is how I handle increases in pledge values on
Kickstarter:

04-23=03-22 * Item - Kickstarter
    account for item        $99.00
    source account

04-23=04-04 * Item boost - Kickstarter
    account for item       $199.00
    account for item       $-99.00
    source account

Where the transaction date is that of the pledge or increase and the
clear date is when the kickstarter ended.

And then if the Kickstarter fails:

04-23 * Item failure - Kickstarter
    account for item      $-199.00
    source account

Having all the clear dates be the same ensures that no assertion is
problematic, but I can also accurately budget before it ends.

[simonmichael]

Maybe adding such a layout should be put into some kind of documentation?

Yes please! (But where ? Too big a topic to discuss here, but a good h/ledger cookbook is needed).

[mathstuf]

Well, some place on the website would be nice, but I think soliciting advice from the h/ledger lists on how others approach it would be a first step.

[simonmichael]

Back to the topic at hand - if you come up with a design for this, let me know. I assume I can't just make it work that way by default, because it would cause unexpected breakage and confusion when switching between Ledger and hledger.

[mathstuf]

Maybe use == or some other (currently invalid) token? Maybe =~ would work too, but maybe that is too perl/shell-ish.

[simonmichael]

I was going to use == for "all commodities, not just one" - how could we cover all four of these cases (or should we ?)

[simonmichael]

Well, answering myself, you could do the obvious:

=  - one commodity, exclude subaccounts (Ledger default)
==  - all commodities, exclude subaccounts
===  - one commodity, include subaccounts
====  - all commodities, include subaccounts

I think not, too hard to remember.

[mathstuf]

I don't think multiple commodities is needed; can't you just have multiple assertion transactions?

[simonmichael]

We need a way to say that no other commodities are present, too.

[ony]

@simonmichael, I'd agree with @mathstuf that attempt to assert total amount of multi-currency account is nearly impossible without specifying rates of conversion in the same posting.
Though scenario when you put all prices right before a transaction and never split this pair is still valid.
Thus I would expect that most popular usages would be = and ===. Re-ordering this scheme according to an estimation of usage might be useful.
I.e. consider using == for single currency including subaccounts.

[simonmichael]

That makes sense, thanks for the comment. Asserting a multicommodity zero (account is completely empty) might still be a reason to support ====.

Would a * suffix for "include subaccounts" be more mnemonic ? = =* == ==*

[ony]

@simonmichael, agree *= for sub-accounts, =* for multi-currency, and *=* for both might be more mnemonic.

Note that I'm not sure how to represent right side of multi-currency assertion. I guess it is safe to have zero assertions, but whenever we refer to other value we have to deal with commodities exchange rates. As well multi-currency assertions cannot produce implicit flow.

2017/12/01 selling some commodities
    assets:bank:investment  -1 TSLA
    assets:bank:investment  -2 GOOG
    assets:bank:current  $203 @ *

2017/12/31 closing portfolio
    assets:bank:investment  =* 0
    assets:bank:current  $203 @ *

I just came back to issue with sub-accounts again when I was touching one of my accounts balances:

2017/01/01
    assets:bank:compte vert  = €1000
    assets:bank:current

2017/01/01 rent deposit
    assets:bank:compte vert  €-600
    assets:bank:compte vert:hold

2017/12/31 interest accrual report
    assets:bank:compte vert  *= €1003
    expenses:taxes:income  €1
    income:interest

Worth to mention that with such syntax there is no way to specify different sub-account as a recipient of balance difference. Though I have no such cases I can imagine them:

2018/01/02 balance report
    {assets:bank}:current  = €4004
    expenses:taxes:income  ; tax return

which is effectively

2018/01/02 balance report
    assets:bank:current  €1
    assets:bank   €0 = €4004
    expenses:taxes:income  €-1  ; tax return

Though I'd expect that initial implementation should only allow assertion without implicit flow generation.

P.S. Actually other way to represent "hold" on account which I use right now is to add sub-account with negative balance and in tree-view of balance see it as available funds:

2017/01/01 rent deposit
    assets:bank:compte vert:hold  €-600
    liabilities  ; payee: landlord

2017/12/31 interest accrual report
    assets:bank:compte vert  = €1003
    expenses:taxes:income  €1
    income:interest

Updated: I actually assert with positive balance

[mathstuf]

P.S. Actually other way to represent "hold" on account which I use right now is to add sub-account with negative balance and in tree-view of balance see it as available funds:

This is similar to the way I've done credit lines (my original use case for multi-account assertions):

01-01 Credit card issued
    acct:for:credit:card:credit   $1000.00
    null:credit

01-02 Use credit card
    personal:gifts       $100.00
    acct:for:credit:card

02-01 Statement for credit card
    [acct:for:credit:card]    = $-100.00

[mildred]

Note that ledger-cli supports them using the global assert directive:

assert account("assets").total == 100

except that they don't work well (ledger/ledger#1679). If correct syntax is difficult to come by, especially if we want to be compatible with ledger-cli, perhaps an assert directive could be used.

[simonmichael]

That's a dateless assertion, entirely dependent on parse order; I don't think we want those.

[mildred]

here:

hledger/hledger-lib/Hledger/Data/Journal.hs

Line 537 in 2960c92

actualbal = fromMaybe nullamt $ find ((== assertedcomm) . acommodity) (amounts amt)

[mildred]

parsing here:

hledger/hledger-lib/Hledger/Read/JournalReader.hs

Line 559 in 3de8c11

massertion <- partialbalanceassertionp

[mildred]

About the syntax, could it be possible to represent multi currencies assertions by specifying the different currencies on the right hand side? Something like that:

2018-10-01 Bankins assertion
  banking  = $100.00 + 10.00 EUR + 0

Here we want to assert that the banking account has $100, 100€, and nothing more. A sign like / could be used too to indicate the end of amounts and that we want a full assertion over all currencies.

Then, to assert over sub-accounts as well, we could use another operator like *= or == as it has been suggested, to signify we want to recurse to sub-accounts.

[simonmichael]

@mildred yes, #871 is exploring this.

[mildred]

Something like that for the parser...

diff hledger-lib/Hledger/Read/Common.hs hledger-lib/Hledger/Read/Common.hs
 balanceassertionp :: JournalParser m BalanceAssertion
 balanceassertionp = do
   sourcepos <- genericSourcePos <$> lift getSourcePos
   char '='
   exact <- optional $ try $ char '='
+  recurse <- optional $ try $ char '*'
   lift (skipMany spacenonewline)
   a <- amountp <?> "amount (for a balance assertion or assignment)" -- XXX should restrict to a simple amount
   return BalanceAssertion
     { baamount = a
+    , barecurse = isJust recurse
     , baexact = isJust exact
     , baposition = sourcepos
     }

[simonmichael]

#934 is the latest discussion on that.

[simonmichael]

@mildred I mean, #934 is a proposal to allow writing multi-commodity amounts in assertions and elsewhere. (If you have a real need for this feature, do write about it there.)

Regarding this issue #290, I have lost the context. Maybe someone can review it and make a recommendation on whether we need subaccount-inclusive assertions and how they should look and work.

[simonmichael]

Reminding myself.. here are some distinct, interrelated balance assertion issues under discussion:

• multicommodity balance assertions, ie writing multiple amounts separated by + to assert a multicommodity balance in a single assertion (Amount expressions, multi-commodity postings #934). In current hledger you can assert a multicommodity balance by writing multiple postings/assertions. But in either case, the balance might contain additional unasserted commodities. To disallow that you need...

• total balance assertions, ie asserting that the balance is as written, with no extra commodities in the account. Our current syntax for this is == (exact (multicommodity, total) assertions #902). I sometimes wish this was the default behaviour, of =.

• subaccount-inclusive assertions, ie asserting the balance of an account including all its subaccounts' balances (Support assertions which include subaccounts #290).

[for completeness: we also have]

• partial assertions, ie the traditional Ledger-style single-commodity, subaccount-exclusive, non-total assertions (=).

[simonmichael]

Inviting list discussion

[mildred]

Sorry, my comments might seem a little difficult to understand. I'm tracking this issue because I have a real need for this. Unfortunately, Iḿ not well versed into Haskell. if I was, I would have made a pull request long ago. So until someone works on this, I'm looking at bits and pieces to decipher the code to eventually make a contribution.

I need this feature to ensure that my ledger file is consistent with the amount shown on my bank account. Until then, I make manual checks, when I find the time to, and when I do, I often find surprises... Basically, I have my accounts organized like this:

• bank
  • bank:current
  • bank:savings
  • bank:provisions
    • bank:provisions:annual taxes
    • bank:provisions:insurances
    • bank:provisions:water
    • bank:provisions:electricity
• cash
  • cash:savings
  • cash:provisions
    • cash:provisions:car

But in reality, I have only one bank account. I'd like to only have a single place where I store cash too, but there I can arrange for different physical envelopes. I need to make sure that each month, the amount shown on my bank account matches all the ledger accounts under bank:. I'd like to be able to write:

2018-02-01 February balance assertion
    bank  0 =* xxx EUR
    cash  0 =* xxx EUR

In the future, I'd also like to reorganize accounts to have a single provisions hierarchy, and different commodities for cash and virtual money on the bank account, so I also need recursive multi commodity assertions. Write something like:

2018-02-01 February balance assertion
    provisions  0 =* xxx BankEUR
    provisions  0 =* xxx CashEUR

[simonmichael]

So until someone works on this, I'm looking at bits and pieces to decipher the code to eventually make a contribution.

That's appreciated! Feel free to ask for more help on #hledger. Also thanks for the extra details, which make sense. I wonder why I haven't felt this need myself ? I have ~300 assertions in 2018, excluding opening/closing transactions. It seems my assertions tend to be on accounts which don't have subaccounts. And when reconciling, I tend to just do a one time check of the inclusive account balances in hledger-ui. I don't usually record that as an assertion.

[simonmichael]

Making some progress on this: #974

[simonmichael]

Now in master. https://groups.google.com/d/msg/hledger/47F-ZiajbMU/9edMGm2OBAAJ, http://hledger.org/journal.html#assertions-and-subaccounts

=*  AMT   ; partial balance assertion including subaccounts
==* AMT   ; total balance assertion including subaccounts

[mildred]

Thank you a lot !

[ony]

Ehh... I hoped * to be associated with what part is aggregated. I.e.
*= (closer to account) - all sub-accounts
=* (closer to amount) - all currencies
*=* (on both sides) - all sub-accounts, all currencies.

Anyway thank you. I also wanted this feature. I think I still can associate difference between = and == to be similar to == and === in JavaScript (type aware equality).

[simonmichael]

Hmm. I don't love what we have, but everything else so far, including this, seems a bit harder to remember or intuit.