Fix Schwab parsing

[amaramrahul]

Updated parser logic for Schwab

Fixes #451
Fixes #457
Fixes #464

[vmartinv]

Can you try again? I downloaded my transactions and it looks like Quantity and Price columns are swapped.

[amaramrahul]

Can you try again? I downloaded my transactions and it looks like Quantity and Price columns are swapped.

Ah yes. You are right. Not sure if this is temporary or permanent. I guess the proper fix would be to determine the column purpose based on the header.

[vmartinv]

Ah yes. You are right. Not sure if this is temporary or permanent. I guess the proper fix would be to determine the column purpose based on the header.

I agree. If you add it I can test it on my statement. You can do something like this:

    for upper_row, lower_row in zip(lines[::2], lines[1::2]):
        # in this format each row is split into two rows, so we combine them safely below
        row = []
        for upper_col, lower_col in zip(upper_row, lower_row):
            assert upper_col == "" or lower_col == ""
            row.append(upper_col + lower_col)

        if len(row) != len(header):
            raise UnexpectedColumnCountError(
                row, len(header), schwab_award_transactions_file or ""
            )

        row = dict(zip(header, row))
        date_str = row["Date"]

[amaramrahul]

@vmartinv I have made the changes and tested using Individual Transaction History and Equity Awad Transaction History file. Please test and if you have permission, merge the changes.

[vmartinv]

I think it looks good but can you also add a validation to make sure the columns are as expected? Something like this:

    expected_header = {
        "Date",
        "Action",
        "Symbol",
        "Description",
        "Quantity",
        "FeesAndCommissions",
        "DisbursementElection",
        "Amount",
        "AwardDate",
        "AwardId",
        "FairMarketValuePrice",
        "SalePrice",
        "SharesSoldWithheldForTaxes",
        "NetSharesDeposited",
        "Taxes",
    }
    header = lines[0]
    if set(header) != expected_header:
        raise ParsingError(
            schwab_award_transactions_file,
            f"Missing or unknown columns in awards file: {expected_header.symmetric_difference(header)}",
        )

Otherwise a format change might go unnoticed.

[amaramrahul]

I did not do this because I didn't want the script to break if an additional column is added that we do not use. However, I also see the reason why we need to check for headers. So, I have taken an intermediate ground and updated the code to ensure that the headers that we are using are present in the transactions file. We do not care if there are additional headers in the transactions file that we are not using.

[vmartinv]

Sounds good to me. I can confirm this PR works as expected, it parsed my latest Schwab files and generated the report correctly. Can you confirm @KapJI ?

[KapJI]

Looks good, thank you!

And thanks @vmartinv for help with reviewing this.