Various errors and workarounds

[dfarnand]

Hi! I just wanted to start off by saying thank you for writing this! I had the fairly heavy task of importing almost six years of transactions from classic YNAB to Firefly, and despite the issues I'll be listing, this script simplified the task immensely!

That being said, I had some issues come up in the process. I was able to work through them all, but I wanted to post both how I solved them for other users, and also for the possibility of solutions in the code.

1. I was getting an error with the Pre-YNAB Debt Categories (possibly because I had them all hidden?)

AssertionError: Unable to process transaction with unknown budget: |Visa (Citibank) (hidden)|
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in <module>
----> 1 import_transactions(
      2     config_file="config.toml",
      3     register_file = "register.csv",
      4     budget_file = "budget.csv",
      5     dry_run = True,

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in import_transactions(config_file, register_file, budget_file, dry_run, firefly_url, firefly_access_token, filter_min_date, filter_max_date)
     27         filter_max_date=filter_max_date,
     28     )
---> 29     importer.run(dry_run)
     30 

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in run(self, dry_run)
    556         self._process_budgets()
    557         self._process_accounts()
--> 558         self._process_transactions()
    559         if not dry_run:
    560             self._load_cache()

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in _process_transactions(self)
    808                 budget = self._budget(tx)
    809                 if budget:
--> 810                     assert budget in self.data.budgets, f"Unable to process transaction with unknown budget: |{budget}|"
    811                     # ignore hidden categories
    812                     if self.data.budgets[budget].active:

AssertionError: Unable to process transaction with unknown budget: |Visa (Citibank) (hidden)|

I fixed this by a) manually removing initial balance items from the register csv, and b) changing the lines with other transactions that were assigned to the category (mainly interest that was being charged the to pre-ynab debt balance) to be a misc. transaction.

2. Creation of the budget categories didn't work (possibly because of a change to the API?)

{'message': 'The given data was invalid.', 'errors': {'auto_budget_amount': ['The amount is required.']}}
HTTPError: 422 Client Error: Unprocessable Entity for url: <server redacted>/api/v1/budgets
---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in <module>
----> 1 import_transactions(
      2     config_file="config.toml",
      3     register_file="register.csv",
      4     budget_file="budget.csv",
      5     dry_run=False,

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in import_transactions(config_file, register_file, budget_file, dry_run, firefly_url, firefly_access_token, filter_min_date, filter_max_date)
   1356         filter_max_date=filter_max_date,
   1357     )
-> 1358     importer.run(dry_run)
   1359 # if __name__ == "__main__":
   1360 # cli()

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in run(self, dry_run)
    568             self._create_currencies()
    569             self._create_categories()
--> 570             self._create_budgets()
    571             self._create_budget_limits()
    572             self._create_available_budgets()

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in _create_budgets(self)
    988                         self.firefly_data.budgets[budget.name] = response.json()["data"]
    989                 else:
--> 990                     response = self._session.post("/api/v1/budgets", json=data)
    991                     self.firefly_data.budgets[budget.name] = response.json()["data"]
    992             except requests.HTTPError as e:

~/.local/share/virtualenvs/Firefly-YNAB4-Importer-br1XYdXy/lib/python3.8/site-packages/requests/sessions.py in post(self, url, data, json, **kwargs)
    588         """
    589 
--> 590         return self.request('POST', url, data=data, json=json, **kwargs)
    591 
    592     def put(self, url, data=None, **kwargs):

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in request(self, method, url, **kwargs)
    511                 print(response.json())
    512                 print(f"{kwargs=}")
--> 513             response.raise_for_status()
    514         return response
    515     def get_all_pages(self, url, params=None, **kwargs) -> dict:

~/.local/share/virtualenvs/Firefly-YNAB4-Importer-br1XYdXy/lib/python3.8/site-packages/requests/models.py in raise_for_status(self)
    951 
    952         if http_error_msg:
--> 953             raise HTTPError(http_error_msg, response=self)
    954 
    955     def close(self):

HTTPError: 422 Client Error: Unprocessable Entity for url: <server redacted>/api/v1/budgets

I manually created all the budget categories in firefly to get around this error.

3. The function to create transactions is commented out by default (line 673 in main.py). Is this on purpose for some reason?

Fixed by uncommenting that line.

4. I had the same error reported in #2

This seems to occur on a subsequent execution when the previous attempt crashed. I fixed it by deleting the cache json before running (as mentioned in the issue)

5. I had some issues with special characters in the Payee column (e.g. é in café)

I fixed this by changing them to their ascii equivilent in both the csv and and the expense accounts page in Firefly.

Other Notes

Just a few things I jotted down while working through this that I think would be nice-to-haves:

• The config template has some values that are different from the default. I found this challenging because I wondered which would be the "best" setting for most users. I think the default should be something that a new user can always try first, either with no config file or with the template.
• Some sort of verbose mode that lists transactions and other info as its processing them would be very helpful to figure out where its breaking (I ended up needing to drop print statements into spots in the code to figure out the issues I had).