Sensors not showing up

[mr-p666]

As with @tofuSCHNITZEL's implementation, there seem to be issues if you restart it at a time when there are no departures.

[jozefKruszynski]

You mean if there are no departures then the sensors aren't created?
Some time next year I might poke around at a few other api endpoints and/or try to make some sense of the gtfs data.

[mr-p666]

Exactly.
I had to restart my HA instance at 2am this morning and after that no departures were shown until I restarted it again ~30 minutes ago.
It would be great if we could see some progress on your integration soon!

[jozefKruszynski]

Leaving this link here to remind me when I pick this up.

https://developers.home-assistant.io/docs/core/entity/sensor/?_highlight=sensor#restoring-sensor-states

[tofuSCHNITZEL]

Yes this is something I struggled with when thinking about redoing the integration...
the problem is that if you add a stopid the lines that stop there might not be in the data returned from the api depending on the time of day. sometimes its more than one line, sometimes its no line - sometimes its a completely different line that usual (e.g. nightline)
this makes it equaly difficult to detect if an entered stopid is even valid... might be that its invalid or just at a time of they where nothing is stopping there (the api always returns "ok"
so I think what sensors are added to HA cannot be determined on the time the stop is added alone.. the integration probably needs to add (but probably should not remove) sensors dynamically - if new / different lines appear for a stop ID the sensor need to be added or set to "unknown" - but this in turn means if you setup the integration at a time of day where there are no stops you will always receive "nothing" and need to wait until the lines you expect there are actually stopping there (= appearing in the api)

@jozefKruszynski:
my current plan would be to add a drop down selection menu (with search function) to the Config flow where you select the Stop and the Line (and direction) when adding the monitor - this would be to ensure that the stopid is the correct one - and if there is no data available we can at least add the sensor but with "currently unknown" value... (and in the background ha can periodically retry to fetch the data for the sensor)
this selection would be "fed" by the CSV files found here:
https://www.data.gv.at/katalog/dataset/wiener-linien-echtzeitdaten-via-datendrehscheibe-wien/resource/45f06281-5a9f-441c-9977-0fda610f963a#resources

the tricky part would be I guess to consolidate multiple stop/line/direction sensors that would be fetchable through the same stopid so we dont make unnecessary extra calls to the api

if you would be interested we could do a little meetup in vienna if you have time over the holidays

[tofuSCHNITZEL]

quick update @jozefKruszynski:
I'm currently working on the modified flow that lets you select from a list... I think I have the basics down but the issue is with the data from wienerlinen... because I need to fetch three different csv files (2mb in sum) and iterate through them to compile a usable list of stops - I'm not sure this is feasible... I'm not even sure if wienerlinien is keeping these csvs up to date...
I was thinking... maybe the first step would be to create a python package for interacting with the wienerlinien api - so basically move all the businesslogic that you implemented directly here in the "ha code" into a seperate lib - which is also the mandatory way for core ha integrations... and maybe this ext. library has the stop data "backed" in and would just need to be updated if WL updates their list of stops etc... but fetching all these CSVs every time you want to add a stop, I realised, is madness...

I was just now brainstorming on how to solve this and thought about creating a github repo that fetches and creates an sqllite file or a json file automatically and refreshes it every day and hosts it on github pages so it can be fetched easily...
when researching I noticed that someone else also realised this madness and build a (kind of) solution:
https://www.data.gv.at/katalog/application/a7d8dd6a-f42b-478c-b296-effabefccec6
https://github.com/hactar/WL-Generator

[jozefKruszynski]

I've also been working on exactly the same thing 🤣 as in the stops.
The csvs are in theory updated several times a day, as the timestamps of the version change throughout the day.
As we would only need those stops during the config flow, I don't see a problem with their size.

I agree with the idea of the external lib for all the api interaction.

And I also stumbled upon that same wlgenerator code which led me eventually to the source of those CSV files.

Looks like you and I were on the same research path over the last coue of weeks. 🤣

[tofuSCHNITZEL]

well, then lets pool our (time) ressources - it would be a pity if we work on the same thing! No pressure or deadlines or anything, this is something we do in our spare (free) time - I get it.
maybe we can also get @chief-nerd on board. Would you be up for a call or meetup?

[jozefKruszynski]

well, then lets pool our (time) ressources - it would be a pity if we work on the same thing! No pressure or deadlines or anything, this is something we do in our spare (free) time - I get it. maybe we can also get @chief-nerd on board. Would you be up for a call or meetup?

In all honesty, I'm not sure that I have the time to put into this. I would happily hand it back to you as I have a lot going on with work and with Music Assistant.
I originally picked this up to see if I could come up with some ideas. I don't think I want to get too deep into another project.

[tofuSCHNITZEL]

Yes I totally understand - I also have my normal company work going on - and what you did until now is amazing and exactly what needed to be done to get this integration back on track - I just have the feeling I need someone that also understands Home Assistant dev to discuss ideas and frustrations ;)