Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JLR blocked API access again (Unable to connect to the JLRIncontrol servers) #143

Open
JonSilver opened this issue May 17, 2024 · 121 comments
Open

Comments

@JonSilver
Copy link

JLR app is fine, the HA integration isn't, since midnight today.

@msp1974
Copy link
Owner

msp1974 commented May 17, 2024

Adding @ardevd to this issue. I see the same with a 407 Proxy Auth Required error.

@alainsch
Copy link

me too. Same in wattcat

@BenjiBoisemont
Copy link

BenjiBoisemont commented May 17, 2024

The same for me, connection lost with jlr last night.
MyPace app also broken on iPhone

@ardevd
Copy link

ardevd commented May 17, 2024

I'm on it.

@tdipilot01
Copy link

My Android Jaguar Remote app works, but HA integration's been down all day.

@ardevd
Copy link

ardevd commented May 18, 2024

It's going to take some time to figure out the new update but I think it's doable.

@krenlund
Copy link

Phew, this is a relief! Thank you @ardevd in advance!

@ardevd
Copy link

ardevd commented May 18, 2024

Phew, this is a relief! Thank you @ardevd in advance!

No promises! :) But I'll try!

@Jazmodo
Copy link

Jazmodo commented May 18, 2024

Apologies if this has already been answered, and many many thanks for getting this working at all!

I just wondered if the changes by JLR were simply updates that are breaking the integration? Or are JLR purposefully breaking it?

If the latter, the I’d be happy to start harassing JLR to stop doing that. I’m sure many JLR drivers find this (and other) 3rd party apps useful, and they need to realise they’re at risk of losing future business if they keep a walled-garden approach.

@callumeveratt
Copy link

Apologies if this has already been answered, and many many thanks for getting this working at all!

I just wondered if the changes by JLR were simply updates that are breaking the integration? Or are JLR purposefully breaking it?

If the latter, the I’d be happy to start harassing JLR to stop doing that. I’m sure many JLR drivers find this (and other) 3rd party apps useful, and they need to realise they’re at risk of losing future business if they keep a walled-garden approach.

It's the latter - they've taken a stance against third party apps recently.

@pmharris77
Copy link

pmharris77 commented May 18, 2024

Apologies if this has already been answered, and many many thanks for getting this working at all!

I just wondered if the changes by JLR were simply updates that are breaking the integration? Or are JLR purposefully breaking it?

If the latter, the I’d be happy to start harassing JLR to stop doing that. I’m sure many JLR drivers find this (and other) 3rd party apps useful, and they need to realise they’re at risk of losing future business if they keep a walled-garden approach.

Same question asked here on jlrpy repo (the API library this integration uses to talk to JLR servers)
ardevd/jlrpy#131 (comment)

They are doing it on purpose in a game of cat and mouse with third party apps.

Looks like they've added an extra layer of authentication this time according to @ardevd. It's annoying as I find the jlrpy library and this integration incredibly useful, but I guess I can see that JLR don't want potentially malicious actors hacking JLR cars using their API, especially with the recent Range Rover security scandal...

@pmharris77
Copy link

FYI, for those interested in the underlying issue within the jlrpy Python library, see here: ardevd/jlrpy#131

@ardevd
Copy link

ardevd commented May 18, 2024

I'd argue that

a) you can't really lock out third party API access, you can just make it tedious for people to access it.

b) locking out API access is the wrong solution to a problem that doesn't exist.

JLR could easily put restrictions on sensitive remote operations. If they wanted to prevent malicious remote unlocks they could make the double lock disable that feature for example.

I doubt any Land Rovers have been stolen through "malicious" third party API access.

Furthermore, the way JLR have consistently undermined years of community effort by arbitrarily changing their APIs without notice or explanation is incredibly hostile.

@pmharris77
Copy link

I'd argue that

a) you can't really lock out third party API access, you can just make it tedious for people to access it.

b) locking out API access is the wrong solution to a problem that doesn't exist.

JLR could easily put restrictions on sensitive remote operations. If they wanted to prevent malicious remote unlocks they could make the double lock disable that feature for example.

I doubt any Land Rovers have been stolen through "malicious" third party API access.

Furthermore, the way JLR have consistently undermined years of community effort by arbitrarily changing their APIs without notice or explanation is incredibly hostile.

@ardevd I agree. You can't drive a JLR away without the key being present. At most you can unlock the car and could steal stuff inside, but you'd have to know the car owner's app credentials, which would only be possible if they had a data breach leaking car reg or VIN mapped to user account details...

I wonder what other manufacturer's approach to third party API access has been?

@ismarslomic
Copy link
Collaborator

I agree that this approach from JLR does not improve the overall security, in regards to third party usage.

But from my understanding, the main issue in this context is that the JLR API we are using is not designed nor handled as third party API. Its basically private API designed for JLR app(s), which they have full control of and can handle any breaking changes in the API. The API is not documented, nor supported for third parties by JLR. We basically did reverse engineering to understand how it works and how to make use of it. So the main problem is that JLR does not provide any official public API with close collaboration with the open source community.

In that meaning, we should not be frustrated over breaking changes in the private API, but rather that JLR ignores third party apps and innovative usage.

@ardevd
Copy link

ardevd commented May 18, 2024

JLR can do what they want with their private APIs. My argument is simply that it's incredibly dumb to agitate your most enthusiastic customers by cutting off community tools that have been developed and maintained over several years.

@ismarslomic
Copy link
Collaborator

Yeah, agreed to that!

@Jazmodo
Copy link

Jazmodo commented May 18, 2024

Thanks for the replies @ardevd 🙏 && I completely agree.

As you say, cutting off access to some of their more enthusiastic owners seems counter-productive. I love the ability to integrate Car charging/location features into Home Assistant, fantastic you've managed to get it working at all.

Not sure I can help much, but I can be a small but persistent voice, if nothing else.

FYI, for those interested in the underlying issue within the jlrpy Python library, see here: ardevd/jlrpy#131

I'll check this out! Cheers 😃

@msp1974
Copy link
Owner

msp1974 commented May 18, 2024

To add to both @ismarslomic and @ardevd, most other manufacruters also have 'private' and undocumented apis but do not forceably try and stop 3rd party apps like this. If you just look at the list in HA for different makes, there are many.

I feel their approach is both narrow minded and old fashioned. We can also note that they have blocked Octopus Energy from using it too with their energy product designed to allow cheaper charging of EV and PHEV cars, thus meaning JLR customers will pay more!

I think we need to find some route into JLR to have a grown up discussion as to how we can utilise this api to the benefit of their paying customers, which we all are.

As a note, just had to pay for my Incontrol service for the first time @ £280 for the year!

@ismarslomic
Copy link
Collaborator

Agree with you @msp1974! We should initiate dialog with JLR. They already know that their API is used by many third parties, so it should not be an surprise that this approach does not work for anyone. Im using Tibber as energy provider in Norway, and they also use the same API to connect to the vehicle in order to do smart charging when the electricity is cheap. This service is currently broken too, and I need to manually start charging when the price is cheap.

Does anyone know a name of the product owner or developer of the JLR API? I think we need to reach out directly and not through the customer support.

@pmharris77
Copy link

Considering they are about to exclusively sell electric cars, blocking third party API access, whilst not even providing any official integrations themselves, is just madness and is just going to make current and future customers go to their competitors. Way to go bust JLR! 😅

@ardevd
Copy link

ardevd commented May 18, 2024

@ismarslomic all my previous JLR contacts with positions related to connected car tech have left the company. I haven't heard from JLR reps for years.

@adec
Copy link

adec commented May 18, 2024

Until recently I used to work for JLR but now work for one of their suppliers. I'll see if I can find a way to raise this internally with them. I can't promise anything but I will do what I can. I'd quite like to use the integration myself but would be more comfortable if the public API allowed some of the more risky options to be disabled (such as opening the car doors).

@ardevd
Copy link

ardevd commented May 18, 2024

Until recently I used to work for JLR but now work for one of their suppliers. I'll see if I can find a way to raise this internally with them. I can't promise anything but I will do what I can. I'd quite like to use the integration myself but would be more comfortable if the public API allowed some of the more risky options to be disabled (such as opening the car doors).

That would be awesome!

Personally, I don't think remote car lock/unlock should be a thing. But a way to opt out would be nice.

@pmharris77
Copy link

Been doing a lot of reading about the whole Octopus smart charging saga with API access being v suddenly removed for energy companies back in March. There has been a very public backlash on X, and someone mentioned that JLR stated that they were working on improving the API security. Given the amount of complaints that both Octopus and JLR have had about removing support, it would be difficult to imagine they aren't talking about how to resolve the issue.

It got me thinking, maybe the extra app secret layer added to the API authentication is being introduced as a way to allow JLR-approved third party apps, such as Octopus, to access to the API in the future with a JLR-issued app secret? If so, this might actually be a positive sign, if they are willing to engage with third-party app developers. I can live in hope 🤞

@pmharris77
Copy link

Some people have gotten the following response from JLR recently, which does state they are "working to develop authorised solutions to smart home and energy services as quickly as possible"

Full response:

“We are responsible for the performance and safety of our vehicles and provide a warranty for this. We are not able to test, approve and warrant the safety and security of unauthorised third-party access to our vehicles and its data. This is a permanent change and only approved, authorised and tested apps will have access to our vehicles and their data. We are working to develop authorised solutions to smart home and energy services as quickly as possible. We apologise for any inconvenience this may have caused, but hope you understand why this was necessary.”

@ardevd
Copy link

ardevd commented May 19, 2024

Been doing a lot of reading about the whole Octopus smart charging saga with API access being v suddenly removed for energy companies back in March. There has been a very public backlash on X, and someone mentioned that JLR stated that they were working on improving the API security. Given the amount of complaints that both Octopus and JLR have had about removing support, it would be difficult to imagine they aren't talking about how to resolve the issue.

It got me thinking, maybe the extra app secret layer added to the API authentication is being introduced as a way to allow JLR-approved third party apps, such as Octopus, to access to the API in the future with a JLR-issued app secret? If so, this might actually be a positive sign, if they are willing to engage with third-party app developers. I can live in hope 🤞

It's possible but so far JLR has shown exceptionally little interest in dialog with third party developers. Maybe Octopus would be an exception, but their equivalent here in Norway, Tibber, have had no contact. I've reached out to JLR repeatedly and they have never responded.

@CorceiroPT
Copy link

I would say this time will be more difficult. It could be that they (JLR) used a proxy with different credentials (and maybe temporary rotating passwords for it) to access and validate the API keys. This would be a way to block third-party access to it and I supposed they did it...

@ardevd
Copy link

ardevd commented May 19, 2024

From my findings so far it seems like the only change here is that the app secret now rotates (at least) every time you start the app. It seems to be happening offline from what I can tell.

@ismarslomic
Copy link
Collaborator

Is it possible to replicate the code for code generation in jlrpy, if the generation happens on client side (app)?

@onorbe
Copy link

onorbe commented Nov 18, 2024

The software integration of Dimo ​​with incontrol now works. Will this help us integrate incontrol with Homeassistant?

@msp1974
Copy link
Owner

msp1974 commented Nov 18, 2024

@ardevd and I have been working on a Dimo integration and yes, it will allow HA to have access to JLR vehicle data. There are a few bugs that need fixing on the Dimo side and instructions of how to set it up needing to be written before this can be released. But we expect soon (like as in hopefully soon but we are relying on others to fix stuff).

ATM though, control maybe limited to lock/unlock only but Dimo are progressing features all the time so who knows what the future may bring.

@ardevd
Copy link

ardevd commented Nov 18, 2024

Added benefit here being that we'll have a solution that's not specific to the I-Pace or JLR but to many vehicle models out there. Hopefully we also get more and better data than what the InControl API offers.

The onboarding process isn't the smoothest currently but will improve in the future.

More details to come shortly! With @msp1974 we've made a pretty cool integration and I'm excited for it to be released!

@CorceiroPT
Copy link

As of today, I now have access to some InControl data in my DIMO application, including the SoC. At the moment, to me, the SoC is the only thing that would really be essential to have in Home Assistant (to control de charging).
By creating an association between the DIMO itself and Home Assistant, It could possible to get some info...
This integration doesn't exist yet, does it? I can't find it...

@ardevd
Copy link

ardevd commented Nov 20, 2024

As of today, I now have access to some InControl data in my DIMO application, including the SoC. At the moment, to me, the SoC is the only thing that would really be essential to have in Home Assistant (to control de charging).

By creating an association between the DIMO itself and Home Assistant, It could possible to get some info...

This integration doesn't exist yet, does it? I can't find it...

The integration close to being released. @msp1974 and I are just chasing down some minor remaining issues before we open source the repo and submit it to HACS, etc.

@onorbe
Copy link

onorbe commented Nov 20, 2024

If you need, I can test it.. I'm very excited! 😁

@Jazmodo
Copy link

Jazmodo commented Nov 20, 2024

Ditto, I’ve just linked Dimo to my Jaguar account. Looking forward to the HACS integration.

I’m ideally after location & SoC data to run some automations, so fingers crossed 🤞

@ardevd
Copy link

ardevd commented Nov 20, 2024

Ditto, I’ve just linked Dimo to my Jaguar account. Looking forward to the HACS integration.

I’m ideally after location & SoC data to run some automations, so fingers crossed 🤞

That should be very doable! @msp1974 implemented device tracker support yesterday!

Quick preview (from a Defender)

Screenshot 2024-11-20 at 19 28 50

@ardevd
Copy link

ardevd commented Nov 23, 2024

Integration repository is now live here: https://github.com/ardevd/ha-dimo

Note that it's still in a very early stage, onboarding in a bit tricky, and HACS integration is now live yet :) Please read the README.

@Sooty70
Copy link

Sooty70 commented Nov 23, 2024

I’m thrilled to try this, but I’m facing an issue. I’ve reached the point where I attempt to purchase the DCX app, but it fails due to the UK not being included in the list of supported payment countries. Could you please provide any view on when the personal use app will become free on the DIMO platform?

@ardevd
Copy link

ardevd commented Nov 23, 2024

I’m thrilled to try this, but I’m facing an issue. I’ve reached the point where I attempt to purchase the DCX app, but it fails due to the UK not being included in the list of supported payment countries. Could you please provide any view on when the personal use app will become free on the DIMO platform?

Easiest solution right now is either to transfer DIMO tokens from your DIMO app (if you have a sufficient amount), or buy POL tokens from any major crypto currency marketplace. (Coinbase, etc).

I don't have an ETA on the new free-license deal. The console literally launched this week, along with a new accounting system.

However, if you're stuck reach out to me on Discord and I'll see if I can help you out ;)

@onorbe
Copy link

onorbe commented Nov 23, 2024

Im in the same situation I can’t purchase dcx in my country.. can you help me?

@ardevd
Copy link

ardevd commented Nov 23, 2024

Im in the same situation I can’t purchase dcx in my country.. can you help me?

Same answer. Purchase POL, or reach out to me on Discord :)

@dawnofwar911
Copy link

Got it working, although HACS wouldn't even clone it even when adding it as a new repo... had to do it all manually! But it's up and running!

I Will have a play tomorrow! Thanks for all your hard work both!

@onorbe
Copy link

onorbe commented Nov 23, 2024

Working!!!

@ardevd
Copy link

ardevd commented Nov 24, 2024

Got it working, although HACS wouldn't even clone it even when adding it as a new repo... had to do it all manually! But it's up and running!

I Will have a play tomorrow! Thanks for all your hard work both!

Yeah I don't think the repo structure is compatible with direct clone through HACS but we'll get it added soon! :)

@Sooty70
Copy link

Sooty70 commented Nov 24, 2024

@ardevd many thanks for your work and help on this! I still need to do a bit of tweaking, however, very happy with the initial results leveraging the JLR API into DIMO/Smartcar.

image

@ardevd
Copy link

ardevd commented Nov 24, 2024

Super cool dashboard!

@onorbe
Copy link

onorbe commented Nov 24, 2024

Hello! Can I make two questions?

  1. Change the refresh time
  2. Sometimes Dimo app give me ev km another fuel arrange in km. Exist the way shoe me both at the same time like I control app?

@ardevd
Copy link

ardevd commented Nov 24, 2024

Hello! Can I make two questions?

  1. Change the refresh time

  2. Sometimes Dimo app give me ev km another fuel arrange in km. Exist the way shoe me both at the same time like I control app?

Firstly, let's keep issues related to the DIMO Integration in that repo :)

I'm not quite sure I understand your point no 2.

@CorceiroPT
Copy link

However, if you're stuck reach out to me on Discord and I'll see if I can help you out ;)

@ardevd, I also can't pass the purchase phase and can't find a way to transfer BTC ou other currency to POL to try to pass that stage. How can I reach you on Discord? You are not accepting frind inviters. My user on Discor is .corceiro

@ardevd
Copy link

ardevd commented Nov 24, 2024

However, if you're stuck reach out to me on Discord and I'll see if I can help you out ;)

@ardevd, I also can't pass the purchase phase and can't find a way to transfer BTC ou other currency to POL to try to pass that stage. How can I reach you on Discord? You are not accepting frind inviters. My user on Discor is .corceiro

Sent you a friend request :)

@CorceiroPT
Copy link

Sent you a friend request :)

Didn't receive the request, @ardevd. I think Discord is not so user friendly when trying to get new friend. Please try by my nickname (Wolfy) or my originally name (corceiro#4604). Sorry for the inconvenience...

@ardevd
Copy link

ardevd commented Nov 24, 2024

Tried adding all of those ^^

@ardevd
Copy link

ardevd commented Nov 24, 2024

Sent you a friend request :)

Didn't receive the request, @ardevd. I think Discord is not so user friendly when trying to get new friend. Please try by my nickname (Wolfy) or my originally name (corceiro#4604). Sorry for the inconvenience...

Alternatively, join the DIMO discord server and you can add me as a friend

@CorceiroPT
Copy link

Alternatively, join the DIMO discord server and you can add me as a friend

Can you share the DIMO discord server invite here (or on private Discord message)?

@BunpGhost
Copy link

Hi Guys, Excellent work here.
As anybody considered using smartCar directly without DIMO, to integrate to Homeassistant?
Just curious...

@msp1974
Copy link
Owner

msp1974 commented Nov 27, 2024

Thought about it but you can only do 500 calls on a single vehicle per month on free tier. So less than once an hour.

Oh, and you would have to write something to onboard vehicles to it also. Dimo has done this for us.

@Sooty70
Copy link

Sooty70 commented Nov 27, 2024

Thought about it but you can only do 500 calls on a single vehicle per month on free tier. So less than once an hour.

Hi @msp1974 , how do I find out more about this? I'd like to understand if it is possible to increase the frequency and if so what the associated cost would be.

@msp1974
Copy link
Owner

msp1974 commented Nov 28, 2024

https://smartcar.com/pricing#pricing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests