Just FYI - Forced Inverter Report using NEWSCAN

[givecoinsupport]

Hi, How are you ? Thank you for the great effort.
Just Let me share a small top to force to refresh report from Inverters for 1 minute interval.

In the old Local API, if call, COMCHECK. it can refresh report time to 1min.

BUT, in new API (v7) COMCHECK method is not working, have to use PUT NEWSCAN to force to refresh Inverter Data Report. and, can get 1 minute resolution.

if using Enphase's document to get a token with 'user[email]', only can get an owner token. I am trying to find a way to get a installer token to get installer access

[mr-manuel]

Can you please explain better where do you call COMCHECK against and NEWSCAN?

[mr-manuel]

@Matthew1471 maybe you know how to achieve that the inverters are reporting every minute instead of every 5 minutes?

[Matthew1471]

Apologies for the delay, I've been upgrading my Enphase to another 7 more panels (and experiencing PLC incompatibility with Tesla PowerWall 2).

Have a look at these lines, I previously noticed calling the communication check would trigger that.. I haven't tried running this recently however. I know I say it a lot but the examples are a really good place to check for API usage.

Given it's in /installer/ usual caveats apply on the token role type. Would prefer to not say this too loudly (hopefully burying this in an issue should be discrete enough) but Enphase don't actually strictly enforce who is an installer or not, so it isn't too difficult to use a different e-mail address, register yourself as an installer and then as the owner "invite" your new "installer" to also have access... that's how most of us have access to the installer functionality.

A new scan is a PLC scan, basically a broadcast request for provisioning.. and probably inadvisable to run routinely (stress on the box, noise on the wire and the potential to accidentally provision your neighbour's microinverters!). I'll do some more digging on what other things trigger a PCU / inverter comm check, there might be a path from an owner token but I'd need to spend some more time reading the source and doing a Find In Files.

Enphase themselves moved over to CT clamps and meters for production stats (Enlighten lets installers choose where the measurements come from but CT Clamps are much more preferred), the 5 minute granularity is usually appropriate for the panel by panel breakdown (Enlighten itself is every 15 minutes don't forget!) but for second by second power measurement the IQ Gateway Metered with properly configured CT Clamps are the preferred way to go. Again, in the examples folder is some code to get those meter readings.

[mr-manuel]

I'm already using the CT clamps for measuring. I would only need it for some specific situations as I'm discussing since over 6 months with Enphase for a bug in the microinverter firmware and they continue to not understand me and to don't believe me. The only thing they do is testing different grid profiles for 48 hours, since months...

I have no shading and all panels are exposed to exactly the same amount of sun, still a lot of panels do not produce correctly in a micro grid.

Will test that, since I already have the needed rights. Thanks!

[Matthew1471]

I assume you have also performed a PLC noise test and checked to see if the panels are producing but failing to report (the panels produce so long as they have a grid profile and favourable grid conditions, they don't really need the IQ Gateway - the IQ Gateway and it's PLC chatter is just for data)? That's the issue I have at the moment, et al this person. 7 panels on the same cabling as the Powerwall and its heavy duty switching inverter get drowned out by all the Powerwall (EMI? EMF?) noise on the wire. Enphase recommend filtering it with a line filter or trying to chuck some ferrite cores on the Powerwall AC to absorb its noise.

I haven't delved that deeply into the panel reporting (although I was excited to find they report their individual temperatures too).. the other thing to check might be the event logs in the installer portal, if they're shutting down due to something they don't like they should issue a dying gasp and the gateway should generate an event in the log (it will mark it as resolved but you can still go into either System Dashboard or unfiltered Homeowner Events to view them). If they fail to report then the IQ Gateway itself will issue a "failed to report" message for every microinverter that doesn't check in.

I haven't attempted to look for microinverter uptime stats... I'll try to improve the documentation when I have more time and hopefully we can figure something out. Hope the comm_check trick works for you in the meantime!

[mr-manuel]

Unfortunately the error/status reporting of Enphase sucks. It always only says DC power too low. Even if there is full sun. There is no noise with the Victron Energy MultiPlus, but I still added some ferrite cores.

The problem raises after the inverters go over 50 W in the morning. If then I turn off power production for 60 seconds and turn it back on, all is working as it should.

[Matthew1471]

/installer/pcu_comm_check calls commcheck.lua on the IQ Gateway with --installer as one of the command line arguments which is documented in the script as Flip the installer flag for back-to-back polling for 60 secs. I couldn't find any other meaningful calls to this script apart from maybe /ivp/ensemble/comm_check.

The commcheck.lua itself is reliant on /ivp/peb/pmi/app/ssi which I believe is a native process and also locked down to roles higher than owner so you'd still need an installer token in practice anyway but potentially could shave some time off the calls... although you are running pretty low level close to the PLC modem then (potential for bricking goes up).

There is also /ivp/peb/pollperiod which may allow you to change how frequently they poll - but I'm not sure changing the internals of the IQ Gateway would be wise if you're trying to get them to support the device - they could argue the issue is caused by non-standard config.

Sticking to the standard installer friendly /installer/pcu_comm_check is the way to go I suspect. There won't be any log lines suggesting you've run any more esoteric queries then that could come back to haunt you.

[Matthew1471]

Unfortunately the error/status reporting of Enphase sucks. It always only says DC power too low. Even if there is full sun. There is no noise with the Victron Energy MultiPlus, but I still added some ferrite cores.

Ferrite cores apparently dampen the AC noise but they won't eliminate it and if the noise is that high might be too much. Do you have an export of your PLC noise scan from either Toolkit or System Dashboard to work out if PLC AC noise is low/medium/high. For me simple debugging was turning off my batteries and doing a PLC noise scan showing the before/after (not that I needed to as the microinverter comms red lights instantly went green on the IQ Gateway then anyway). Along with the "microinverters failing to report" message self clearing as I turn off the Powerwall batteries.

DC Low doesn't sound AC grid related tbh anyway, that will be solar panel to microinverter connection.. the panels are supported for the microinverters (using Enphase compatibility calculator online?)? There's some DC params that the inverters need that are beyond my electrical skill.

DC Power low will occur quite naturally early in morning and late at night as you'd expect.. I'm not an electrician but I'd be looking to see if I could measure the DC voltage/current on those wires to rule out if the panels are having a wobble or internal short or something that is causing DC power loss to the PCU (Power Conditioning Unit) / Microinverter. I almost wonder if you're able to take a good working panel and give that to the microinverters that are crashing out?

The problem raises after the inverters go over 50 W in the morning. If then I turn off power production for 60 seconds and turn it back on, all is working as it should.

Is this some of the panels or all of them? Are there some good panels and some bad panels? Have the panel makers been much help?

[mr-manuel]

There is no issue of the panels. Just the message DC power too low is wrong. It also appears, if I have full sun, grid connected and the inverters are throttled. There is no message like power reduced because of zero feed in selected, power limited because of frequency shift or anything else. Deep troubleshooting is not possible. Also the Enphase support is quite primitive with the tools they have to analyze a problem.

An Enphase engineer confirmed me that a service in the microinverter gets stuck. Therefore the power freezes to the current value and cant go up or the power goes to zero. Only a reboot of the inverter fixes the problem. There is no hardware issue, just a firmware bug they don't believe in.

Sometimes 12 of 16 inverters have the issue, sometimes only 8. Mostly at least 8 of 16 have the issue. Every time other ones. This also confirms me that this is firmware issue. I also tested all panels, they are all brand new and fine.

I'm using the IQ7A, just in case someone likes to know.

[mkimid]

Can you please explain better where do you call COMCHECK against and NEWSCAN?

in D5 Envoy firmware, when access(reading/get) http://envoy.local/installer/pcu_comm_check, I guess, envoy broadcast a command to report the status by inverters. so, just access it every minute, can get 1 minute data.

in D7 Envoy firmware, I guess, no more broadcast a command with /pcu_comm_check anymore.
when NewScanPD http://enovy.local/ivp/peb/newscan, I guess, enovy broadcast a command to report the status by inverter.
if Enable NEWSCAN every minute, can get the status of inverters in 1 minute. ( I have verified in Firefox - Just sending the packet again to force to update data from inverters)

You need 'INSTALLER TOKEN' to do this job.
When I get a token using "https://enlighten.enphaseenergy.com/login/login.json?", I just can get an owner's token.
When I try OAuth method on "https://entrez.enphaseenergy.com/login" to get an installer token , I can login and can get a session id, but, can not get a token using this session_id. it makes an invalid format/data error.

PUT method for NEWSCAN has explained in Mattew's API.
and, just need 3 parameters to enable it

duration-in-minutes
devices-to-be-discovered
nsrb-to-be-discovered

[Edited]

http://www.bomijoa.com/solar/images/chart_panel_30.csv
http://www.bomijoa.com/solar/?size=1&debug=1&detail=0&offset=0

[tmaestas95]

Hi folks - sorry for jumping on an old Issue but I'm looking for some feedback and @Matthew1471 seems to understand the working of IQ Gateway better than probably most of enphase support and this Issue sort of speaks to what I've experienced and what I'm trying to do.

I've got 19 IQ7HS MI's that stop reporting every evening when the sun goes down (as expected) but never start reporting again the next day, unless I issue a new PLC scan. https://www.reddit.com/r/enphase/comments/1gpp9ja/all_microinverters_not_reporting/ if you're interested.

Once I issue a new scan through the local envoy UI or via Enlighten, all MIs start reporting again for the rest of the day, then go back to not reporting the next day (although I do have to tell it to expect more devices than I actually have, otherwise the scan doesn't actually run as it thinks its discovered everything already).

At first I was going to start issuing daily /installer/pcu_comm_check calls but as noted earlier in this Issue in current versions this is no longer initiating the poll, so I'm now looking at /ivp/peb/newscan. I'm able to use this to initiate a scan, which "wakes up" the MI's - but only for so long as the scan is running. Once it finishes whatever duration-in-minutes I've specified, the MI's stop reporting again.

Reading through some of the code in this project has raised a question in my mind: Is the "reporting" in of the microinverters really the responsibility of the MI's, or is it actually a poll from the IQ Gateway? Could my problem be caused by whatever mechanism that is supposed to be triggering a poll in gateway not occurring? Any suggestions on how to further troubleshoot this? Having to issue /ivp/peb/newscan every day seems like a hack but I don't know enough to pin down what is causing the MI's to not report (or not be polled?) after the power down at the end of each day's sunlight.

[tmaestas95]

And i just realized that the reason the scan initiated from Enlighten page seemed to get the MIs to report all day is because it initiates an 8 hour scan. So bottom line is my MIs are only reporting so long as a scan is active. Anyone know why this is? enphase support so far has been useless. Thank you @Matthew1471 for this awesome project. It's given me a much deeper understanding of and visibility into my system.

[Matthew1471]

There's code on the IQ Gateway to calculate sunlight hours.. bit of an odd guess but is the IQ Gateway's date and time correct, have you VLAN'd it off and accidentally blocked NTP or something?

I'd also check your PLC noise levels and check your chart especially at times you are not seeing reporting.. the scan makes everything artificially chatty - whereas typically they chat only on a timer - if you have a lot of interference I have seen micros being unable to get through for a while under some operations but not others.

[mkimid]

http://www.bomijoa.com/solar/report_panels_power.php?days=18&power=10

if see the timestamp 2024-12-01, most right channel, the reporting from MI to Envoy is not limited at night.
The older D6 early firmware is limiting the time, actually, it has been generated an error if MI is working at night.

I guess, PLC noise check will be needed,

http://www.bomijoa.com/www/blog/2801

one of way to isolate Envoy, never use Q-LCF-064 (too expensive and not effective)

[tmaestas95]

No blocks on the gateway's network, time and date should be correct - but where can I verify on the gateway?

PLC noise shows low whenever I run the noise check.

I've considered trying to install a ferrite core and/or filter but it just doesn't seem like I have a noise problem according to the noise check graph. I just realized the correlation today of everything "working" while the scan was active but not when it's not - but I'm not sure what how else to troubleshoot. Perhaps I'll try the gear @mkimid suggested.

@mkimid how can the MIs continue to report through the night if they powerdown without DC power with no sun? What is the far right column #32 data point?

Side question - do you know the proper PUT body to /ivp/peb/newscan needed to cancel or suspend an active scan? And can anyone say definitively whether the reporting of the MI's has to be induced via a poll from the gateway, or do they do it asynchronously on their own?

[tmaestas95]

(All the timestamps in enlighten regarding caught-up date and last seen date are correct so I assume the date/time on the gateway is correct)

[mkimid]

@tmaestas95 the channel #32c I have connected on my battery and pumping out the battery when it is fully charged with my DC coupled solar panels. When I dumping, I am running the inverter for 7 ~ 30 days in a month for 24 hours/day.

when my gateway has D6 firmware, it is showing an error if running at night. but, no more error in D7 or D8.
Even there is an error or not, MIs are reporting well.

Anyway, let me share a glitch in Enphase website.

1. log in Enlightene.enphaseenergy.com
2. select Grid Profile in Data & Privacy
3. Click “Dashboard” / Grid Profiles < not menu, path over the Grid Profile title
4. Click Activation(0) in Systems
5. Click View Full List
6. you can see your system in the list, Click your system under Name
7. You will see the system configuration ^^
8. Left list menu - select Gateway software
   • check your firmware is up to date

note 1. Enter Serial Number : you can select any grid profile which (more than official profiles)

[tmaestas95]

Thanks @mkimid . Gateway software is up to date:

I did notice that accessing the Activation via your method that some of the steps were not listed with Green checks as completed (because when I originally created the Activation I could get through the wizard because it insisted on an Installer name which, being a self-install of Gateway, I didn't have). I had originally to click on the "Use Classic Interface" link, which allowed me to get through the activation creation. This time I was able to click through everything (all MI information and everything else was already populated, all MI's show provisioned etc) and all applicable steps are now Green check marks. We'll see if anything changes tomorrow when the sun comes up.

[tmaestas95]

I do note that the Last Update Date of 11/1/2024 is exactly when I began experiencing this issue. I had gone several weeks after installing my IQ Gateway without noticing this reporting issue, but then on 11/1 I began having the MI's not reporting each day. Seems suspicious that the software was apparently updated on that date.

[mkimid]

@tmaestas95 Maybe it is a stupid question, Did you ever turn-off gateway at night for 5 minutes, and then turn on after firmware update ?

After update firmware to 8.2.4345, MIs are reporting once an hour, after power cycle (Turn-off my gateway and turn-on after 5 minutes, just using the envoy breaker - right/mid breaker if combiner 3), it starts to report normally. It is happen on Nov 12

[tmaestas95]

I've recycled the gateway many times, but never at night - I suppose I can try that tonight?

Still trying to determine if MI reporting is simply a function of gateway polling the MIs, or if the MIs initiate the report in on their own? On /ivp/peb/newscan I notice a polling-period-secs which on my gateway always returns 1 hr :

"mins-until-next-cycle": 1,
"xdom-disabled-scan": false,
"polling-period-secs": 3600,
"polling-is-off": false,
"forget-all-scan": false

@Matthew1471 documents this as:

But I'm not sure what polling this is referring to.

[mkimid]

I recommend two things, Just remove all MIs and Listing again,

(1) power cycle at night - just turn-off the backfed breaker (between envoy and load center) for 5 minutes
(2) remove the all micro-inverters after recording the serial number at some place ::: Select "Retire" not Remove
and then, provisioning, even it is not needed, just do
(3) power cycle of gateway, and then, make sure the inverters are not showing any more
(4) Add MIs using Un-Retire -or- Serial Number which has been recorded at (2)
and then, provisioning, even it is no needed, just do
(5) power cycle of gateway and then, make sure the inverters are listed

**** for NEWSCAN *****
If use firefox and debug mode while connect on Envoy, you can check the packet.
(1) Envoy should be logged in using CALLBACK method (same as login Envoy Local Interface), I feel, there is no Token has involved
(2) And then, PUT newscan method will work
(3) I have tried to login using INSTALLER TOKEN which has generated manually from the enphase website
- COMCHECK is working with the installer token
- NEWSCAN is not working with installer token
(4) I have tried using VB.NET coding in various way (I am not a user for Python), but, not success.
(5) for NEWSCAN the device number should be more than the number of what you have

FYI 1.
- open envoy local using web browser (just open with IP address of Envoy)
- Error pop message [ok] to ignore
- Click Login with Enphase
- will redirect to Enphase, log-in
- will call back on your enovy local inverface

    connect on 
            https://envoy_local_ip_address/home?locale=en&classic=1

            and, Select 'Administration', it will give you more control

**** This is pretty obvious way to fix Enovy's Issues, NOT recommended

IF not working, contact customer service and, Request "Database Initializing"

There are two options, Database reset or Initializing, Initializing will lost all previous Production Metering Information (will reset to 0), BUT, will fix any issue which has related with database at Envoy, But, careful, will lost the Production Metering Information, If related with SREC Coupon/Rebate, Database initializing will makes an issue

[tmaestas95]

Thanks for the advice @mkimid . Last weekend I tried removing and re-adding the MI's and it didn't fix anything, but I didn't do it probably in exactly the order you listed (and I did a full remove not a retire which, as I understand it, retire will retain the previous statistics and remove will not). I perhaps will try it again after doing a night time power cycle of the gateway. If that all fails I will ask support to do a database reset or initializing. At this point I don't really care about any of the historical metering info (I don't need it for any coupon/rebate programs) - I just want the system to behave like it's supposed to. Is that reset purely on the enphase end of things, or does it actually reset anything on the Envoy/Gateway. Is there a way to essentially "factory reset" the envoy?

I have been using the classic Envoy web UI when needed (the "new" UI is slow and clunky). My enlighten credentials always return an installer token and I haven't had any problems using it for newscan or commcheck API calls (I'm using @Matthew1471's framework to manage the auth).

[mkimid]

@tmaestas95 I have reset my envoy twice to resolve some issue

(1) with D6 firmware, my enovy is displaying some wrong information such as no database usage, no partnumber, and others. after initialize database, it has fixed

(2) after D8 upgrade in last Nov, some error flags are in my system, and grid profile is not downloading correctly. so, initialize database again. but, still my system has an issue to download grid profile to MIs...

(3) I didn't asked the factory reset, maybe, that is a good idea to restart from the ground zero. I feel, the latest firmware has too many issues for the solar only system.

I have started to talk with Enphase Team from 2022, to resolve the grid instability failure. and, I can feel your frustrating, because it is hard to meet a right person to resolve an issue when contact on Enphase.

I have registered as Owner, and there is an installer. I just keep the installer because I have plan to move within 2 years. For the new home owner, I have decided to keep it. I have got the installer token manually, and I have tried with my code, and success to call commcheck and newscan. I just failed to post a newscan call in my code. I am using VB.NET / WebClient. and, I have gave-up to improve my code. Because I just needed to track the grid instability issue, but, I have found Enpahse has fixed this issue in their latest firmware (even detect DC Arc failure, they just reset instead rebooting, so, no more power interrupt)

[tmaestas95]

One question regarding database : I see this on the "new" Envoy UI page regarding the database, seems it is missing correct data elements to display whatever it's supposed to be displaying - could this be related to my problem?

I can't find anywhere in the classic envoy UI that shows something like this, nor could I find any API endpoints or sections in the Enlighten webapp that show it so I'm not sure if it's just a legacy data point that doesn't exist anymore or is pointing to some problem on the system. If so, perhaps a database initialization will fix it.

[mkimid]

@tmaestas95 the db size display issue in D8 firmware will not be fixed after database initialization, I have requested the initializing because the issue. but, not been fixed. but, The other information is OK, so, recommend to ignore it. ONE thing, after initializing you may lost all grid profile (I have lost all), and then, if need to install a new profile, need to use ToolKit, but, the grid profile in Toolkit support the official release only.

[Matthew1471]

One question regarding database : I see this on the "new" Envoy UI page regarding the database, seems it is missing correct data elements to display whatever it's supposed to be displaying - could this be related to my problem?

I can't find anywhere in the classic envoy UI that shows something like this, nor could I find any API endpoints or sections in the Enlighten webapp that show it so I'm not sure if it's just a legacy data point that doesn't exist anymore or is pointing to some problem on the system. If so, perhaps a database initialization will fix it.

https://support.enphase.com/s/question/0D53m00009HaArjCAF/d76175-f79c8d-bugs

[tmaestas95]

Interesting thread, @Matthew1471 . I notice your following comment:

Do you know if that still holds true? I'm a PV-only system but my gateway when I first commissioned it upgraded itself to some version, and then apparently again on 11/1 to what it's running now - D8.2.4345. When the gateway is power cycled and starts a PLC scan it seems to want to find 2 esub devices, and my system has none. Don't know if this is just a default or if it means anything.