Replies: 20 comments 47 replies
-
|
Hi @thekayman - there are two of the three already for 5 minute energy. The suggestion to have a self-consumed solar sensor is a good one and I'll look into it. Let me know if I've misunderstood anything.
Please stay tuned. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Peter, Yes that is right, however it seems like the example above the sensors you have given are showing the accumulated total for the day? generation to grid (export) To get similar data from the SA dashboard like this but in live view and logging so I can chose history card etc. With the new 2023.3 update can the converted wh to kwh sensors be removed? |
Beta Was this translation helpful? Give feedback.
-
|
The 2023.2 update provided the Wh to kWh options for energy. This was fixed in my release 4 if you've not implemented it as yet - together with some other additions that you might find useful. Now that 2023.3 provides W to kW I'll remove the same for the power sensors. This will be included in the coming release 5 in testing - see Issues #12 for the latest code if you are interested in testing - #12 (comment) |
Beta Was this translation helpful? Give feedback.
-
|
@thekayman - are you using the HA Energy Manager? Where you can select different views of energy by hour (within a day), by day (within a week or month), by month (within a year). There are then various energy cards you can add to your dashboard. For example: |
Beta Was this translation helpful? Give feedback.
-
|
@thekayman - Try adding this code as a sensor to your solar_analytics.yaml. Please let me know if you find any issues. btw, it was more complicated than expected as the new kWh options in HA confuse the calculations and display - hence the calcs from the source data, not the derived sensors (i.e. sa_todays_energy_consumed_total - (sa_todays_energy_generated_total - sa_todays_energy_exported). |
Beta Was this translation helpful? Give feedback.
-
|
Couldn't help myself ... here's some more ... |
Beta Was this translation helpful? Give feedback.
-
|
WOW!!! I wish I had your skills! I have also just added in the above 3 extra sensors and will test / report back. the main reason I wanted the live data for the below 3 sensors is so I can trigger automations based on usage. I.e. if generating enough and not consuming all the solar than turn the AC on colder and when consuming too much back off etc. Also sometimes if i have dishwasher and ac on and I want to know if I have enough spare solar to turn on washing machine, at the moment I have to fire up solar analytics and check there but if it was all in HA with live data that would be awesome! generation to grid (export) |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thekayman You're welcome - you can buy me a coffee if you think there's value in my work. https://www.buymeacoffee.com/peter24x7 For triggering things on/off based on the excess of generated energy or not you could simply use the sa_import_export_power sensor. It can jump up and down by the minute depending on what's going on in your house or clouds passing by. As such you may wish to set a minimum on and off interval so you don't over work your equipment. A good point regarding the release note in the source yaml - will do. Lastly there is a release 5 coming, but I'm trying to iron out one more problem. |
Beta Was this translation helpful? Give feedback.
-
|
I will defiantly buy you a coffee. I am trying to get my head around the difference between Percentage Consumed Energy Generated & Percentage Generated Energy Consumed but cant, should the sum of both both = to 100% as mine are 65% & 31% The PV Performance sensor doesn't seem to have any values now, not sure if it takes a while to generate or something broke. I'm trying to create this card with the Tesla-style-power-card but with live data however cant get the correct sensors. I changed all the entities / sensors to kW or kWh and it seems like the Power Import Export isn't calculating right vs when in W |
Beta Was this translation helpful? Give feedback.
-
|
As noted below the import export calculations for power are confused by the unit changes. Suggest you leave your lovelace units as Watts for the time being. Your energy numbers look as expected. |
Beta Was this translation helpful? Give feedback.
-
|
Try this code to replace the existing sa_import_export_power sensor. I've changed the value_template calculation to use the source data direct from sa_live_site_data, rather than from the derived sensors which are confused by the units. You may also note that the state_attr is also updated/fixed for the case when the sa_live_site_data fails. I've fixed this else where in the main source too. |
Beta Was this translation helpful? Give feedback.
-
|
That worked beautifully, seems like now no matter what unit I put for power gen or power consumption the import export value calculates correctly. |
Beta Was this translation helpful? Give feedback.
-
|
Just noticed that sensor.sa_todays_generated_consumed_percentage and sensor.sa_todays_consumed_generated_percentage defaults to a high number digits of precision. i.e. mine show 26.5707636979056% and 44.4950113775599%. Might be useful to define decimal places to 1 or 2 places, i.e. 26.6% / 44.5% |
Beta Was this translation helpful? Give feedback.
-
|
Yes noticed this too and have adjusted it to .1 of decimal thank you :) |
Beta Was this translation helpful? Give feedback.
-
|
Yes the new 2023.3 Watts to kW unit conversions in Lovelace are proving to be a headache! I presumed they just changed the units for display purposes, but they also change the units for calculation purposes. Not very useful. I will try to develop a code workaround. This doesn't seem to have been the case for energy units in Wh to kWh released in 2023.2. They are two different % - both are ratios based on the new consumed-solar kWh. See image below. The first is a % of consumed-solar vs all consumed kWh (yellow). The second is a % of consumed-solar vs all generated kWh (orange). You will note that I've been charging my EV today, so most of the generated solar has been captured, but I've also consumed extra from the grid. You can otherwise define the level of % decimal places in the sensors Lovelace display precision settings but will nonetheless put in some code to round to 0 decimal places(?). |
Beta Was this translation helpful? Give feedback.
-
|
I have gone through and grabbed all the sensors / templates in your code and dumped them here with my explanations (Hope I understood them correctly) I thought you might want to add it into the documentation, I thought it might be a easy way for all to reference the available sensors and what they do, just finding ways to give something back. sa_status - Shows the status of site (updated every hour) sa_todays_energy_total_net_cost - Reports the net cost (imported cost minus exported revenue) sa_todays_air_conditioner_total - Used if you have monitoring on certain SA Channels |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thekayman - thanks for this effort. I'll use it with some changes as part of the documentation in the next release. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thekayman and @Roving-Ronin - the new Lovelace ability to change power sensors from Watts to kilo-Watts has been doing my head in; it's clearly not fully implemented and despite much effort I could not fix issues with the sa_live_site_data REST get sensor. I recommend avoiding any issues by keeping your power sensors as Watts for the time being. I've otherwise done a number of additions and fixes as follows. The whole new (but DRAFT) code block is as per this link - #12 (comment) . I'll be testing it myself for another week, but feel free to test too please. Let me know how you go. |
Beta Was this translation helpful? Give feedback.
-
|
@PeterH24x7 seems OK here.....and I've been running the entities set to kWh (just to the default Energy Dashboard, no DIY dash and cards though). |
Beta Was this translation helpful? Give feedback.
-
|
Time for becoming an official integration ;-) |
Beta Was this translation helpful? Give feedback.
-
|
I have been running it since the day you posted it... everything running as it should and also have set the entities to kWh . Only one thing I noticed is the energy dashboard and the net cost entity are different but I think they might be calculating differently?
|
Beta Was this translation helpful? Give feedback.
-
|
Hi @thekayman - I've noticed the EM Grid Total and the sa_todays_energy_total_net_cost can get out of sync if you do any HA updates/restarts on any given day. If it nonetheless runs for the day without interruption then they stay synced. I would agree that it's strange that they do get mixed up - there's clearly something special and undocumented about the way both the sa_todays_energy_exported_compensation and sa_todays_energy_imported_cost operate within the EM. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thekayman - another observation is that the display of kWh and costs in the Energy Manager only update every hour. Whereas the sa_todays_energy_exported_compensation, sa_todays_energy_imported_cost, and hence sa_todays_energy_total_net_cost, are updated in near realtime as the kWh are updated every 5mins. |
Beta Was this translation helpful? Give feedback.
-
|
The issue I was seeing by setting the 3x power sensors sa_generation_power, sa_consumption_power and sa_import_export_power to kW can be seen in the following HA History views - depending on how I coded the error recovery of sa_live_site_data I would either get a blank/gap power sensor state, or a zero state, at 0 minutes of the hour. Have you seen the same? I simply fixed it by changing the sensor units back to W.
The trigger is the way SA responds to the sa_live_site_data get at the 0th minute of the hour with no data (see image below). Normally it's defined to provide the last 6x minutes but this doesn't happen at the start of the hour - first there's a blank, then 1 minute's data, then the 2nd up until the 6th minute when things then operate the same until the end of the hour. If the code is working correctly then the "unknown" get should result in no change in the 3x power sensors which just doesn't happen for me with the power sensors set to kW. Results at minute zero of the hour: Results at minute 2 of the hour: |
Beta Was this translation helpful? Give feedback.
-
|
Looks like I am getting the same and didnt know it.
I've just changed power consumption to W and will report back after a few hours |
Beta Was this translation helpful? Give feedback.
-
|
Same here... drops to zero on the hour. |
Beta Was this translation helpful? Give feedback.
-
|
Surprisingly, and without changing the REST get call, the contents of the sa_live_site_data response now seems to have changed to 6 intervals of data 5 seconds apart, not 1 minute apart. The functioning, fortunately, doesn't change (i.e. just extract the latest consumed/generated data at the end of the array). This REST get is not in the Solar Analytics published API so maybe it's not unreasonable for it to change without warning. Nonetheless a strange coincidence to be reviewing in/around the time of the change.
|
Beta Was this translation helpful? Give feedback.
-
|
I changed consumption to W and left generation to kW, The W doesnt have the dips. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thekayman - good to see you're seeing the same symptoms, but unfortunately I don't know how to fix the problem as it seems related to the HA Lovelace settings for the units. It remains unclear to me how a sensor's display setting impacts both the results of a rest data get and calculations (but now fixed for the sa_import_export_power). Hopefully this gets fixed in HA. It's the end of the week and I've not seen any other issues. So will publish a new release in the next days. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thekayman and @Roving-Ronin - I've needed to make a minor fix to the two new percentage solar used sensors. See comments in code and the screen shots that follow. In short, 1] there was the potential to get strange and very large % numbers at midnight when the SA sensors reset to zero, and 2] the percentages could be negative sometimes due to a small current draw from the inverter (e.g. on start-up). Again the strange behaviour of the HA Lovelace unit setting came into play here and continues to confuse the situation further. |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
Another related problem fixed; this one applying to the sa_todays_energy_import_rate and sa_todays_energy_export_rate sensors which were found to be sometimes spiking at midnight as the day's total kWh reset to zero (or near zero in the first minutes). Just thinking about these sensors, they presume that sa_todays_energy_imported and sa_todays_energy_exported are both set in HA Lovelace to kWh (not their default Wh). If they were in Wh then the results will be incorrect. Again this whole Lovelace sensor unit matter continues to do my head in. |
Beta Was this translation helpful? Give feedback.
-
|
Just did a bunch of HA integration, HACS modules and a HA core upgrades w/ auto-restart at end. No issues. |
Beta Was this translation helpful? Give feedback.
-
|
I just restarted and waited 1 hour than checked....I did not have any spikes. |
Beta Was this translation helpful? Give feedback.
-
|
@PeterH24x7 sorry been crook for a while. Just looked at the Energy Dash and found this for today.
in comparision, a check of the SA portal shows:
So 8.2kWH Solar + 10.9kWh Imported = ~19.1kWh consumed (so far) today. Not the 33.4kWh that HA has spiked to show. Looking in the logs I noticed a few other errors: Logger: homeassistant.components.rest.sensor Empty reply found when expecting JSON data Logger: homeassistant.helpers.template_entity TemplateError('UndefinedError: 'None' has no attribute 'state'') while processing template 'Template("{% if state_attr("sensor.sa_todays_energy_exported", "unit_of_measurement") == "Wh" %} {% if (states.sensor.sa_todays_energy_exported.state | int) > 100 %} {{ ((states.sensor.sa_todays_energy_exported_compensation.state | float) / (states.sensor.sa_todays_energy_exported.state | float) * 1000.0 ) | round(3) }} {% else %} 0.0 {% endif %} {% elif state_attr("sensor.sa_todays_energy_exported", "unit_of_measurement") == "kWh" %} {% if (states.sensor.sa_todays_energy_exported.state | float) > 0.1 %} {{ ((states.sensor.sa_todays_energy_exported_compensation.state | float) / (states.sensor.sa_todays_energy_exported.state | float) ) | round(3) }} {% else %} 0.0 {% endif %} {% else %} unknown {% endif %}")' for attribute '_attr_native_value' in entity 'sensor.sa_todays_energy_export_rate' Logger: homeassistant.components.recorder.db_schema State attributes for sensor.sa_data_by_5min exceed maximum size of 16384 bytes. This can cause database performance issues; Attributes will not be stored |
Beta Was this translation helpful? Give feedback.
-
|
Hi @Roving-Ronin - good pickup. Seems like a repeat issue of mine above. It's strange that the sensor.sa_data_by_5min.attributes.data exceeds the maximum limit in the 12pm data get, but then for the 1pm and later get is okay (i.e. not exceeding the maximum limit). Given it only happens rarely I can only assume there's a data quality issue and we would need to capture the attributes.data of sa_data_by_5min at the time to find out why. It's my experience that attribute information is unfortunately not retained in the history like state data. Strangely the error mentions {% if state_attr("sensor.sa_todays_energy_exported", "unit_of_measurement") == "Wh" %} but this is not part of either sa_todays_energy_imported and sa_todays_energy_exported. Were other sensors dependent on sa_data_by_5min impacted - e.g. sa_todays_energy_consumed_total, sa_todays_energy_generated_total, sa_todays_air_conditioner_total? A fix may involve the use of the Restful sensor, rather than the sensor platform rest, where the get data does not need to be stored as an attribute and the dependent sensors are all part of the one definition. Nonetheless data corruption may still be an issue. I could also try putting some error detection for strange values after the {% endfor %} in the code below. I note the use of the for loop is unique to the sa_todays_energy_imported and sa_todays_energy_exported sensors so this could be related to the cause. Another difference could be {% set energy.imported = energy.imported + sensor_data.energy_consumed - sensor_data.energy_generated %} should be forced to use float numbers? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thekayman and @Roving-Ronin - FYI I've been reworking the code to use the Restful sensor as referenced above. The new implementation (for all REST gets) no longer relys on storing the returned data get as a sensor attribute and the max size issue is fixed. The resulting source sensors, which feed the previously defined sensors, also seem to fix the W|kW and Wh|kWh problems that were being experienced. It has involved a lot of change to the overall structure of the code, but the underlying functions remain practically unchanged and non-breaking from the previous version. Will test for another couple of days and then publish a new base. Please stay tuned. Chrs, Peter |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thekayman and @Roving-Ronin - your thoughts please - default to zero (as coded above), or default to None (proposed as below)? I'm inclined to go with None. I recently discovered another non-numeric sensor state "none" - defined with "Unknown" and "Unavailable" as follows:
Firstly it looks like I've been using Unavailable in some cases when I should have been using Unknown. Secondly, I feel "None" should be used for sa_todays_generated_consumed_percentage, sa_todays_consumed_generated_percentage, sa_todays_energy_import_rate and sa_todays_energy_export_rate rather than defaulting to 0.0 when the sensor state value is undefined. For example, for And the resulting output looks like the following - in effect a data gap from midnight (when the senor resets) to the point where generation actually starts occurring - which is when this sensor actually becomes meaningful.
|
Beta Was this translation helpful? Give feedback.
-
|
Hmmm for example of when it's not generating I would say 0 as it's generating zero.... If it's none it might mean an issue? I notice the PV performance doesn't work after booting until it gets its value so the chart doesn't show which looks ugly.... If that is none might be better? |
Beta Was this translation helpful? Give feedback.
-
|
Again I'm inclined to go with the use of "none" - any major objections? You can of course edit my updated source to whatever you like. Here's the before (orange) and after (yellow) as shown in the history (with some purple which was some implementation noise). While there are data gaps in the latter (using none) I think it looks better than the before (using 0.0) which shows lots of jumping around. Some explanation of why is as follows:
|
Beta Was this translation helpful? Give feedback.
-
|
I see what you mean with the graphs.... yes None works well it seems as it stops the spikes. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @Roving-Ronin, @thekayman and anyone else following this thread, Per my note above regarding redoing the REST gets as RESTful sensors, the code has now been substantially restructured and now almost 1,000 lines. It is functionally the same as previous versions regarding sensor naming and functioning. As a bonus, the sensor unit matter (as selected w/i Lovelace) is also no longer an issue. If you use SA for monitoring EV charging you will note some included bonus sensors that show the breakdown of EV charging energy by generated and imported sources with percentages. I've been doing my own testing over the last week and everything seems to be working as expected. If you could do the same and let me know of any problems I hope to finally finish and publish this release for the benefit of others. I hope this is all useful to you. Thank you for your help. Chrs, Peter UPDATED 15-APR-23 MINOR FIX TO sa_data_by_5min_meter_reset. UPDATED 11-APR-23 WITH FIXES AS NOTED IN THE COMMENTS DOWN BELOW PLUS SENSOR IN SUPPORT OF AN APEX-CHART CARD (INSTALLED VIA HACS) OF RECENT ENERGY USAGE BY CHANNEL NOTE THE HEADER TEXT ALSO INCLUDE ADVICE FOR UNGRADING FROM A PREVIOUS VERSION OF THIS CODE. |
Beta Was this translation helpful? Give feedback.
-
|
@PeterH24x7 |
Beta Was this translation helpful? Give feedback.
-
|
Just noticed that
|
Beta Was this translation helpful? Give feedback.
-
|
Hi @Roving-Ronin - thanks for the valued feedback.
I will shortly update the big code block just above with these changes. Maybe @thekayman is ready to test too if he's not done so already. See: |
Beta Was this translation helpful? Give feedback.
-
|
@PeterH24x7 .... now just need to have this added as an official integration. ;-) stir stir... |
Beta Was this translation helpful? Give feedback.
-
|
I have just managed to use the new code, fyi when deleting the SA yaml file HA wont reboot as it says its referencing the file in configuration.yaml, i did a host reboot and when it came up it went into safe mode, I copied the new file and rebooted again and it seemed to work and doesnt look like i have any duplicates. I had the same decimal issue, when i change from W to KW it goes to the full decimal, if i edit it shows its meant to be 3 decimal places but its clearly not, once I select the 3 decimal places its all sorted. Both my FIT and import are 2 decimal places (powershop) and I believe AGL was the same. weird thing is i noticed the todays export rate started at 0.002 and is increasing which is odd as I have it set as a static rate.
|
Beta Was this translation helpful? Give feedback.
-
|
Hi @Roving-Ronin and @thekayman - sorry couldn't help myself over Easter and added another 200 lines of code (updated as marked in the source above) to generate near realtime energy (shown as 5min average power in kW). You will need the APEX Charts (apexcharts-card) add-on installed via HACS. Once installed, create a new Lovelace Apex Charts card and cut/copy the code below and it should start working without too many issues. You will need to edit the code to suit your channel set-up. Note: due to HA limitations the timespan is limited to a history of 2h20m (or 28x 5 minute intervals) due to the 255 character size limit of sensor state data.
|
Beta Was this translation helpful? Give feedback.
-
|
@PeterH24x7 |
Beta Was this translation helpful? Give feedback.
-
|
@thekayman as another 3phase user, so we can only track import / export / consumption and the EV / AirCon / Oven etc just throw status errors... here's a copy stripped back to only support 3 phase. @PeterH24x7 great work on these new sensors, but is it worth publishing a 3 phase only version (and the "full version" also), given there'd be a lot of 3p only users nowadays ? |
Beta Was this translation helpful? Give feedback.
-
I think I might have worded myself incorrectly, its more so that sensor.sa_todays_energy_export_rate starts of at 0 and then slowly moves up to the export rate (in my case 0.13) seems to be same with import rate, shouldnt these be static as it is set in the energy dashboard? If you see image below you can see its varying. |
Beta Was this translation helpful? Give feedback.
-
|
@Roving-Ronin - good idea regarding the 3 phase version. I'll review in the next days and update the readme and comments to suit before publishing. Thats for providing the code. @thekayman - not sure why you don't get matching average rates for your static rate. I would only note that it doesn't start providing an average until at least 100Wh have been recorded; which isn't much in the case of import, but doesn't start accumulating until you start generating in the case of the export rate. I would also add that your PV inverter may be drawing power when it's not generating so at midnight it starts at zero, goes negative through the night or early morning as it starts up, and then eventually goes positive when you start generating. Maybe this explains your situtation at least for exports. I'm using amber so the rate for import and export varies every 5mins so a daily average is particularly relavant to me; maybe less so yourself. Here's the av. export rate code for reference (and the import rate is similar) - it's just the total cost for the day divided by the total kWh for the day: |
Beta Was this translation helpful? Give feedback.
-
|
I think i am not understanding what sensor.sa_todays_energy_export_rate is. Also just noticed the Total Energy sensors are not adhering to the decimal points, they are all set to 3 decimal points but it's still showing 7, if i edit the sensor and change it to anything other than default it works as it should but if i change it back to default which is 3 decimal points it still shows 7. (screenshots attached). |
Beta Was this translation helpful? Give feedback.
-
|
Hi @thekayman - not sure why the "display precision" is not working as expected where you should be able to set the number of decimal places regardless of the source. Strange too that it provides 7 digits decimal places when each of the source sensors (e.g. sa_data_by_5min_energy_consumed_total which is in turn sa_todays_energy_consumed_total) have a Regarding the average daily unit rate sa_todays_energy_import_rate and sa_todays_energy_export_rate (as mentioned above) these are just the total $ as calculated by the HA EM divided by the total kWh as provided by the SA code. Is your HA EM configured with the correct $/kWh rates (which may be down to 3 to 4 decimal places on your bill - and may have peak/off-peak/shoulder times)? Do the daily total kWh in HA for sa_todays_energy_consumed_total and sa_todays_energy_generated_total match the corresponding numbers in the Solar Analytics Usage page? I'm otherwise not sure what the issue could be. My energy $/kWh rates from Amber vary every 5 mins and my SA daily bill calculations are better than +/-5% accurate on a daily basis (see images below). Given the likely interval timing and measurement discrepancies between my smart meter and my Solar Analytics calculations, I think this is quite good. Even better I'm able to provide 5 minutely updates on my daily energy cost and EV charging costs, neither of which are provided by Amber on their website (i.e. previous day totals only). From HA energy manager - where $4.61 / 17.85 = $0.258/kWh matching the Amber average rate at $0.26/kWh (below): From Amber (noting my rates include 100% GreenPower at ~5c/kWh): Happy to have a Google hangout or other online call to review further if you like. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @Roving-Ronin and @thekayman - Release 5 has now been published for the benefit of others. Thank you for all your contributions and help with testing and debugging. The 3-phase version of the code is also included (with minor comment edits to that provided above) in the repository as <solar_analytics_3phase.yaml>. I will shortly close this discussion thread and a new one can be opened for any future work. |
Beta Was this translation helpful? Give feedback.
-
Is there any way to have the live or 5 min sensor to show seperate data such as
generation to grid (export)
generation to house consumption (solar usage)
grid to house (import)
Beta Was this translation helpful? Give feedback.
All reactions