Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add: epex_spot_data_quantile for the next 24Hours #129

Open
heinemannj opened this issue Jun 3, 2024 · 3 comments
Open

Add: epex_spot_data_quantile for the next 24Hours #129

heinemannj opened this issue Jun 3, 2024 · 3 comments

Comments

@heinemannj
Copy link

Actual quantile calculation window is the actual day (00:00 - 23:00).

For battery optimazation strategie

  • force charge,
  • charge from grid,
  • disable discharge,
  • ...

it really make sense to have a quantile calculation window for the next 24Hours in addition.

image

Legend:

  • Weather: DWD Integration (Deutscher Wetter Dienst)

  • WW: myVaillant Integration (Warm Water Heating)
  • Battery: Huawei Solar Integration (for Battery SOC)
  • Battery FCST: EMHass AddOn (for Battery SOC Forecast [16:00 - 16:00])
  • House: Template Sensor (for House Load)
  • House FCST: EMHass AddOn (for House Load Forecast [16:00 - 16:00])
  • Grid: Huawei Solar Integration (for Grid Import/Export)
  • Grid FCST: EMHass AddOn (for Grid Import/Export Forecast [16:00 - 16:00])
  • Solar: Huawei Solar Integration (for Solar Production)
  • Solar FCST: SolCast Integration (for Solar Production Forecasts [00:00 - 23:00])

  • EPEX Spot (EPEX Spot Data Integration)
  • EPEX Spot (Net) (EPEX Spot Data Integration)
  • EPEX Spot Quantile (EPEX Spot Data Integration)

I'm with Tibber - but your Integration is much more reliable than Tibber Integration!!!
@heinemannj
Copy link
Author

Most simple prediction approach for EPEX Spot Data values after midnight:

  • Simply take the value from the day before

@heinemannj
Copy link
Author

Day ahead forecast by EMHass:

image

@heinemannj
Copy link
Author

heinemannj commented Jun 4, 2024 •
edited
Loading

Day ahead forecast by EMHass and custom template sensor finetuning 'EPEX Spot Quantile':

image

      - name: epex_spot_data_net_quantile_forcast
        unique_id: epex_spot_data_net_quantile_forcast
        #friendly_name: "epex_spot_data_net_quantile_forcast"
#        state_class: "total"
        state_class: measurement
#        unit_of_measurement: "ct/kWh"
        icon: mdi:eye
        state: >-
          {%- set var = namespace(data = []) %}

          {%- set var.current_price = states('sensor.epex_spot_data_net_price')|float %}
          {%- set var.net_price = state_attr('sensor.epex_spot_data_net_price', 'data') %}

          {# Use only EPEX Spot data starting with the actual interval #}
          {%- set var.actual_start_time = (now() - timedelta(minutes=(now().minute+1)))|as_timestamp %}

          {%- for record in var.net_price if record.start_time|as_timestamp >= var.actual_start_time %}
            {%- set var.data = var.data + [record.price_ct_per_kwh] %}
          {%- endfor %}

          {%- set var.min_price = var.data|min|float %}
          {%- set var.max_price = var.data|max|float %}
          {%- set var.quantile = ((var.current_price - var.min_price) / (var.max_price - var.min_price))|float %}
          {{ var.quantile }}
        attributes:
          data: >-
            {%- set var = namespace(data = []) %}
            {%- set var.result = [] %}

            {# EPEX Spot data for at least today and after 13:00 CET for tommorrow in addition #}
            {# - in 60 min interval #}
            {%- set var.net_price = state_attr('sensor.epex_spot_data_net_price', 'data') %}
            {%- set var.net_price_last_start_time = var.net_price|map(attribute='start_time')|list|last %}
            {%- set var.net_price_array_length = var.net_price|map(attribute='start_time')|list|length %}

            {# Solar Forecast for today and tomorrow #}
            {# - in 30 min interval #}
            {%- set var.solcast = state_attr('sensor.solcast_pv_forecast_prognose_heute', 'detailedForecast') %}
            {%- set var.solcast = var.solcast + state_attr('sensor.solcast_pv_forecast_prognose_fur_morgen', 'detailedForecast') %}

            {# Check if EPEX Spot data for tomorrow is already available #}
            {# - if not available add data from today as an quick and dirty forecast for tomorrow #}
            {# - will be overwritten after 13:00 CET with real EPEX Spot data #}
            {% if (var.net_price_last_start_time|as_datetime).day < now().day+1 %}

              {%- set var.net_price_forcast = var.net_price %}

              {% for i in range((var.net_price_array_length-24),var.net_price_array_length) %}
                {%- set var.net_price_forcast = var.net_price_forcast +
                  [{
                    'start_time': ((var.net_price[i].start_time)|as_timestamp + 86400)|timestamp_local,
                    'end_time': ((var.net_price[i].end_time)|as_timestamp + 86400)|timestamp_local,
                    'price_ct_per_kwh': var.net_price[i].price_ct_per_kwh
                  }] %}
              {% endfor %}

              {%- set var.net_price = var.net_price_forcast %}
            {% endif %}

            {# Use only EPEX Spot data starting with the actual interval #}
            {%- set var.actual_start_time = (now() - timedelta(minutes=(now().minute+1)))|as_timestamp %}

            {# Min/Max for quantile calculation #}
            {%- for record in var.net_price if record.start_time|as_timestamp >= var.actual_start_time%}
              {%- set var.data = var.data + [record.price_ct_per_kwh] %}
            {%- endfor %}
            {%- set var.min_price = var.data|min|float %}
            {%- set var.max_price = var.data|max|float %}

            {# Construct new array putting everthing together #}
            {%- for record in var.net_price if record.start_time|as_timestamp >= var.actual_start_time %}
              {%- set var.quantile = ((record.price_ct_per_kwh - var.min_price) / (var.max_price - var.min_price))|round(4) %}

              {# Optional: Estimation of EPEX Spot price level #}
              {% if record.price_ct_per_kwh < 25.0 %}
                {%- set var.price_level = 'Very Low (Green)' %}
              {% elif record.price_ct_per_kwh < 30.0 %}
                {%- set var.price_level = 'Low (Green)' %}
              {% elif record.price_ct_per_kwh < 40.0 %}
                  {%- set var.price_level = 'Neutral (Yellow)' %}
              {% elif record.price_ct_per_kwh < 45.0 %}
                {%- set var.price_level = 'High (Orange)' %}
              {% elif record.price_ct_per_kwh < 50.0 %}
                {%- set var.price_level = 'Very High (Red)' %}
              {% else %}
                {%- set var.price_level = 'Spike (Purple)' %}
              {% endif %}

              {# Split into 30 min interval #}
              {# Solar Forecast is already in in 30 min interval #}

              {# - 1. half hour #}
              {# - if clause is only necessary for the first iteration to deceide starting the var.result array with 0 or 30 min interval #}
              {% if record.start_time|as_timestamp >= (now()|as_timestamp - 1800) %}
                {%- set var.start_time = record.start_time %}
                {%- set var.end_time = (record.end_time|as_timestamp - 1800)|timestamp_local %}
                {%- set var.p_pv = 'N/A' %}

                {%- for record in var.solcast if record.period_start|as_timestamp == var.start_time|as_timestamp %}
                  {%- set var.p_pv = (record.pv_estimate * 1000)|round(0) %}
                {%- endfor %}

                {%- set var.result = var.result +
                  [{
                    'start_time': var.start_time|as_timestamp|timestamp_local,
                    'end_time': var.end_time|as_timestamp|timestamp_local,
                    'price_eur_per_kwh': (record.price_ct_per_kwh/100)|round(4),
                    'price_level': var.price_level,
                    'quantile': var.quantile,
                    'p_pv_forecast': var.p_pv,
                  }] %}
              {% endif %}

              {# - 2. half hour #}
              {%- set var.start_time = (record.start_time|as_timestamp + 1800)|timestamp_local %}
              {%- set var.end_time = record.end_time %}
              {%- set var.p_pv = 'N/A' %}

              {%- for record in var.solcast if record.period_start|as_timestamp == var.start_time|as_timestamp %}
                {%- set var.p_pv = (record.pv_estimate * 1000)|round(0) %}
              {%- endfor %}

              {%- set var.result = var.result +
                [{
                  'start_time': var.start_time|as_timestamp|timestamp_local,
                  'end_time': var.end_time|as_timestamp|timestamp_local,
                  'price_eur_per_kwh': (record.price_ct_per_kwh/100)|round(4),
                  'price_level': var.price_level,
                  'quantile': var.quantile,
                  'p_pv_forecast': var.p_pv,
                }] %}
            {%- endfor %}
            {{ var.result }}
        availability: >-
          {{ (states('sensor.epex_spot_data_net_price')|is_number)
              and (states('sensor.solcast_pv_forecast_prognose_heute')|is_number)
              and (states('sensor.solcast_pv_forecast_prognose_fur_morgen')|is_number) }}

Sensor output:

state_class: measurement
data:
  - start_time: "2024-06-04T19:00:00+02:00"
    end_time: "2024-06-04T19:30:00+02:00"
    price_eur_per_kwh: 0.41
    price_level: High (Orange)
    quantile: 0.7192
    p_pv_forecast: 1133
  - start_time: "2024-06-04T19:30:00+02:00"
    end_time: "2024-06-04T20:00:00+02:00"
    price_eur_per_kwh: 0.41
    price_level: High (Orange)
    quantile: 0.7192
    p_pv_forecast: 705
  - start_time: "2024-06-04T20:00:00+02:00"
    end_time: "2024-06-04T20:30:00+02:00"
    price_eur_per_kwh: 0.4799
    price_level: Very High (Red)
    quantile: 1
    p_pv_forecast: 354
  - start_time: "2024-06-04T20:30:00+02:00"
    end_time: "2024-06-04T21:00:00+02:00"
    price_eur_per_kwh: 0.4799
    price_level: Very High (Red)
    quantile: 1
    p_pv_forecast: 147
  - start_time: "2024-06-04T21:00:00+02:00"
    end_time: "2024-06-04T21:30:00+02:00"
    price_eur_per_kwh: 0.367
    price_level: Neutral (Yellow)
    quantile: 0.5467
    p_pv_forecast: 47
  - start_time: "2024-06-04T21:30:00+02:00"
    end_time: "2024-06-04T22:00:00+02:00"
    price_eur_per_kwh: 0.367
    price_level: Neutral (Yellow)
    quantile: 0.5467
    p_pv_forecast: 0
  - start_time: "2024-06-04T22:00:00+02:00"
    end_time: "2024-06-04T22:30:00+02:00"
    price_eur_per_kwh: 0.3273
    price_level: Neutral (Yellow)
    quantile: 0.3874
    p_pv_forecast: 0
  - start_time: "2024-06-04T22:30:00+02:00"
    end_time: "2024-06-04T23:00:00+02:00"
    price_eur_per_kwh: 0.3273
    price_level: Neutral (Yellow)
    quantile: 0.3874
    p_pv_forecast: 0
  - start_time: "2024-06-04T23:00:00+02:00"
    end_time: "2024-06-04T23:30:00+02:00"
    price_eur_per_kwh: 0.2898
    price_level: Low (Green)
    quantile: 0.2366
    p_pv_forecast: 0
  - start_time: "2024-06-04T23:30:00+02:00"
    end_time: "2024-06-05T00:00:00+02:00"
    price_eur_per_kwh: 0.2898
    price_level: Low (Green)
    quantile: 0.2366
    p_pv_forecast: 0
  - start_time: "2024-06-05T00:00:00+02:00"
    end_time: "2024-06-05T00:30:00+02:00"
    price_eur_per_kwh: 0.2673
    price_level: Low (Green)
    quantile: 0.146
    p_pv_forecast: 0
  - start_time: "2024-06-05T00:30:00+02:00"
    end_time: "2024-06-05T01:00:00+02:00"
    price_eur_per_kwh: 0.2673
    price_level: Low (Green)
    quantile: 0.146
    p_pv_forecast: 0
  - start_time: "2024-06-05T01:00:00+02:00"
    end_time: "2024-06-05T01:30:00+02:00"
    price_eur_per_kwh: 0.2519
    price_level: Low (Green)
    quantile: 0.0843
    p_pv_forecast: 0
  - start_time: "2024-06-05T01:30:00+02:00"
    end_time: "2024-06-05T02:00:00+02:00"
    price_eur_per_kwh: 0.2519
    price_level: Low (Green)
    quantile: 0.0843
    p_pv_forecast: 0
  - start_time: "2024-06-05T02:00:00+02:00"
    end_time: "2024-06-05T02:30:00+02:00"
    price_eur_per_kwh: 0.2526
    price_level: Low (Green)
    quantile: 0.0872
    p_pv_forecast: 0
  - start_time: "2024-06-05T02:30:00+02:00"
    end_time: "2024-06-05T03:00:00+02:00"
    price_eur_per_kwh: 0.2526
    price_level: Low (Green)
    quantile: 0.0872
    p_pv_forecast: 0
  - start_time: "2024-06-05T03:00:00+02:00"
    end_time: "2024-06-05T03:30:00+02:00"
    price_eur_per_kwh: 0.2592
    price_level: Low (Green)
    quantile: 0.1135
    p_pv_forecast: 0
  - start_time: "2024-06-05T03:30:00+02:00"
    end_time: "2024-06-05T04:00:00+02:00"
    price_eur_per_kwh: 0.2592
    price_level: Low (Green)
    quantile: 0.1135
    p_pv_forecast: 0
  - start_time: "2024-06-05T04:00:00+02:00"
    end_time: "2024-06-05T04:30:00+02:00"
    price_eur_per_kwh: 0.2568
    price_level: Low (Green)
    quantile: 0.1041
    p_pv_forecast: 0
  - start_time: "2024-06-05T04:30:00+02:00"
    end_time: "2024-06-05T05:00:00+02:00"
    price_eur_per_kwh: 0.2568
    price_level: Low (Green)
    quantile: 0.1041
    p_pv_forecast: 0
  - start_time: "2024-06-05T05:00:00+02:00"
    end_time: "2024-06-05T05:30:00+02:00"
    price_eur_per_kwh: 0.2843
    price_level: Low (Green)
    quantile: 0.2146
    p_pv_forecast: 0
  - start_time: "2024-06-05T05:30:00+02:00"
    end_time: "2024-06-05T06:00:00+02:00"
    price_eur_per_kwh: 0.2843
    price_level: Low (Green)
    quantile: 0.2146
    p_pv_forecast: 26
  - start_time: "2024-06-05T06:00:00+02:00"
    end_time: "2024-06-05T06:30:00+02:00"
    price_eur_per_kwh: 0.3228
    price_level: Neutral (Yellow)
    quantile: 0.3692
    p_pv_forecast: 93
  - start_time: "2024-06-05T06:30:00+02:00"
    end_time: "2024-06-05T07:00:00+02:00"
    price_eur_per_kwh: 0.3228
    price_level: Neutral (Yellow)
    quantile: 0.3692
    p_pv_forecast: 163
  - start_time: "2024-06-05T07:00:00+02:00"
    end_time: "2024-06-05T07:30:00+02:00"
    price_eur_per_kwh: 0.3695
    price_level: Neutral (Yellow)
    quantile: 0.5567
    p_pv_forecast: 278
  - start_time: "2024-06-05T07:30:00+02:00"
    end_time: "2024-06-05T08:00:00+02:00"
    price_eur_per_kwh: 0.3695
    price_level: Neutral (Yellow)
    quantile: 0.5567
    p_pv_forecast: 563
  - start_time: "2024-06-05T08:00:00+02:00"
    end_time: "2024-06-05T08:30:00+02:00"
    price_eur_per_kwh: 0.334
    price_level: Neutral (Yellow)
    quantile: 0.414
    p_pv_forecast: 927
  - start_time: "2024-06-05T08:30:00+02:00"
    end_time: "2024-06-05T09:00:00+02:00"
    price_eur_per_kwh: 0.334
    price_level: Neutral (Yellow)
    quantile: 0.414
    p_pv_forecast: 1484
  - start_time: "2024-06-05T09:00:00+02:00"
    end_time: "2024-06-05T09:30:00+02:00"
    price_eur_per_kwh: 0.2998
    price_level: Low (Green)
    quantile: 0.2765
    p_pv_forecast: 2149
  - start_time: "2024-06-05T09:30:00+02:00"
    end_time: "2024-06-05T10:00:00+02:00"
    price_eur_per_kwh: 0.2998
    price_level: Low (Green)
    quantile: 0.2765
    p_pv_forecast: 2673
  - start_time: "2024-06-05T10:00:00+02:00"
    end_time: "2024-06-05T10:30:00+02:00"
    price_eur_per_kwh: 0.2828
    price_level: Low (Green)
    quantile: 0.2085
    p_pv_forecast: 2987
  - start_time: "2024-06-05T10:30:00+02:00"
    end_time: "2024-06-05T11:00:00+02:00"
    price_eur_per_kwh: 0.2828
    price_level: Low (Green)
    quantile: 0.2085
    p_pv_forecast: 3062
  - start_time: "2024-06-05T11:00:00+02:00"
    end_time: "2024-06-05T11:30:00+02:00"
    price_eur_per_kwh: 0.2734
    price_level: Low (Green)
    quantile: 0.1708
    p_pv_forecast: 3316
  - start_time: "2024-06-05T11:30:00+02:00"
    end_time: "2024-06-05T12:00:00+02:00"
    price_eur_per_kwh: 0.2734
    price_level: Low (Green)
    quantile: 0.1708
    p_pv_forecast: 3745
  - start_time: "2024-06-05T12:00:00+02:00"
    end_time: "2024-06-05T12:30:00+02:00"
    price_eur_per_kwh: 0.2572
    price_level: Low (Green)
    quantile: 0.1056
    p_pv_forecast: 4063
  - start_time: "2024-06-05T12:30:00+02:00"
    end_time: "2024-06-05T13:00:00+02:00"
    price_eur_per_kwh: 0.2572
    price_level: Low (Green)
    quantile: 0.1056
    p_pv_forecast: 4244
  - start_time: "2024-06-05T13:00:00+02:00"
    end_time: "2024-06-05T13:30:00+02:00"
    price_eur_per_kwh: 0.2355
    price_level: Very Low (Green)
    quantile: 0.0185
    p_pv_forecast: 4288
  - start_time: "2024-06-05T13:30:00+02:00"
    end_time: "2024-06-05T14:00:00+02:00"
    price_eur_per_kwh: 0.2355
    price_level: Very Low (Green)
    quantile: 0.0185
    p_pv_forecast: 4264
  - start_time: "2024-06-05T14:00:00+02:00"
    end_time: "2024-06-05T14:30:00+02:00"
    price_eur_per_kwh: 0.2309
    price_level: Very Low (Green)
    quantile: 0
    p_pv_forecast: 4195
  - start_time: "2024-06-05T14:30:00+02:00"
    end_time: "2024-06-05T15:00:00+02:00"
    price_eur_per_kwh: 0.2309
    price_level: Very Low (Green)
    quantile: 0
    p_pv_forecast: 4105
  - start_time: "2024-06-05T15:00:00+02:00"
    end_time: "2024-06-05T15:30:00+02:00"
    price_eur_per_kwh: 0.2354
    price_level: Very Low (Green)
    quantile: 0.018
    p_pv_forecast: 3841
  - start_time: "2024-06-05T15:30:00+02:00"
    end_time: "2024-06-05T16:00:00+02:00"
    price_eur_per_kwh: 0.2354
    price_level: Very Low (Green)
    quantile: 0.018
    p_pv_forecast: 3442
  - start_time: "2024-06-05T16:00:00+02:00"
    end_time: "2024-06-05T16:30:00+02:00"
    price_eur_per_kwh: 0.2741
    price_level: Low (Green)
    quantile: 0.1734
    p_pv_forecast: 2946
  - start_time: "2024-06-05T16:30:00+02:00"
    end_time: "2024-06-05T17:00:00+02:00"
    price_eur_per_kwh: 0.2741
    price_level: Low (Green)
    quantile: 0.1734
    p_pv_forecast: 2365
  - start_time: "2024-06-05T17:00:00+02:00"
    end_time: "2024-06-05T17:30:00+02:00"
    price_eur_per_kwh: 0.2953
    price_level: Low (Green)
    quantile: 0.2587
    p_pv_forecast: 1700
  - start_time: "2024-06-05T17:30:00+02:00"
    end_time: "2024-06-05T18:00:00+02:00"
    price_eur_per_kwh: 0.2953
    price_level: Low (Green)
    quantile: 0.2587
    p_pv_forecast: 1121
  - start_time: "2024-06-05T18:00:00+02:00"
    end_time: "2024-06-05T18:30:00+02:00"
    price_eur_per_kwh: 0.3247
    price_level: Neutral (Yellow)
    quantile: 0.3766
    p_pv_forecast: 687
  - start_time: "2024-06-05T18:30:00+02:00"
    end_time: "2024-06-05T19:00:00+02:00"
    price_eur_per_kwh: 0.3247
    price_level: Neutral (Yellow)
    quantile: 0.3766
    p_pv_forecast: 570
  - start_time: "2024-06-05T19:00:00+02:00"
    end_time: "2024-06-05T19:30:00+02:00"
    price_eur_per_kwh: 0.3507
    price_level: Neutral (Yellow)
    quantile: 0.4811
    p_pv_forecast: 459
  - start_time: "2024-06-05T19:30:00+02:00"
    end_time: "2024-06-05T20:00:00+02:00"
    price_eur_per_kwh: 0.3507
    price_level: Neutral (Yellow)
    quantile: 0.4811
    p_pv_forecast: 335
  - start_time: "2024-06-05T20:00:00+02:00"
    end_time: "2024-06-05T20:30:00+02:00"
    price_eur_per_kwh: 0.3851
    price_level: Neutral (Yellow)
    quantile: 0.6194
    p_pv_forecast: 243
  - start_time: "2024-06-05T20:30:00+02:00"
    end_time: "2024-06-05T21:00:00+02:00"
    price_eur_per_kwh: 0.3851
    price_level: Neutral (Yellow)
    quantile: 0.6194
    p_pv_forecast: 174
  - start_time: "2024-06-05T21:00:00+02:00"
    end_time: "2024-06-05T21:30:00+02:00"
    price_eur_per_kwh: 0.3876
    price_level: Neutral (Yellow)
    quantile: 0.6296
    p_pv_forecast: 82
  - start_time: "2024-06-05T21:30:00+02:00"
    end_time: "2024-06-05T22:00:00+02:00"
    price_eur_per_kwh: 0.3876
    price_level: Neutral (Yellow)
    quantile: 0.6296
    p_pv_forecast: 4
  - start_time: "2024-06-05T22:00:00+02:00"
    end_time: "2024-06-05T22:30:00+02:00"
    price_eur_per_kwh: 0.368
    price_level: Neutral (Yellow)
    quantile: 0.5505
    p_pv_forecast: 0
  - start_time: "2024-06-05T22:30:00+02:00"
    end_time: "2024-06-05T23:00:00+02:00"
    price_eur_per_kwh: 0.368
    price_level: Neutral (Yellow)
    quantile: 0.5505
    p_pv_forecast: 0
  - start_time: "2024-06-05T23:00:00+02:00"
    end_time: "2024-06-05T23:30:00+02:00"
    price_eur_per_kwh: 0.3257
    price_level: Neutral (Yellow)
    quantile: 0.3809
    p_pv_forecast: 0
  - start_time: "2024-06-05T23:30:00+02:00"
    end_time: "2024-06-06T00:00:00+02:00"
    price_eur_per_kwh: 0.3257
    price_level: Neutral (Yellow)
    quantile: 0.3809
    p_pv_forecast: 0
icon: mdi:eye
friendly_name: epex_spot_data_net_quantile_forcast

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@heinemannj