Rolling horizon #1327

abelsiqueira · 2025-09-02T10:02:48Z

Full PR with all rolling horizon commits (except for docs, see #1387 for that).

Changes:

Change rep_period profiles to handle rolling horizon
- Implement ProfileWithRollingHorizon and make profiles.rep_period use that instead of Vector{Float64}.
- Change _profile_aggregate to dispatch on Vector{Float64} and ProfileWithRollingHorizon.
- These should allow the rolling horizon variables to be aggregated the same way as the normal profiles.
Implement high-level run_rolling_horizon function with placeholders
- Implement run_rolling_horizon with the main idea. Use placeholders to specify places where a more complicated function needs to be called.
- Update the EnergyProblem structure to hold an inner EnergyProblem
  for the rolling horizon model.
Create rolling horizon parameters
- Add ParametricOptInterface.
- Implement adding the rolling horizon parameters in src/rolling-horizon/create.jl.
- Update create_model to handle the rolling horizon parameters and wrap the solver in POI.Optimizer if it's rolling horizon.
Implement the update functions for rolling horizon and add them
to the run_rolling_horizon function.
Test adding rolling horizon parameters
Add functions for validation of data preparation for rolling horizon
- Add validation of input in the rolling horizon function.
- Add function to prepare the input to get in the rolling horizon and to get out.
- Add function to save the solution in the relevant tables.
Add unit tests for rolling horizon
- Add tests of the various parts of rolling horizon.
Update the storage constraint to use the rolling initial storage level
- Update src/constraints/storage.jl to use the initial_storage_level stored in the rolling horizon.
- Add test for the rolling horizon objective values to control future changes.

Related issues

Closes #1365

Checklist

I am following the contributing guidelines
Tests are passing
Lint workflow is passing
Docs were updated and workflow is passing

github-actions · 2025-09-02T10:03:00Z

🤖 CompareMPS report

✅ MPS files match

github-actions · 2025-09-02T10:46:49Z

Benchmark Results

	`eb442c5`...	`e3bb910`...	`eb442c5`... / `e3bb910`...
energy_problem/create_model	42.3 ± 2.1 s	41.5 ± 6.2 s	1.02 ± 0.16
energy_problem/input_and_constructor	19.4 ± 0.11 s	19.3 ± 0.13 s	1 ± 0.0089
time_to_load	2.26 ± 0.023 s	2.27 ± 0.018 s	0.997 ± 0.013

	`eb442c5`...	`e3bb910`...	`eb442c5`... / `e3bb910`...
energy_problem/create_model	0.223 G allocs: 16.6 GB	0.253 G allocs: 17.3 GB	0.959
energy_problem/input_and_constructor	0.0476 G allocs: 1.77 GB	0.0475 G allocs: 1.77 GB	1
time_to_load	0.151 k allocs: 11.5 kB	0.151 k allocs: 11.5 kB	1

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

codecov · 2025-09-02T10:51:00Z

Codecov Report

❌ Patch coverage is 97.29730% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.47%. Comparing base (eb442c5) to head (e3bb910).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
src/rolling-horizon/rolling-horizon.jl	90.47%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1327      +/-   ##
==========================================
- Coverage   98.61%   98.47%   -0.14%     
==========================================
  Files          38       42       +4     
  Lines        1368     1511     +143     
==========================================
+ Hits         1349     1488     +139     
- Misses         19       23       +4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

abelsiqueira

@datejada, I've created a first draft of the rolling horizon implementation. I don't expect you to read everything, but I've marked a few interesting places, and we can discuss and go over the details if you have time.

I'm trying something with the profiles which hides the complexity in _profile_aggregate. This avoids changing the code everywhere and manually handling rolling horizon inside the model. I am still not sure whether this is efficient or not, but it's what I thought of so far.

I also have the following questions:

Are there other places that will require rolling horizon? Are we trying to make it more generic than this? If so, how generic?
I am not sure how to validate that the rolling horizon is working. I've created some random problem that possibly doesn't make a lot of sense. Do you have some Tulipa-compatible test? If not, maybe we can go over the TulipaBuilder implementation and see what comes out of it.

abelsiqueira · 2025-09-02T10:53:56Z

src/model-preparation.jl

    return
 end

 function prepare_profiles_structure(connection)


@datejada, here I changed the defaults profiles structure to hold in parallel the values (as before) and the rolling horizon parameters (initially empty)

abelsiqueira · 2025-09-02T10:54:51Z

src/rolling-horizon.jl

+
+function add_rolling_horizon_parameters!(connection, model, variables, profiles, window_length)
+    # Profiles
+    for (_, profile_object) in profiles.rep_period


@datejada, here we initialise the profiles' rolling horizon variables

abelsiqueira · 2025-09-02T10:55:50Z

src/rolling-horizon.jl

+            @variable(model, [1:window_length] in JuMP.Parameter(0.0))
+    end
+
+    # initial_storage_level


@datejada, for the initial storage level, I have to create a table just for that.

abelsiqueira · 2025-09-02T10:57:02Z

src/rolling-horizon.jl

+    return
+end
+
+function update_rolling_horizon_profiles!(profiles, window_start, window_end)


@datejada, the straightforward update of the rolling horizon parameters

abelsiqueira · 2025-09-02T11:41:34Z

src/rolling-horizon.jl

+    return
+end
+
+function update_initial_storage_level!(param_initial_storage_level::TulipaVariable, connection)


@datejada, the not-so-straightforward update of the storage level.

abelsiqueira · 2025-09-02T11:45:25Z

src/utils.jl

+    profile_object = profiles[tuple_key]
+
+    # Rolling horizon is inferred by the existence and non-emptyness of rolling_horizon_variables
+    is_rolling_horizon =
+        hasproperty(profile_object, :rolling_horizon_variables) &&
+        length(profile_object.rolling_horizon_variables) > 0
+
+    if is_rolling_horizon
+        profile_value = profile_object.rolling_horizon_variables
+        return agg_function(skipmissing(profile_value[time_block]))
+    else
+        profile_value = profile_object.values
+        return agg_function(skipmissing(profile_value[time_block]))
+    end


@datejada, here's the trick with the profile. If is_rolling_horizon, we use the rolling horizon variables, otherwise use the actual values.

abelsiqueira · 2025-09-02T12:26:37Z

src/model-preparation.jl


    # Completely separate calculation for inflows_profile_aggregation
    if is_storage_level
+        # TODO: Fix this for rolling horizon


@datejada, missed this. This is the one place that uses the profiles_rep_periods explicitly, which will probably require refactoring

datejada · 2025-09-04T09:31:31Z

test/test-case-studies.jl

+    # Rolling horizon
+    energy_problem =
+        TulipaEnergyModel.run_rolling_horizon(connection, 24 * 7, 48 * 7; show_log = false)
+    # @test energy_problem.objective_value ≈ 10000.0 atol = 1e-5


We need to test the objective function of each one of the windows instead of the overall objective function

datejada · 2025-09-04T10:03:31Z

@datejada, I've created a first draft of the rolling horizon implementation. I don't expect you to read everything, but I've marked a few interesting places, and we can discuss and go over the details if you have time.

I'm trying something with the profiles which hides the complexity in _profile_aggregate. This avoids changing the code everywhere and manually handling rolling horizon inside the model. I am still not sure whether this is efficient or not, but it's what I thought of so far.

I also have the following questions:

Are there other places that will require rolling horizon? Are we trying to make it more generic than this? If so, how generic?

I am not sure how to validate that the rolling horizon is working. I've created some random problem that possibly doesn't make a lot of sense. Do you have some Tulipa-compatible test? If not, maybe we can go over the TulipaBuilder implementation and see what comes out of it.

@abelsiqueira, thanks for the PR. It is starting to take shape 😃

I like the approach with the profiles. Is it general enough to add the cases in issue Add profiles to the more parameters to consider time variations in a year #1269? I am trying to see how this works with the changes we foresee in the model.
(TL;DR: we need a generic implementation for the scalar parameters).
The initial storage level looks okay, but I am afraid we need something more general here. The big picture is that more "Float" parameters can be updated in the model during the rolling horizon. Some of those parameters will be boundary conditions when k=1 (similar to the initial storage level). We don't have them yet in the model, but we need to add them soon, because they are relevant for the rolling_horizon. For instance, the initial flow for the ramping constraints Add initial flow parameter for ramping constraints #1328, for a whole year, it is not that relevant to this initial condition, but if we solve weekly windows, then it becomes relevant since then you are not including this condition 52 times 🤷 There a couple more of this "initial/boundary condition" parameters that will come from the developments in the unit commitment we have with the TU-Delft students.
This is how it think we should test the case study of the rolling horizon (but open to discussion, with the rest of the team): https://github.com/TulipaEnergy/TulipaEnergyModel.jl/pull/1327/files#r2321431664
The last point leads me to question how the results are handled. For debugging, it would be nice to have the results of each optimization per window, and then one on which we aggregate the final result, similar to here: https://jump.dev/JuMP.jl/stable/tutorials/algorithms/rolling_horizon/#Solution
It will be nice to have the same example that is in the JuMP documentation (https://jump.dev/JuMP.jl/stable/tutorials/algorithms/rolling_horizon/), which could be the example for rolling horizon (or an extra one, this feature might need a couple of examples). What do you think?

Thanks again 😄 I like the changes

abelsiqueira · 2025-09-09T10:58:52Z

Thanks for the initial review, @datejada. Some comments:

I don't know what issue Add profiles to the more parameters to consider time variations in a year #1269 will require. It's not clear what would need to be implemented to handle that. That said, from working on the inflows changes is that we need to use _profile_aggregate. I.e., the aggregation cannot happen at SQL level, it must happen in Julia level. If that's the case, it should be fine.
There are currently two categories of "Rolled Parameters":
- Profiles (starting value and updates clearly defined from the window)
- Starting values (starting value is the same as non-rolling, and the rolling update uses the solution from a window as new starting value)
  Are you proposing a new category, or do these scalar parameters fall into these categories?
Makes sense. Probably some individual components can be tested as well. It might not be enough to capture everything, like a PR that only passes because it's not covered in the rolling horizon case study, as we've seen with other features.
Indeed. I was considering whether it made sense to attach all solutions to existing variables, but it will probably be messy. Instead, maybe we can create new tables for the rolling horizon windows with ids and some tables for solutions.
It would be nice, but I have no idea how to translate that into TEM format. If you have the time to help translate that into TEM, it would be great. Maybe we can sit together and write a TulipaBuilder script for it, if you have some time.

Thanks again for the review

datejada · 2025-09-09T14:06:16Z

Thanks for the initial review, @datejada. Some comments:

I don't know what issue Add profiles to the more parameters to consider time variations in a year #1269 will require. It's not clear what would need to be implemented to handle that. That said, from working on the inflows changes is that we need to use _profile_aggregate. I.e., the aggregation cannot happen at SQL level, it must happen in Julia level. If that's the case, it should be fine.

There are currently two categories of "Rolled Parameters":

Profiles (starting value and updates clearly defined from the window)

Starting values (starting value is the same as non-rolling, and the rolling update uses the solution from a window as new starting value)
Are you proposing a new category, or do these scalar parameters fall into these categories?

Makes sense. Probably some individual components can be tested as well. It might not be enough to capture everything, like a PR that only passes because it's not covered in the rolling horizon case study, as we've seen with other features.

Indeed. I was considering whether it made sense to attach all solutions to existing variables, but it will probably be messy. Instead, maybe we can create new tables for the rolling horizon windows with ids and some tables for solutions.

It would be nice, but I have no idea how to translate that into TEM format. If you have the time to help translate that into TEM, it would be great. Maybe we can sit together and write a TulipaBuilder script for it, if you have some time.

Thanks again for the review

Great! Thanks for the reply. So:

I envision using the same approach as in the inflows for the Add profiles to the more parameters to consider time variations in a year #1269, so it should be fine, then.
I also see two categories of things to update: profiles and starting values. The new parameter is commented before it falls into the starting values category. @g-moralesespana or @gnawin, can you think of something else we are going to update in the rolling horizon? I don't want to miss something, so I ask your opinion.
Agree.
I like that approach. @g-moralesespana and @gnawin: what do you think? Should we always save the results per window? Or should we have an option to store them or not?
And how do the results from the rolling horizon work in AIMMS? I remember that in the other model I used to work in, the rolling horizon results were not kept by window by default, and debugging (or explaining a result) was complicated without them (so, you normally had to run again, saving the results per window). Saving the results per window sounds like a better approach, but it might involve too much data. I would like to know your opinion on this as well.
Sure thing! We can sit on the next Tulipa day and map it.

Thanks!

github-actions · 2025-10-13T09:10:59Z

📝 Check the documentation preview: https://tulipaenergy.github.io/TulipaEnergyModel.jl/previews/PR1327

Implement ProfileWithRollingHorizon and make profiles.rep_period use that instead of Vector{Float64}. Change _profile_aggregate to dispatch on Vector{Float64} and ProfileWithRollingHorizon. These should allow the rolling horizon variables to be aggregated the same way as the normal profiles. Part of #1365

Implement run_rolling_horizon with the main idea. Use placeholders to specify places where a more complicated function needs to be called. Update the EnergyProblem structure to hold an inner EnergyProblem for the rolling horizon model. Part of #1365

Add ParametricOptInterface. Implement adding the rolling horizon parameters in src/rolling-horizon/create.jl. Update create_model to handle the rolling horizon parameters and wrap the solver in POI.Optimizer if it's rolling horizon. Part of #1365

Implement the update functions for rolling horizon and add them to the run_rolling_horizon function. Part of #1365

Part of #1365

Add validation of input in the rolling horizon function. Add function to prepare the input to get in the rolling horizon and to get out. Add function to save the solution in the relevant tables. Part of #1365

Add tests of the various parts of rolling horizon. Part of #1365

Update src/constraints/storage.jl to use the initial_storage_level stored in the rolling horizon. Add test for the rolling horizon objective values to control future changes. Part of #1365

abelsiqueira · 2025-10-15T10:08:23Z

@datejada, I've made the updates I commented yesterday, and I've squashed them in the existing commits. Now you can review them one commit at a time.

The is a missing note that I mentioned from the docs: the solution of thermal and battery in RH vs no-RH were not equal. I though it was because of the ordering of the update of the initial value, but I've the code update and I also noticed that in the rolling horizon, we have only 1 storage asset, so there is no error in the ordering, because we have only 1 value.
Investigating a bit, I noticed that if I compute all outgoing = thermal+solar+battery and then compare RH vs no-RH, then they're equal: https://tulipaenergy.github.io/TulipaEnergyModel.jl/previews/PR1387/10-tutorials/31-rolling-horizon/#Error

My understanding is that there isn't actually an issue, but what actually happens is that there are multiple solutions. One where you use the stored energy at time t, and the other where you use the stored energy at time t + 1. Does that make sense? I don't know if there is a parameter for "stored energy decay" or "cost of storing energy", or something like that, that we can use to try to force the uniqueness of the solution. Or maybe you have a different idea of why they're different.

datejada

Reviewed in-site after comments included from previous round.

abelsiqueira added the benchmark PR only - Run benchmark on PR label Sep 2, 2025

abelsiqueira commented Sep 2, 2025

View reviewed changes

datejada reviewed Sep 4, 2025

View reviewed changes

abelsiqueira force-pushed the rolling-horizon-playground branch from d1d2b3a to 4433c3c Compare September 23, 2025 18:07

abelsiqueira force-pushed the rolling-horizon-playground branch from 9458b70 to 16b2a2a Compare October 13, 2025 09:06

abelsiqueira force-pushed the rolling-horizon-playground branch from 16b2a2a to 41a0422 Compare October 13, 2025 09:12

abelsiqueira changed the title ~~[WIP] Rolling horizon~~ Rolling horizon Oct 13, 2025

This was referenced Oct 13, 2025

Change rep_period profiles to handle rolling horizon #1375

Closed

Modify storage constraint to compute inflows using _profile_aggregate #1371

Merged

abelsiqueira marked this pull request as ready for review October 14, 2025 09:07

abelsiqueira mentioned this pull request Oct 14, 2025

Add solution column when creating variables #1391

Closed

abelsiqueira added 8 commits October 15, 2025 11:31

Create rolling horizon parameters

7add735

Add ParametricOptInterface. Implement adding the rolling horizon parameters in src/rolling-horizon/create.jl. Update create_model to handle the rolling horizon parameters and wrap the solver in POI.Optimizer if it's rolling horizon. Part of #1365

Implement update of rolling horizon parameters

e9c8ff6

Implement the update functions for rolling horizon and add them to the run_rolling_horizon function. Part of #1365

Test adding rolling horizon parameters

9acdc3b

Part of #1365

Add functions for validation and data preparation for rolling horizon

aef2825

Add validation of input in the rolling horizon function. Add function to prepare the input to get in the rolling horizon and to get out. Add function to save the solution in the relevant tables. Part of #1365

Add unit tests for rolling horizon

0ee290b

Add tests of the various parts of rolling horizon. Part of #1365

Update the storage constraint to use the rolling initial storage level

e3bb910

Update src/constraints/storage.jl to use the initial_storage_level stored in the rolling horizon. Add test for the rolling horizon objective values to control future changes. Part of #1365

abelsiqueira force-pushed the rolling-horizon-playground branch from 41a0422 to e3bb910 Compare October 15, 2025 09:32

abelsiqueira requested a review from datejada October 15, 2025 09:57

datejada approved these changes Oct 20, 2025

View reviewed changes

abelsiqueira mentioned this pull request Oct 20, 2025

Save dual variable solution in rolling horizon #1395

Closed

datejada merged commit 0477fde into main Oct 20, 2025
9 of 10 checks passed

datejada deleted the rolling-horizon-playground branch October 20, 2025 14:27

Rolling horizon #1327

Rolling horizon #1327

Uh oh!

Conversation

abelsiqueira commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related issues

Checklist

Uh oh!

github-actions bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🤖 CompareMPS report

✅ MPS files match

Uh oh!

github-actions bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark Results

Benchmark Plots

Uh oh!

codecov bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

abelsiqueira left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

datejada commented Sep 4, 2025

Uh oh!

abelsiqueira commented Sep 9, 2025

Uh oh!

datejada commented Sep 9, 2025

Uh oh!

github-actions bot commented Oct 13, 2025

Uh oh!

abelsiqueira commented Oct 15, 2025

Uh oh!

datejada left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

abelsiqueira commented Sep 2, 2025 •

edited

Loading

github-actions bot commented Sep 2, 2025 •

edited

Loading

github-actions bot commented Sep 2, 2025 •

edited

Loading

codecov bot commented Sep 2, 2025 •

edited

Loading