Skip to content

Commit cf96046

Browse files
JBrisJBris
committed
Adding optimisation features and steps
1 parent 8adbffc commit cf96046

12 files changed

Lines changed: 346 additions & 25 deletions

‎behave.ini‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
[behave]
2+
paths = features
3+
tags = not @skip
4+
logging_level = INFO
5+
stop = False
6+
show_skipped = True

‎data/ground_truth.json‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"TSUM1": 255,
3+
"TSUM2": 1400,
4+
"TBASEM": 3.0,
5+
"TSUMEM": 170.0,
6+
"TEFFMX": 18.0,
7+
"SPAN": 37,
8+
"TDWI": 75,
9+
"RGRLAI": 0.016,
10+
"Q10": 2.0
11+
}

‎data/synthetic_test_data.csv‎

Lines changed: 144 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/source/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ For our community code of conduct, please also view `CODE_OF_CONDUCT.md. <https:
2222
:maxdepth: 1
2323
:caption: Contents:
2424

25+
tutorials/quickstart.rst
2526
api_reference/index.rst
2627
changelogs/changelog.rst
2728
community/index.rst

‎docs/source/tutorials/quickstart.rst‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
Quickstart
2+
==========
3+
4+
First, run the following code to install all dependencies:
5+
6+
.. code-block:: bash
7+
8+
poetry install --no-interaction
9+
10+
Secondly, execute the following to run the behaviour tests:
11+
12+
.. code-block:: bash
13+
14+
behave --verbose --summary
15+
16+
Python Environment
17+
==================
18+
19+
behave
20+
------
21+
22+
`The behave package was used to implement all BDD tests. <https://behave.readthedocs.io/en/stable/index.html>`_
23+
These tests have been written using Gherkin syntax. `For more information on Gherkin, click here. <https://cucumber.io/docs/gherkin/reference/>`_
24+
25+
* Feature files containing human-readable Gherkin can be found here: :file:`features`
26+
* The implementation of the scenario steps can be found here: :file:`features/steps`
27+
28+
Data
29+
====
30+
31+
* The potato data were originally retrieved from here: `https://github.com/ajwdewit/pcse_notebooks/tree/master/data`_

‎features/environment.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
1+
import optuna
12
from behave.model import Feature, Scenario
23
from behave.runner import Context
34

5+
optuna.logging.set_verbosity(optuna.logging.WARNING)
6+
47

58
def before_feature(context: Context, feature: Feature) -> None:
69
pass

‎features/optimisation.feature‎

Lines changed: 21 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,21 @@
11
Feature: Calibration of WOFOST potato using black-box optimisation
2-
Because crop model parameters are uncertain
2+
Because potato crop model parameters are uncertain
33
We want to calibrate WOFOST by exploring the parameter space with black-box optimisation
4+
And minimising the discrepancy for end-of-season LAI and TWSO
45
So potato crop development and growth is both accurate and well-constrained
56

67
Background:
78
Given we are using WOFOST site data with a WAV of "50.0"
89
And we are using default soil parameter values
910
And we are using crop data stored in the "data" directory
1011
And our state variables are "LAI and TWSO"
12+
And we are using "mean squared error" as our error metric
13+
And we are using observed data from the "data/synthetic_test_data.csv" file
14+
And we are using ground truth data from the "data/ground_truth.json" file
1115
And the following parameter specification is used for calibration:
1216
| name | description | range | distribution | type |
1317
| TSUM1 | Temperature sum from emergence to anthesis | 150, 280 | uniform | continuous |
14-
| TSUM2 | Temperature sum from anthesis to maturity | 1400, 2100 | uniform | continuous |
18+
| TSUM2 | Temperature sum from anthesis to maturity | 1550, 2100 | uniform | continuous |
1519
| TBASEM | Base temperature for emergence | 2, 4 | uniform | continuous |
1620
| TSUMEM | Temperature sum required for crop emergence | 170, 255 | uniform | continuous |
1721
| TEFFMX | Maximum effective temperature for emergence | 18, 32 | uniform | continuous |
@@ -20,11 +24,19 @@ Feature: Calibration of WOFOST potato using black-box optimisation
2024
| RGRLAI | Relative growth rate of leaf area index | 0.008, 0.02 | uniform | continuous |
2125
| Q10 | Temperature response factor for respiration (Q10 coefficient) | 2, 3 | uniform | continuous |
2226
And the following specification is used for the calibration procedure:
23-
| name | description | value |
24-
| experiment_name | The name of the current experiment | WOFOST optimisation |
25-
| n_jobs | The number of simulations to run in parallel | -1 |
27+
| name | description | value |
28+
| experiment_name | The name of the current experiment | WOFOST optimisation |
29+
| n_jobs | The number of simulations to run in parallel | -1 |
30+
| random_seed | The random seed for replicability | 100 |
2631

27-
Scenario: run a simple test
28-
Given we have behave installed
29-
When we implement a test
30-
Then behave will test it for us!
32+
@netherlands @optimisation
33+
Scenario: Black-box optimisation of WOFOST potato for end-of-season LAI and TWSO in Limburg, Netherlands
34+
Given we are using NASA weather data with a latitude of "51" and a longitude of "5"
35+
And we are using agronomy management data in the "data/potato_netherlands_2021.agro" file
36+
When we execute an optimisation procedure using the "TPES" method and the "Optuna" library with "5" iterations
37+
38+
@india @optimisation
39+
Scenario: Black-box optimisation of WOFOST potato for end-of-season LAI and TWSO in Gujarat, India
40+
Given we are using NASA weather data with a latitude of "23" and a longitude of "73"
41+
And we are using agronomy management data in the "data/potato_india_2021.agro" file
42+
When we execute an optimisation procedure using the "TPES" method and the "Optuna" library with "5" iterations

‎features/sensitivity.feature‎

Lines changed: 21 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
Feature: Sobol sensitivity analysis for WOFOST potato
2-
Because parameter uncertainty drives crop development and growth
2+
Because parameter uncertainty drives potato crop development and growth
33
We want to quantify global sensitivities using Sobol indices
4-
So we can identify influential parameters for potato LAI and TWSO
4+
So we can identify influential parameters for end-of-season LAI and TWSO
55

66
Background:
77
Given we are using WOFOST site data with a WAV of "50.0"
@@ -23,17 +23,30 @@ Feature: Sobol sensitivity analysis for WOFOST potato
2323
| name | description | value |
2424
| experiment_name | The name of the current experiment | WOFOST sensitivity analysis |
2525
| n_jobs | The number of simulations to run in parallel | 10 |
26+
| random_seed | The random seed for replicability | 100 |
2627

2728
@netherlands @sensitivity
28-
Scenario: Sobol sensitivity analysis for LAI and TWSO in Limburg, Netherlands
29+
Scenario: Sobol sensitivity analysis for end-of-season LAI and TWSO in Limburg, Netherlands
2930
Given we are using NASA weather data with a latitude of "51" and a longitude of "5"
3031
And we are using agronomy management data in the "data/potato_netherlands_2021.agro" file
31-
When we execute a sensitivity analysis using the "Sobol" method and the "SALib" library with "4" samples
32-
Then behave will test it for us!
32+
When we execute a sensitivity analysis using the "Sobol" method and the "SALib" library with "256" samples
33+
Then the "1st" highest "first order" sensitivity index for "LAI" should be "SPAN"
34+
And the "1st" highest "total order" sensitivity index for "LAI" should be "SPAN"
35+
And the "2nd" highest "total order" sensitivity index for "LAI" should be "TSUM1"
36+
And the "1st" highest "first order" sensitivity index for "TWSO" should be "SPAN"
37+
And the "2nd" highest "first order" sensitivity index for "TWSO" should be "Q10"
38+
And the "1st" highest "total order" sensitivity index for "TWSO" should be "SPAN"
39+
And the "2nd" highest "total order" sensitivity index for "TWSO" should be "TSUM2"
3340

3441
@india @sensitivity
35-
Scenario: Sobol sensitivity analysis for LAI and TWSO in Gujarat, India
42+
Scenario: Sobol sensitivity analysis for end-of-season LAI and TWSO in Gujarat, India
3643
Given we are using NASA weather data with a latitude of "23" and a longitude of "73"
3744
And we are using agronomy management data in the "data/potato_india_2021.agro" file
38-
When we execute a sensitivity analysis using the "Sobol" method and the "SALib" library with "4" samples
39-
Then behave will test it for us!
45+
When we execute a sensitivity analysis using the "Sobol" method and the "SALib" library with "256" samples
46+
Then the "1st" highest "first order" sensitivity index for "LAI" should be "SPAN"
47+
And the "1st" highest "total order" sensitivity index for "LAI" should be "SPAN"
48+
And the "2nd" highest "total order" sensitivity index for "LAI" should be "TSUM1"
49+
And the "1st" highest "first order" sensitivity index for "TWSO" should be "TSUM1"
50+
And the "2nd" highest "first order" sensitivity index for "TWSO" should be "TSUM2"
51+
And the "1st" highest "total order" sensitivity index for "TWSO" should be "TSUM1"
52+
And the "2nd" highest "total order" sensitivity index for "TWSO" should be "TSUM2"

‎features/steps/step_calibration.py‎

Lines changed: 75 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,15 @@
1-
from behave import given, register_type, when
1+
import json
2+
3+
import pandas as pd
4+
from behave import given, register_type, then, when
25
from behave.runner import Context
36
from pcse.base import ParameterProvider
47

58
from wofostat import (
69
end_of_season_sensitivity_func,
710
get_parameter_spec,
11+
objective_func,
12+
run_optimisation,
813
run_sensitivity_analysis,
914
snake_case_string,
1015
)
@@ -23,6 +28,22 @@ def specify_calibration(context: Context) -> None:
2328
context.calibration_spec = {row["name"]: row["value"] for row in context.table}
2429

2530

31+
@given('we are using "{distance_metric}" as our error metric')
32+
def set_distance_metric(context: Context, distance_metric: str) -> None:
33+
context.distance_metric = distance_metric
34+
35+
36+
@given('we are using observed data from the "{fpath}" file')
37+
def get_observed_data(context: Context, fpath: str) -> None:
38+
context.observed_data = pd.read_csv(fpath)
39+
40+
41+
@given('we are using ground truth data from the "{fpath}" file')
42+
def get_ground_truth(context: Context, fpath: str) -> None:
43+
with open(fpath) as f:
44+
context.ground_truth = json.load(f)
45+
46+
2647
def _get_params(context: Context) -> ParameterProvider:
2748
params = WOFOST.get_params(
2849
cropd=context.cropd, sited=context.sited, soild=context.soild
@@ -52,3 +73,56 @@ def execute_sensitivity(
5273
engine=engine,
5374
**context.calibration_spec,
5475
)
76+
77+
78+
@then(
79+
'the "{position}" highest "{order}" sensitivity index for "{state_var}" '
80+
'should be "{param_name}"'
81+
)
82+
def check_sensitivity_index(
83+
context: Context, position: str, order: str, state_var: str, param_name: str
84+
) -> None:
85+
index = "".join(c for c in position if c.isdigit())
86+
index = int(index) - 1
87+
88+
if order == "total order":
89+
order = "ST"
90+
else:
91+
order = "S1"
92+
93+
sensitivity_param = context.sp_df[state_var][order].iloc[index].name
94+
if sensitivity_param != param_name:
95+
raise RuntimeWarning(f"Parameter is {sensitivity_param}")
96+
97+
assert sensitivity_param == param_name
98+
99+
100+
@when(
101+
'we execute an optimisation procedure using the "{method:SnakeCaseString}" method '
102+
'and the "{engine:SnakeCaseString}" library with "{n_iterations:d}" iterations'
103+
)
104+
def execute_optimisation(
105+
context: Context, method: str, engine: str, n_iterations: int
106+
) -> None:
107+
params = _get_params(context)
108+
109+
(
110+
context.calibrator,
111+
context.param_importances,
112+
context.trials_df,
113+
context.parameter_estimates,
114+
) = run_optimisation(
115+
parameter_spec=context.parameter_spec,
116+
n_iterations=n_iterations,
117+
wdp=context.wdp,
118+
agro=context.agro,
119+
state_vars=context.state_vars,
120+
calibration_func=objective_func,
121+
params=params,
122+
method=method,
123+
engine=engine,
124+
ground_truth=context.ground_truth,
125+
observed_data=context.observed_data,
126+
distance_metric=context.distance_metric,
127+
**context.calibration_spec,
128+
)

‎notebooks/6_full_sensitivity_wofost.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -221,7 +221,7 @@
221221
"source": [
222222
"state_vars = [\"LAI\", \"TWSO\"]\n",
223223
"\n",
224-
"run_sensitivity_analysis(\n",
224+
"calibrator, sp = run_sensitivity_analysis(\n",
225225
" experiment_name = \"netherlands_sensitivity_analysis\",\n",
226226
" parameter_spec=parameter_spec,\n",
227227
" n_samples = 8,\n",

0 commit comments

Comments
 (0)