Skip to content

Commit 8adbffc

Browse files
JBrisJBris
committed
Adding features and steps for SA
1 parent 431e2a5 commit 8adbffc

15 files changed

Lines changed: 319 additions & 40 deletions

File tree

‎README.md‎

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,13 +17,47 @@
1717
- [WOFOST Assumptions Tracking](#wofost-assumptions-tracking)
1818
- [Table of contents](#table-of-contents)
1919
- [Introduction](#introduction)
20+
- [Quickstart](#quickstart)
21+
- [Python Environment](#python-environment)
22+
- [behave](#behave)
23+
- [Data](#data)
2024
- [Communication](#communication)
2125
- [Contributions and Support](#contributions-and-support)
2226
- [License](#license)
2327

2428
# Introduction
2529

26-
This repository contains code examples for assumptions tracking of WOFOST Potato using behaviour driven development and model cards.
30+
This repository contains code examples for assumptions tracking of WOFOST Potato using behaviour driven development. The overall motivation behind this demonstration is to synthesise the behavior-driven development (BDD) testing approach that is more commonly used within agile software development with the calibration of mechanistic simulation models. The aim is to demonstrate how BDD can enable and facilitate communication among different disciplines using a common human-readable language. This is particularly important within highly technical scientific domains where cross-disciplinary communication may be more challenging.
31+
32+
BDD was chosen as it (at least in theory) facilitates collaboration and communication within multidisciplinary projects. Namely, it encourages business analysts and developers to collaborate in specifying the behaviour of software, via the use of user stories. These user stories should be written to a formal document. For this demonstration, these documents are written in the popular Gherkin syntax. Model assumptions can be documented in Gherkin, and would be recorded by business analysts as user stories.
33+
34+
# Quickstart
35+
36+
First, run the following code to install all dependencies:
37+
38+
```
39+
poetry install --no-interaction
40+
```
41+
42+
Secondly, execute the following to run the behaviour tests:
43+
44+
```
45+
behave --verbose --summary
46+
```
47+
48+
# Python Environment
49+
50+
## behave
51+
52+
[The behave package was used to implement all BDD tests.](https://behave.readthedocs.io/en/stable/index.html) These tests have been written using Gherkin syntax. [For more information on Gherkin, click here.](https://cucumber.io/docs/gherkin/reference/)
53+
54+
* [Feature files containing human-readable Gherkin can be found here.](features)
55+
* [The implementation of the scenario steps can be found here.](features/steps)
56+
57+
# Data
58+
59+
* [All relevant datasets and files can be found here.](data)
60+
* [The potato data were originally retrieved from here.](https://github.com/ajwdewit/pcse_notebooks/tree/master/data)
2761

2862
# Communication
2963

‎docs/source/api_reference/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,5 @@ API Reference
55
:maxdepth: 1
66

77
calibration.rst
8+
utils.rst
89
wofost.rst

‎docs/source/api_reference/utils.rst‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Utilities
2+
=========
3+
4+
.. automodule:: wofostat.utils
5+
:members:

‎docs/source/index.rst‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,9 @@
11
WOFOST Assumptions Tracking
22
---------------------------
33

4-
This repository contains code examples for assumptions tracking of WOFOST Potato using behaviour driven development and model cards.
4+
This repository contains code examples for assumptions tracking of WOFOST Potato using behaviour driven development. The overall motivation behind this demonstration is to synthesise the behavior-driven development (BDD) testing approach that is more commonly used within agile software development with the calibration of mechanistic simulation models. The aim is to demonstrate how BDD can enable and facilitate communication among different disciplines using a common human-readable language. This is particularly important within highly technical scientific domains where cross-disciplinary communication may be more challenging.
5+
6+
BDD was chosen as it (at least in theory) facilitates collaboration and communication within multidisciplinary projects. Namely, it encourages business analysts and developers to collaborate in specifying the behaviour of software, via the use of user stories. These user stories should be written to a formal document. For this demonstration, these documents are written in the popular Gherkin syntax. Model assumptions can be documented in Gherkin, and would be recorded by business analysts as user stories.
57

68
Communication
79
-------------

‎features/optimisation.feature‎

Lines changed: 25 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,28 @@
1-
Feature: Black-box optimisation for WOFOST potato
2-
Because parameter uncertainty drives canopy development
3-
We want to quantify global sensitivities using Sobol indices
4-
So we can identify dominant parameters for potato LAI
1+
Feature: Calibration of WOFOST potato using black-box optimisation
2+
Because crop model parameters are uncertain
3+
We want to calibrate WOFOST by exploring the parameter space with black-box optimisation
4+
So potato crop development and growth is both accurate and well-constrained
5+
6+
Background:
7+
Given we are using WOFOST site data with a WAV of "50.0"
8+
And we are using default soil parameter values
9+
And we are using crop data stored in the "data" directory
10+
And our state variables are "LAI and TWSO"
11+
And the following parameter specification is used for calibration:
12+
| name | description | range | distribution | type |
13+
| TSUM1 | Temperature sum from emergence to anthesis | 150, 280 | uniform | continuous |
14+
| TSUM2 | Temperature sum from anthesis to maturity | 1400, 2100 | uniform | continuous |
15+
| TBASEM | Base temperature for emergence | 2, 4 | uniform | continuous |
16+
| TSUMEM | Temperature sum required for crop emergence | 170, 255 | uniform | continuous |
17+
| TEFFMX | Maximum effective temperature for emergence | 18, 32 | uniform | continuous |
18+
| SPAN | Life span of leaves | 20, 50 | uniform | continuous |
19+
| TDWI | Initial total dry weight of the crop | 75, 700 | uniform | continuous |
20+
| RGRLAI | Relative growth rate of leaf area index | 0.008, 0.02 | uniform | continuous |
21+
| Q10 | Temperature response factor for respiration (Q10 coefficient) | 2, 3 | uniform | continuous |
22+
And the following specification is used for the calibration procedure:
23+
| name | description | value |
24+
| experiment_name | The name of the current experiment | WOFOST optimisation |
25+
| n_jobs | The number of simulations to run in parallel | -1 |
526

627
Scenario: run a simple test
728
Given we have behave installed

‎features/sensitivity.feature‎

Lines changed: 33 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,39 @@
11
Feature: Sobol sensitivity analysis for WOFOST potato
2-
Because parameter uncertainty drives canopy development
2+
Because parameter uncertainty drives crop development and growth
33
We want to quantify global sensitivities using Sobol indices
4-
So we can identify dominant parameters for potato LAI
4+
So we can identify influential parameters for potato LAI and TWSO
55

6-
Scenario: run a simple test
7-
Given we have behave installed
8-
When we implement a test
6+
Background:
7+
Given we are using WOFOST site data with a WAV of "50.0"
8+
And we are using default soil parameter values
9+
And we are using crop data stored in the "data" directory
10+
And our state variables are "LAI and TWSO"
11+
And the following parameter specification is used for calibration:
12+
| name | description | range | distribution | type |
13+
| TSUM1 | Temperature sum from emergence to anthesis | 150, 280 | uniform | continuous |
14+
| TSUM2 | Temperature sum from anthesis to maturity | 1550, 2100 | uniform | continuous |
15+
| TBASEM | Base temperature for emergence | 2, 4 | uniform | continuous |
16+
| TSUMEM | Temperature sum required for crop emergence | 170, 255 | uniform | continuous |
17+
| TEFFMX | Maximum effective temperature for emergence | 18, 32 | uniform | continuous |
18+
| SPAN | Life span of leaves | 20, 50 | uniform | continuous |
19+
| TDWI | Initial total dry weight of the crop | 75, 700 | uniform | continuous |
20+
| RGRLAI | Relative growth rate of leaf area index | 0.008, 0.02 | uniform | continuous |
21+
| Q10 | Temperature response factor for respiration (Q10 coefficient) | 2, 3 | uniform | continuous |
22+
And the following specification is used for the calibration procedure:
23+
| name | description | value |
24+
| experiment_name | The name of the current experiment | WOFOST sensitivity analysis |
25+
| n_jobs | The number of simulations to run in parallel | 10 |
26+
27+
@netherlands @sensitivity
28+
Scenario: Sobol sensitivity analysis for LAI and TWSO in Limburg, Netherlands
29+
Given we are using NASA weather data with a latitude of "51" and a longitude of "5"
30+
And we are using agronomy management data in the "data/potato_netherlands_2021.agro" file
31+
When we execute a sensitivity analysis using the "Sobol" method and the "SALib" library with "4" samples
932
Then behave will test it for us!
1033

11-
Scenario: run another simple test
12-
Given we have behave installed
13-
When we implement a test
34+
@india @sensitivity
35+
Scenario: Sobol sensitivity analysis for LAI and TWSO in Gujarat, India
36+
Given we are using NASA weather data with a latitude of "23" and a longitude of "73"
37+
And we are using agronomy management data in the "data/potato_india_2021.agro" file
38+
When we execute a sensitivity analysis using the "Sobol" method and the "SALib" library with "4" samples
1439
Then behave will test it for us!

‎features/steps/step_calibration.py‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
from behave import given, register_type, when
2+
from behave.runner import Context
3+
from pcse.base import ParameterProvider
4+
5+
from wofostat import (
6+
end_of_season_sensitivity_func,
7+
get_parameter_spec,
8+
run_sensitivity_analysis,
9+
snake_case_string,
10+
)
11+
from wofostat.wofost import WOFOST
12+
13+
register_type(SnakeCaseString=snake_case_string)
14+
15+
16+
@given("the following parameter specification is used for calibration:")
17+
def specify_parameters(context: Context) -> None:
18+
context.parameter_spec = get_parameter_spec(context.table)
19+
20+
21+
@given("the following specification is used for the calibration procedure:")
22+
def specify_calibration(context: Context) -> None:
23+
context.calibration_spec = {row["name"]: row["value"] for row in context.table}
24+
25+
26+
def _get_params(context: Context) -> ParameterProvider:
27+
params = WOFOST.get_params(
28+
cropd=context.cropd, sited=context.sited, soild=context.soild
29+
)
30+
WOFOST(params, context.wdp, context.agro)
31+
return params
32+
33+
34+
@when(
35+
'we execute a sensitivity analysis using the "{method:SnakeCaseString}" method and '
36+
'the "{engine:SnakeCaseString}" library with "{n_samples:d}" samples'
37+
)
38+
def execute_sensitivity(
39+
context: Context, method: str, engine: str, n_samples: int
40+
) -> None:
41+
params = _get_params(context)
42+
43+
context.calibrator, context.sp_df = run_sensitivity_analysis(
44+
parameter_spec=context.parameter_spec,
45+
n_samples=n_samples,
46+
wdp=context.wdp,
47+
agro=context.agro,
48+
state_vars=context.state_vars,
49+
calibration_func=end_of_season_sensitivity_func,
50+
params=params,
51+
method=method,
52+
engine=engine,
53+
**context.calibration_spec,
54+
)

‎features/steps/step_wofost.py‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
from behave import given, then, when
2+
from behave.runner import Context
3+
4+
from wofostat.wofost import WOFOST
5+
6+
7+
@given('we are using WOFOST site data with a WAV of "{WAV:f}"')
8+
def get_sited(context: Context, WAV: float) -> None:
9+
context.sited = WOFOST.get_sited(WAV)
10+
11+
12+
@given("we are using default soil parameter values")
13+
def get_soild(context: Context) -> None:
14+
context.soild = WOFOST.get_soild()
15+
16+
17+
@given('we are using crop data stored in the "{fpath}" directory')
18+
def get_cropd(context: Context, fpath: str) -> None:
19+
context.cropd = WOFOST.get_cropd(fpath)
20+
21+
22+
@given('we are using agronomy management data in the "{fpath}" file')
23+
def get_agro(context: Context, fpath: str) -> None:
24+
context.agro = WOFOST.get_agro(fpath)
25+
26+
27+
@given('our state variables are "{state_vars}"')
28+
def set_state_variables(context: Context, state_vars: str) -> None:
29+
context.state_vars = (
30+
state_vars.replace(", and", ",")
31+
.replace(" and ", ",")
32+
.replace(" ", "")
33+
.split(",")
34+
)
35+
36+
37+
@given(
38+
'we are using NASA weather data with a latitude of "{latitude:d}" '
39+
'and a longitude of "{longitude:d}"'
40+
)
41+
def get_wdp(context: Context, latitude: float, longitude: float) -> None:
42+
context.wdp = WOFOST.get_wdp(latitude, longitude)
43+
44+
45+
@given("we have behave installed")
46+
def behave_installed(context: Context) -> None:
47+
pass
48+
49+
50+
@when("we implement a test")
51+
def implement_test(context: Context) -> None:
52+
assert True is not False
53+
54+
55+
@then("behave will test it for us!")
56+
def test_for_us(context: Context) -> None:
57+
assert context.failed is False

‎features/steps/tutorial.py‎

Lines changed: 0 additions & 17 deletions
This file was deleted.

‎notebooks/6_full_sensitivity_wofost.ipynb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -224,7 +224,7 @@
224224
"run_sensitivity_analysis(\n",
225225
" experiment_name = \"netherlands_sensitivity_analysis\",\n",
226226
" parameter_spec=parameter_spec,\n",
227-
" n_samples = 256,\n",
227+
" n_samples = 8,\n",
228228
" wdp = netherlands_wdp,\n",
229229
" agro=netherlands_agro,\n",
230230
" state_vars = state_vars,\n",

0 commit comments

Comments
 (0)