message_ix_models.project.ssp.data.SSPUpdate

class message_ix_models.project.ssp.data.SSPUpdate(*args, **kwargs)[source]

Bases: SSPDataSource

Provider of exogenous data from the SSP Update database.

This database is accessible from https://data.ece.iiasa.ac.at/ssp.

To use data from this source:

Read the general documentation for project.ssp.data.
If necessary, obtain copy of the original data file(s).
Call SSPUpdate.add_tasks() with keyword arguments corresponding to SSPUpdate.Options. In particular:
- For release up to “3.1” and measure “GDP”, model must be one of “IIASA GDP 2023” or “OECD ENV-Growth 2023”.
- For release “3.2.beta” and measure “GDP”:
  - model must be “OECD ENV-Growth 2025”.
  - unit must be given, with a value such as “billion USD_2010/yr”, “billion USD_2015/yr”, or “billion USD_2017/yr”. Without these, keys are not unique.

Example

>>> keys = SSPUpdate.add_tasks(
...     computer,
...     context,
...     release="3.1",
...     ssp_id="3",
...     measure="GDP"
...     model="IIASA GDP 2023",
... )
>>> result = computer.get(keys[0])

__init__(*args, **kwargs) → None[source]

Create an instance and prepare info for transform()/get().

The base implementation:

Sets options—if not already set—by passing kwargs to Options.
Raises an exception if there are other/unhandled args or kwargs.
If key is not set, constructs it with:
- Name name or measure in lower case.
- Dimensions dims.
Subclasses may pre-empt this behaviour by setting key statically or dynamically.

A concrete class implementation must:

Set options, either directly or by calling super().__init__() with or without keyword arguments.
Set key, either directly or by calling super().__init__(). In the latter case, it may set name, measure, and/or dims to control the behaviour.
Raise an exception if unrecognized or invalid kwargs are passed.

and may:

Transform kwargs or options arguments into other values, for instance by mapping certain values to others, applying regular expressions, or other operations.
Store those values as instance attributes for use in get().
Log messages that give information that helps to debug exceptions.

It must not perform any time- or memory-intensive operations, such as actually loading or fetching data. Those operations should be in get().

Methods

`__init__`(args, *kwargs)	Create an instance and prepare info for `transform()`/`get()`.
`add_tasks`(c, *args[, context, strict])	Add tasks to c to provide and transform the data.
`get`()	Return the data.
`make_query`(dim_case, model_scenario, unit)	Assemble and store a `pandas.DataFrame.query()` string.
`transform`(c, base_key)	Add tasks to c to transform raw data from base_key.

Attributes

`filename`	File names containing the data, according to the release.
`replace`	replace argument to `iamc.to_quantity()`.
`unique`	unique argument to `iamc.to_quantity()`.
`use_test_data`	`True` to allow the class to look up and use test data.
`variable`	Alias from short measure IDs to IAMC 'variable'.
`where`	`where` argument to `path_fallback()`.
`options`	Instance of the `Options` class.
`key`	Key for the returned `Quantity`.

class Options(aggregate: bool = True, interpolate: bool = True, measure: str = '', name: str = '', dims: tuple[str, ...] = ('n', 'y'), model: str = '', source: str = '', ssp_id: str = '', release: str = '', unit: str = '')[source]

Bases: Options

aggregate: bool = True: True if ExoDataSource.transform() should aggregate data on the \(n\) dimension.

dims: tuple[str, ...] = ('n', 'y'): Dimensions for the returned Key/Quantity.

classmethod from_args(source_id: str | ExoDataSource, *args, **kwargs)

Construct an instance from keyword arguments.

Parameters:: source_id – For backwards-compatibility with prepare_computer().

handle_source(prefix: str) → None: Check that source starts with prefix; update ssp_id.

interpolate: bool = True: True if ExoDataSource.transform() should interpolate data on the \(y\) dimension.

measure: str = '': Identifier for the primary measure of retrieved/returned data.

model: str = '': Model name.

name: str = '': Name for the returned Key/Quantity.

release: str = ''

one of the keys of filename. Required.

Type:: Release

source: str = ''

SSP(2017).1”. ssp_id should be preferred.

Type:: Partial URN for a code in the SSP code list, e.g. “ICONICS

ssp_id: str = '': Short ID of the SSP code, e.g. “1”.

unit: str = '': Selector for the IAMC ‘UNIT’ dimension.

classmethod add_tasks(c: Computer, *args, context: Context | None = None, strict: bool = True, **kwargs) → tuple

Add tasks to c to provide and transform the data.

The first returned key is key, and will trigger the following tasks:

Load or retrieve data by invoking ExoDataSource.get().
If BaseOptions.aggregate is True, aggregate on the \(n\) (node) dimension according to Config.regions.
If BaseOptions.interpolate is True, interpolate on the \(y\) (year) dimension according to Config.years.

Steps (2) and (3) are added by transform() and may differ in concrete classes.

Other returned keys include further transformations:

key + "y0_indexed": same as key, but indexed to the values as of the first model period.

Other keys that are created but not returned can be accessed on c:

key + "message_ix_models.foo.bar.CLASS": the raw data, with a tag from the fully-qualified name of the ExoDataSource class.

To support the loading and transformation of data, add_structure() is first called with c.

Todo

Add option/tasks to index to a particular label on the \(n\) dimension.

Parameters:

context – Passed to add_structure().
strict – Passed to add_structure().

Return type:

tuple of Key

filename = {'3.0': '1706548837040-ssp_basic_drivers_release_3.0_full.csv.gz', '3.0.1': '1710759470883-ssp_basic_drivers_release_3.0.1_full.csv.gz', '3.1': '1721734326790-ssp_basic_drivers_release_3.1_full.csv.gz', '3.2.beta': '0000000000000-ssp_basic_drivers_release_3.2.beta_full.csv.xz', 'preview': 'SSP-Review-Phase-1.csv.gz'}: File names containing the data, according to the release.

get()

Return the data.

Implementations in concrete classes may load data from file, retrieve from remote sources or local caches, generate data, or anything else.

The Quantity returned by this method must have dimensions corresponding to key. If the original/upstream/raw data has different dimensionality (fewer or more dimensions; different dimension IDs), a concrete class must transform these, make appropriate selections, etc.

key: Key: Key for the returned Quantity. This may either be set statically on a concrete subclass, or created via __init__().

make_query(dim_case: Callable[[str], str], model_scenario: Iterable[tuple[str, str]], unit: str) → None

Assemble and store a pandas.DataFrame.query() string.

Parameters:

dim_case – Function to apply to IAMC dimension IDs, for instance str.upper() to use “MODEL”.
model_scenario – Iterable of (model_name, scenario_name) pairs. model_name may be an empty string.
unit – Units. May be an empty string.

options: Options

Instance of the Options class.

A concrete class that overrides Options should redefine this attribute, to facilitate type checking.

replace: dict[str, str | dict[str, str]] = {}: replace argument to iamc.to_quantity().

transform(c: Computer, base_key: Key) → Key[source]

Add tasks to c to transform raw data from base_key.

If necessary, data for period 2020 or 2025 are deduplicated.

unique: str = 'MODEL SCENARIO VARIABLE UNIT': unique argument to iamc.to_quantity().

use_test_data: bool = False: True to allow the class to look up and use test data. If no test data exists, this setting has no effect. See _where().

variable = {'GDP': 'GDP|PPP', 'POP': 'Population'}: Alias from short measure IDs to IAMC ‘variable’. See make_query().

where: list['str | Path'] = ['local', 'package', 'private']

where argument to path_fallback(). In order:

Currently data is stored in message-static-data, cloned and linked from within the user’s ‘local’ data directory.
Previously some files were stored directly within message_ix_models (available in an editable install from a clone of the git repository, ‘package’) or in message_data (‘private’). These settings are only provided for backward compatibility.

Fuzzed/random test data (‘test’) is also available, but not enabled by default.