Skip to content

Latest commit

 

History

History
226 lines (133 loc) · 31.6 KB

METHODOLOGY.md

File metadata and controls

226 lines (133 loc) · 31.6 KB

CPR-CCLW combined methodology

Contents

About this page

Welcome to the methodology page for the Climate Policy Radar database and app, which makes it possible to find information from within the full text of a large corpus of climate law and policy documents using semantic search. This methodology page also applies to the Climate Change Laws of the World Interface for the Climate Policy Radar database.

This page provides information about the scope and structure of our database, our data collection methods and terminology, updates to the database and planned future developments.

As we expand and evolve, this methodology document will evolve to reflect developments. All previous versions will be linked here (this is the first one, so no previous versions yet).

Documents and sources

The database contains national law and policy documents and various documents submitted to the UNFCCC, including NDCs, National Communications, Biennial Reports, submissions to the Global Stocktake, IPCC reoprts and more. The sources of the documents in our database are as follows:

Document type Source
National laws and policies https://climate-laws.org/
Nationally Determined Contributions https://unfccc.int/NDCREG
Adaptation Communications https://unfccc.int/ACR
Fast-Start Finance Reports https://unfccc.int/climatefinance?submissions
IPCC Assessment Reports https://www.ipcc.ch/reports/
Annual Compilation and Accounting Reports https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Biennial Reports https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Biennial Update Reports https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Facilitative Sharing of Views Reports https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Global Stocktake Synthesis Reports https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Intersessional Documents https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Long-Term Low Greenhouse Gas Development Strategies https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
National Communications https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
National Inventory Reports https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Pre-Session Documents https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Progress Reports https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Publications https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Reports https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Statements https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Submissions to the Global Stocktake https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Summary Reports https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Synthesis Reports https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=
Technical Analysis Summary Reports https://unfccc.int/topics/global-stocktake/information-portal?field_document_topic_target_id=All&field_document_type_target_id=4174&field_calculated_author_type_target_id=All&field_document_ca_target_id=

Language

Our dataset contains documents published in many different languages. Non-English documents are assigned English titles, summaries and attributes.

Currently, your searches will return results from all titles and summaries (regardless of language) as well as from the full text of all English documents; they do not return results from the full text of non-English documents.

  • Glimpse into the future: very soon, English-language searches will return translated results from documents in all languages. We are working towards making the app also available to use in French and Spanish.

Updates to the database

The database is continually monitored and updated by teams of experts at the LSE Grantham Research Institute and Climate Policy Radar, in order to ensure accuracy and to reflect the latest developments in climate law and policy. These updates are drawn from official sources such as government websites and parliamentary records, as well as UNFCCC and related websites, where applicable.

  • Glimpse into the future: Frequent updates are planned to incorporate new types of data — including litigation and subnational laws and policies — which will be collected from new sources, including a range of databases and aggregators.

Data structure

In the Climate Policy Radar database, documents are grouped together with supplementary documents that provide additional information, such as annexes, amendments and press releases, as well as any translations or versions of the document published in other languages.

We also indicate wider relationships between documents. For example, we indicate where documents belong to an overarching policy programme (such as the European Union’s Common Agricultural Policy) or publication cycle (such as Nationally Determined Contributions).

Attributes

Documents in the database are assigned attributes (also called metadata) in order to provide information about the contents of documents and to support searchability.

Attributes are manually assigned, and apply to entire documents. Most attributes and definitions used were developed by the LSE Grantham Institute; please see the section on 'Methodology of data sources' for these definitions. The table below provides some additional definitions for document attributes used in the Climate Policy Radar database and app.

Attribute Definition
Jurisdiction A document’s jurisdiction indicates where it was published. Jurisdictions include all parties to the UN Framework Convention on Climate Change (UNFCCC) — 196 countries plus the European Union — as well as a number of territories which are not party to the UN or UNFCCC, namely Taiwan, Palestine and Western Sahara.
Region Jurisdictions are grouped into regions of the world, as defined by the World Bank.
Year Refers to the year in which a document was first published. Note - this may not be year of the most recent update or amendment to a document.

For a full list of the values for each attribute, please see our codebook.

  • Glimpse into the future: attributes will be automatically assigned using machine learning models, and will apply to individual paragraphs and sentences within the text of documents.

Data limitations

The principal limitations to our dataset follow from those of our third-party data providers.

Data collection methods

We currently update our database manually, based on monitoring of developments in climate law and policy internationally. We are developing the capacity to more efficiently capture data from additional sources, including via scraping, in order to increase the pace and scope of the expansion of our dataset.

We also currently rely on data labelled manually by domain experts; this limits the pace at which it is possible for new data to be incorporated into our dataset. We are developing the capacity to automatically label data using machine learning models, to increase the efficiency, sophistication and thoroughness of our data labelling.

Natural language search

When you enter a search term, the Climate Policy Radar app uses natural language processing to allow you to find what you’re looking for without having to type out precise keywords. This is useful because often certain concepts are described in lots of different ways - like internal combustion engines, internal combustion engine vehicle, ICEV, fossil fuel car, and gasoline car all describe most of the cars you’ll find on today’s roads. With natural language search, you can use the terms you use and hear everyday because the tool will recognise them as related and relevant terms, meaning you get a much richer search experience as a result.

We use a machine learning method called dense retrieval alongside a fuzzy string search to perform natural language search. You can find more details in our blog post.

Biases and limitations

The model we use for dense retrieval inherits biases from both its base model, DistilBERT, and the search query dataset it was trained on, MSMARCO. It’s trained on relatively short passages of text (average 60 words in length), so may struggle with queries longer than this.

At the moment we use a general purpose model that has been trained on English Wikipedia and BookCorpus. This means it may misinterpret some climate- or policy-specific concepts, and is something that we look to improve in future.

For queries we’ve identified that could uncover harmful biases in the underlying semantic search model, semantic search falls back to using fuzzy string search only,which relies on matching words rather than returning related terms.

Additions and contributions

We work collaboratively with our users to keep our database at a high standard. Should you encounter any errors or inaccuracies in the data, we encourage you to get in touch with us at [email protected].

Methodology of data sources

Map projection

This map was made with the Admin o - Countries map package by Natural Earth. Climate Policy Radar's usage of this World map does not represent an opinion on any disputed boundaries.

Climate Change Laws of the World

Introduction

This page was last updated on 18 October 2022.

This page outlines the definitions, scope and principles used to collect and categorise the legislation displayed in the Climate Change Laws of the World database.

Classifications described below have been assigned manually. LSE, Climate Policy Radar, and our partners are currently working to develop classifications powered by AI. Click here to learn more.

Scope of documents included

This database covers all UNFCCC parties (196 countries plus the European Union), and several territories that are not in the UN or UNFCCC, such as Taiwan, Palestine and Western Sahara.

The database focuses exclusively on climate change-related laws and policies. We define climate change-related laws broadly as legal documents that address policy areas directly relevant to climate change mitigation, adaptation, loss and damage or disaster risk management. Typically, to be included in the database one or more aspects of a law or policy must be demonstrably motivated by climate change concerns.

We consider legal documents that: establish rules and procedures related to reducing energy demand; promote low-carbon energy supply; restrict the development of fossil fuel-based infrastructure; promote or mandate low-carbon buildings; set carbon pricing policy; lower industry emissions; tackle deforestation and promote or mandate sustainable land use; contain other mitigation efforts; cover climate-related research and development; promote or mandate low-carbon transport; enhance adaptation capabilities; or put in place natural disaster risk management. In some instances, laws addressing disaster risk management and/or energy-related matters may have been included even where no explicit mention of climate change is made, given the close connection between these areas and climate change. Laws aimed at enhancing other areas of environmental action or environmental protection are not included unless they include provisions with directly relevant impacts on climate change action, such as a reduction in greenhouse gas emissions, significant enhancement or protection of carbon sinks, or if they expressly contribute to climate adaptation and resilience.

The database includes legislation and policy at the national and sectoral levels only. At present, it excludes information about the policy response of sub-national governments. It also excludes international legislation, such as treaties. Documents submitted only to international organisations such as the UNFCCC are generally excluded unless they have been converted into specific legislation or formally adopted as executive policies. In most cases, this means that documents such as the Long-term Low Emissions Development Strategies and National Adaptation Plans submitted to UNFCCC registries are not included. Users are encouraged to consult the relevant UN websites where these documents may be relevant to their search. For EU member states, laws transposing EU Directives are in the process of being added to the database, along with the relevant EU law, since in many instances these may contain additional relevant provisions. Where possible these additional provisions have been noted in the description of the relevant law.

Documents included in the database must have full legal force, having passed through the legislature or through an executive decision-making body, and/or set out a current governmental policy objective or set of policy objectives. To the best of our ability, we capture major amendments to legislation and update document summaries accordingly. Where we become aware that laws are outdated, either because they have been repealed, replaced or reversed, or because they were in force for a limited time period, these documents may be removed from the database. In general, precedence is given to the document with the highest possible status dealing with a given matter. In some instances, policies or regulations aimed at giving effect to legislation or regulation of a higher order (‘parent legislation’), such as statutory instruments or other guidance adopted to provide detail on the implementation of an act of parliament, may be included in the database alongside their ‘parent legislation’ where these provide a significantly more detailed picture.

CCLW also contains information summarising national-level climate targets. See Annex I below for more information.

Definitions and categorisation

We assign a number of codes and categories to all records in the CCLW database to enhance the usability and searchability of the data. The system of codes and categories has evolved to remain in step with developments in climate governance, the growth of the dataset, and the needs of our user community. Codes and categories currently in use include:

Legislation and policy

We categorise documents as legislation or policy depending on if they are enacted by the legislative or executive branch of government. Where the same body holds both legislative and executive power, a determination of whether a document constitutes a law or policy is made based on our best understanding of the country’s legal system.

Responses

Laws and policies are categorised according to the climate policy response(s) to which they are most relevant: mitigation, adaptation and resilience, or loss and damage. Where appropriate, laws and policies may be tagged as relevant to more than one policy response.

1.     Mitigation: Mitigation laws and policies refer to a legislative or executive disposition focused on curbing a country’s greenhouse gases emissions in one sector or more. Measures can be directly related to emission reductions, such as establishing a national carbon budget or cap and trade system, or indirectly related, such as establishing relevant institutions or providing additional funding for research and development into low-carbon technologies. Laws and policies addressing forests and land use are included if they explicitly support climate change mitigation through activities that reduce emissions and increase carbon removals. General forest management and conservation laws are not included, even if they may have implicit consequences for climate change mitigation.

2.     Adaptation:  Adaptation laws and policies contain explicit provisions concerning climate change adaptation, i.e. the need for changes in the management of ecological, social or economic systems in response to the actual or expected impacts of climate change. We conducted a comprehensive review of adaptation laws and policies in 2018 and have added documents since then. Note that in many cases, climate change adaptation responses may be embedded in development policies, general planning policies, risk-reduction and disaster management policies, water policies, land use and forestry policies and health policies, which can make them difficult to identify.[i]

3.     Disaster risk management: Laws and policies governing countries’ approaches to disaster risk management (DRM) and disaster risk reduction (DRR) were first added to the database in 2019. We have adopted a broad approach to assessing whether these documents fall within the database scope, as laws and policies often target natural and human disasters in a holistic manner, making it harder to isolate climate-related adverse events. In determining whether a disaster risk management law or policy falls within scope, our researchers will usually consider whether the document relates to the types of disaster that are expected to become more frequent due to climate change, such as hurricanes, typhoons, flooding, heatwaves, droughts, forest fires and sea-level rise (this list is non-exhaustive). In some cases these laws and policies may constitute exceptions to the general rule that laws and policies must be explicitly ‘climate-motivated’ to justify their inclusion.

4.     Loss and damage: We define loss and damage-related laws and policies as those that explicitly seek to reduce the risk of climate-related loss and damage by increasing resilience and those that provide compensation or other relief measures to support victims of climate-related loss and damage. The guiding definition of loss and damage includes both economic and non-economic losses that arise due to climate impacts that are made more frequent or more severe by anthropogenic greenhouse gas emissions. Policy measures may include national climate relief funds, risk transfer mechanisms, internal relocation arrangements, mainstreaming loss and damage across government departments and ministries, and social protection programmes and safety nets. Loss and damage is the most recent policy response area included in the database. Our current categorisation accounts for documents that explicitly mention the term ‘loss and damage’ and is exhaustive for laws and policies that were passed from 2015 onward. Documents addressing loss and damage with regard to specific impacts, such as biodiversity loss or drought, but which do not use an explicit loss and damage framing, do not have the loss and damage tag.

Frameworks

A number of laws or policies in the database are categorised as ‘framework’ documents. While there is no agreed definition of a ‘climate change framework law’, this term is applied with increasing frequency to a discrete class of laws that share some or all of the following characteristics, and we tag legislative documents within the database as ‘framework laws’ accordingly:

1.     Set out the strategic of travel for national climate change policy

2.     Are passed by the legislative branch of government

3.     Contain national long-term and/or medium targets and/or pathways for change

4.     Set out institutional arrangements for climate governance at the national level

5.     Are multi-sectoral in scope

6.     Involve mechanisms for transparency and/or accountability.[ii]

However, in the case of executive policies, we employ a broader definition than described above. Examples range from overarching multi-sectoral action plans that serve to provide a unifying basis for climate change action within a country, to documents with a narrower focus but that nonetheless establish a governance framework for a specific aspect of climate action, such as a national low-carbon energy policy. Data users interested in this sub-set of climate laws are advised to review the sub-set of laws tagged as frameworks and to use their discretion to determine which of these laws may be most relevant to their subject of inquiry. Framework documents are tagged to indicate the policy response area(s) to which they relate, whether mitigation, adaptation or disaster risk management. 

Instruments

Documents are assessed to determine the primary policy instruments or ‘tools’ on which they rely, in order to facilitate a more nuanced understanding of the ways in which governments around the world seek to influence the drivers and impacts of climate change. We categorise instruments according to an adapted version of the classic Hood and Margetts NATO typology of instruments.[iii] We assess documents to determine the most relevant area(s) of government activity and then tag them with the specific types of instrument they employ to achieve their stated aims and objectives. The vast majority of documents relate to more than one area of government activity and rely on multiple instruments. See Annex II below for a full list of instruments currently in use.

Sectors

Each document is assessed to determine the most relevant sector or sectors to which it relates. The following sectors are currently considered: Agriculture, Transport, Energy, Waste, Environment, Tourism, Land Use, Land-Use Change and Forestry (LULUCF), Industry, Buildings, Water, Health, Public Sector, and Other. Where a document relates to multiple sectors or appears cross-cutting in intention, we assign the labels ‘economy-wide’ or ‘cross-cutting area’. In some instances, multi-sectoral documents may also be categorised as related to Disaster Risk Management (DRM), Adaptation, or Social Development. We also consider whether laws relate primarily to urban or rural sectors of the economy and/or to coastal zones. We currently only cover waste laws that explicitly mention methane, another greenhouse gas, or if they deal with waste-to-energy schemes.

Keywords

Discretionary keywords may be assigned to cases to enable data users to identify these more easily. We do not maintain a comprehensive list of keywords.

Notes [i] Nachmany M, Byrnes R, Surminski S (2019) National laws and policies on climate change adaptation: a global review. London: Grantham Research Institute on Climate Change and the Environment.

[ii] See further:  Nachmany M et al. (2015) The 2015 Global Climate Legislation Study: A Review of Climate Change Legislation in 99 Countries. London: Grantham Research Institute on Climate Change and the Environment; Nash S, Steurer R (2019) Taking stock of Climate Change Acts in Europe: Living policy processes or symbolic gestures? Climate Policy 3:1; World Bank (2020) World Bank Reference Guide to Climate Change Framework Legislation. EFI Insight-Governance. Washington, DC: World Bank.

[iii] See Hood C, Margetts H (2007) The Tools of Government in the Digital Age, London: Macmillan Education. We have loosely mapped Hood and Margetts’ four basic ‘resources’ of government – Nodality, Authority, Treasure and Organization – to areas of government activity as Capacity Building (Nodality), Regulation (Authority), Incentives and Direct Investment (Treasure), and Governance and Planning (Organization). We developed these adaptations to enhance the searchability of the database and to enable observations of particular relevance to climate change practitioners.

Data collection process

The Climate Change Laws of the World Database started life in 2010 as a printed publication, comprising an overview of climate change laws and policies in just 16 countries. Since then, it has expanded to include all countries in the present scope and has moved online to enable users to access relevant information when it becomes available. Updates to the data are now collected in real time from official sources such as government websites, parliamentary records and court documents. Most entries contain a link to the actual text of the law or policy, or to supplementary information that explains a law or policy. The efforts of the CCLW team are regularly supplemented by contributions from lawyers, scholars, policymakers and other colleagues from around the world, who alert us to new data. Please use this link or email us at [email protected] if you wish to contribute.

Principles and limitations

In general, our approach has been to be inclusive and flexible with definitions, to allow for the different regulatory approaches and cultures among 200 countries, and to recognise the elusive boundaries of climate change, which spans multiple sectors and issues. We have also chosen to include policy and strategy documents that do not pass stringent legal tests of being 'legally binding', as omitting them would mean not capturing the country's policy response. Where climate change overlaps significantly with other issues, such as development or disaster risk management, we have attempted to include those laws and policies that will be relevant to both issues.

We aim for the data collected through this resource to be as comprehensive and accurate as possible. However, we do not claim to have identified every relevant law or policy in the countries covered. Language limitations, levels of media coverage, and the specific expertise of our researchers and contributors may all result in information for certain countries being more comprehensive than others. Our current data collection process prioritises legislation pass

Although the Grantham Research Institute has developed rigorous internal protocols to guide the process of data collection, the categorisation of each document has been determined by individuals and has not generally been subject to checks for inter-coder reliability. The records in the database have been published in a wide variety of languages, and researchers frequently use automated translation software to understand the content of these records. The determination of whether a given law is in scope, as well as its categorisation in accordance with the definitions outlined above, may therefore be open to differing interpretations.

As our system of codes and categories evolves to better meet the needs of our user community, new codes may be assigned to records already in the database based on document summaries rather than a full review of the original documents. In some instances, this may mean that our codes and categories do not tell the full picture.

Finally, we acknowledge that our current focus on national level legislation may mean that the database does not capture the full detail of the climate change policy response in many countries, particularly those where a significant quantity of climate action is governed by decision-making at the sub-national level.

We invite users to exercise their judgement when using the data and any data subsets that serve their research and policy purposes.