Open energy system database projects employ

open data Open data is data that is openly accessible, exploitable, editable and shared by anyone for any purpose. Open data is licensed under an open license. The goals of the open data movement are similar to those of other "open(-source)" movements ...

methods to collect, clean, and republish energy-related

dataset A data set (or dataset) is a collection of data. In the case of tabular data, a data set corresponds to one or more database tables, where every column of a table represents a particular variable, and each row corresponds to a given record of the ...

s for open use. The resulting information is then available, given a suitable

open license A free license or open license is a license which allows others to reuse another creator’s work as they wish. Without a special license, these uses are normally prohibited by copyright, patent or commercial license. Most free licenses are ...

, for statistical analysis and for building numerical

energy system An energy system is a system primarily designed to supply energy-services to end-users. The intent behind energy systems is to minimise energy losses to a negligible level, as well as to ensure the efficient use of energy. The IPCC Fifth ...

models A model is an informative representation of an object, person or system. The term originally denoted the plans of a building in late 16th-century English, and derived via French and Italian ultimately from Latin ''modulus'', a measure. Models c ...

, including

open energy system models Open energy system models are energy system models that are open source. However, some of them may use third party proprietary software as part of their workflows to input, process, or output data. Preferably, these models use open data, which ...

. Permissive licenses like

Creative Commons Creative Commons (CC) is an American non-profit organization and international network devoted to educational access and expanding the range of creative works available for others to build upon legally and to share. The organization has release ...

CC0 A Creative Commons (CC) license is one of several public copyright licenses that enable the free distribution of an otherwise copyrighted "work".A "work" is any creative material made by a person. A painting, a graphic, a book, a song/lyric ...

and are preferred, but some projects will house data made public under market transparency regulations and carrying unqualified

copyright A copyright is a type of intellectual property that gives its owner the exclusive right to copy, distribute, adapt, display, and perform a creative work, usually for a limited time. The creative work may be in a literary, artistic, education ...

. The databases themselves may furnish information on national power plant fleets, renewable generation assets, transmission networks,

time series In mathematics, a time series is a series of data points indexed (or listed or graphed) in time order. Most commonly, a time series is a sequence taken at successive equally spaced points in time. Thus it is a sequence of discrete-time data. Exa ...

for electricity loads, dispatch, spot prices, and cross-border trades, weather information, and similar. They may also offer other energy statistics including fossil fuel imports and exports, gas, oil, and coal prices, emissions certificate prices, and information on

energy efficiency Energy efficiency may refer to: * Energy efficiency (physics), the ratio between the useful output and input of an energy conversion process ** Electrical efficiency, useful power output per electrical power consumed ** Mechanical efficiency, a ra ...

costs and benefits. Much of the data is sourced from official or semi-official agencies, including national statistics offices,

transmission system operator File:Electricity grid simple- North America.svg, 380px, Simplified diagram of AC electricity grid from generation stations to consumers rect 2 243 235 438 Power station rect 276 317 412 556 Transformer rect 412 121 781 400 Electric power transmis ...

s, and electricity market operators. Data is also

crowdsourced Crowdsourcing involves a large group of dispersed participants contributing or producing goods or services—including ideas, votes, micro-tasks, and finances—for payment or as volunteers. Contemporary crowdsourcing often involves digit ...

using public

wiki A wiki ( ) is an online hypertext publication collaboratively edited and managed by its own audience, using a web browser. A typical wiki contains multiple pages for the subjects or scope of the project, and could be either open to the pu ...

s and public upload facilities. Projects usually also maintain a strict record of the

provenance Provenance (from the French ''provenir'', 'to come from/forth') is the chronology of the ownership, custody or location of a historical object. The term was originally mostly used in relation to works of art but is now used in similar senses i ...

and version histories of the datasets they hold. Some projects, as part of their mandate, also try to persuade primary data providers to release their data under more liberal licensing conditions. Two drivers favor the establishment of such databases. The first is a wish to reduce the duplication of effort that accompanies each new analytical project as it assembles and processes the data that it needs from primary sources. And the second is an increasing desire to make

public policy Public policy is an institutionalized proposal or a decided set of elements like laws, regulations, guidelines, and actions to solve or address relevant and real-world problems, guided by a conception and often implemented by programs. Public p ...

energy models more transparent to improve their acceptance by policymakers and the public. Better transparency dictates the use of open information, able to be accessed and scrutinized by third-parties, in addition to releasing the

source code In computing, source code, or simply code, is any collection of code, with or without comments, written using a human-readable programming language, usually as plain text. The source code of a program is specially designed to facilitate the wo ...

for the models in question.

General considerations

Background

In the mid-1990s, energy models used structured text files for data interchange but efforts were being made to migrate to

relational database management system A relational database is a (most commonly digital) database based on the relational model of data, as proposed by E. F. Codd in 1970. A system used to maintain relational databases is a relational database management system (RDBMS). Many relatio ...

s for data processing. These early efforts however remained local to a project and did not involve online publishing or open data principles. The first energy information portal to go live was

OpenEI Open Energy Information (OpenEI) is a website for policy makers, researchers, technology investors, venture capitalists, and market professionals with energy data, information, analyses, tools, images, maps, and other resources. It was establ ...

in late 2009, followed by

reegle reegle (lower-case) was a search engine specifically covering the fields of renewable energy, efficient energy use, and climate change issues. It was developed in 2005 by REEEP and REN21, with funding from several European government agencies. At ...

in 2011. A 2012 paper marks the first scientific publication to advocate the

crowdsourcing Crowdsourcing involves a large group of dispersed participants contributing or producing goods or services—including ideas, votes, micro-tasks, and finances—for payment or as volunteers. Contemporary crowdsourcing often involves digita ...

of energy data. The 2012 PhD thesis by Chris Davis also discusses the crowdsourcing of energy data in some depth. A 2016 thesis surveyed the spatial (

GIS A geographic information system (GIS) is a type of database containing Geographic data and information, geographic data (that is, descriptions of phenomena for which location is relevant), combined with Geographic information system software, sof ...

) information requirements for energy planning and finds that most types of data, with the exception of energy expenditure data, are available but nonetheless remain scattered and poorly coordinated. In terms of open data, a 2017 paper concludes that energy research has lagged behind other fields, most notably physics, biotechnology, and medicine. The paper also lists the benefits of open data and open models and discusses the reasons that many projects nonetheless remain closed. A one-page opinion piece from 2017 advances the case for using open energy data and modeling to build public trust in policy analysis. The article also argues that

scientific journals In academic publishing, a scientific journal is a periodical publication intended to further the progress of science, usually by reporting new research. Content Articles in scientific journals are mostly written by active scientists such as s ...

have a responsibility to require that data and code be submitted alongside text for

peer review Peer review is the evaluation of work by one or more people with similar competencies as the producers of the work (peers). It functions as a form of self-regulation by qualified members of a profession within the relevant field. Peer review ...

Database design

Data models are central to the design and organization of databases. Open energy database projects generally try to develop and adhere to well resolved data models, using

defacto ''De facto'' ( ; , "in fact") describes practices that exist in reality, whether or not they are officially recognized by laws or other formal norms. It is commonly used to refer to what happens in practice, in contrast with ''de jure'' ("by la ...

and published standards where applicable. Some projects attempt to coordinate their data models in order to harmonize their data and improve its utility. Defining and maintaining suitable

metadata Metadata is "data that provides information about other data", but not the content of the data, such as the text of a message or the image itself. There are many distinct types of metadata, including: * Descriptive metadata – the descriptive ...

is also a key issue. The life-cycle management of data includes, but is not limited to, the use of

version control In software engineering, version control (also known as revision control, source control, or source code management) is a class of systems responsible for managing changes to computer programs, documents, large web sites, or other collections o ...

to track the provenance of incoming and

cleansed ''Cleansed'' is the third play by the English playwright Sarah Kane. It was first performed in 1998 at the Royal Court Theatre Downstairs in London. The play is set in a university which (according to the blurb of the published script) is op ...

data. Some sites allow users to comment on and rate individual datasets.

Dataset copyright and database rights

Issues surrounding

remain at the forefront with regard to open energy data. As noted, most energy datasets are collated and published by official or semi-official sources. But many of the publicly available energy datasets carry no license, limiting their reuse in numerical and statistical models, open or otherwise. Copyright protected material cannot lawfully be circulated, nor can it be modified and republished. Measures to enforce

market transparency In economics, a market is transparent if much is known by many about: What products and services or capital assets are available, market depth (quantity available), what price, and where. Transparency is important since it is one of the theoreti ...

have not helped much because the associated information is again not licensed to enable modification and republication. Transparency measures include the 2013 European energy market transparency regulation 543/2013. Indeed, 543/2013 "is only an obligation to publish, not an obligation to license". Notwithstanding, 543/2013 does enable downloaded data to be computer processed with legal certainty. Energy databases with hardware located with the European Union are protected under a general database law, irrespective of the legal status of the information they hold. Database rights not waived by public sector providers significantly restrict the amount of data a user can lawfully access. A December 2017 submission by energy researchers in Germany and elsewhere highlighted a number of concerns over the re-use of public sector information within the Europe Union. The submission drew heavily on a recent legal opinion covering electricity data.

Energy statistics

National and international energy statistics are published regularly by governments and international agencies, such as the IEA. In 2016 the United Nations issued guidelines for energy statistics. Annotated as final edited version prior to typesetting. Also covers energy-related greenhouse gas emissions accounting. While the definitions and sectoral breakdowns are useful when defining models, the information provided is rarely in sufficient detail to enable its use in high-resolution energy system models.

Published standards

There are few published standards covering the collection and structuring of high-resolution energy system data. The

IEC The International Electrotechnical Commission (IEC; in French: ''Commission électrotechnique internationale'') is an international standards organization that prepares and publishes international standards for all electrical, electronic and r ...

Common Information Model (CIM) defines data exchange protocols for low and high voltage electricity networks.

Non-open data

Although this page is about genuinely open data, some important databases remain closed. Data collected by the

International Energy Agency The International Energy Agency (IEA) is a Paris-based autonomous intergovernmental organisation, established in 1974, that provides policy recommendations, analysis and data on the entire global energy sector, with a recent focus on curbing carb ...

(IEA) is widely quoted in policy studies but remains nonetheless paywalled. Researchers at

Oxford University Oxford () is a city in England. It is the county town and only city of Oxfordshire. In 2020, its population was estimated at 151,584. It is north-west of London, south-east of Birmingham and north-east of Bristol. The city is home to the ...

have called for this situation to change.

Open energy system database projects

Energy system models are data intensive and normally require detailed information from a number of sources. Dedicated projects to collect, collate, document, and republish energy system datasets have arisen to service this need. Most database projects prefer open data, issued under free licenses, but some will accept datasets with proprietary licenses in the absence of other options. The

OpenStreetMap OpenStreetMap (OSM) is a free, open geographic database updated and maintained by a community of volunteers via open collaboration. Contributors collect data from surveys, trace from aerial imagery and also import from other freely licensed g ...

project, which uses the

Open Database License The Open Database License (ODbL) is a copyleft license agreement intended to allow users to freely share, modify, and use a database while maintaining this same freedom for others. ODbL is published by Open Data Commons, which is part of Open K ...

(ODbL), contains geographic information about energy system components, including transmission lines.

Wikimedia The Wikimedia Foundation, Inc., or Wikimedia for short and abbreviated as WMF, is an American 501(c)(3) nonprofit organization headquartered in San Francisco, California and registered as a charitable foundation under local laws. Best know ...

projects such as

Wikidata Wikidata is a collaboratively edited multilingual knowledge graph hosted by the Wikimedia Foundation. It is a common source of open data that Wikimedia projects such as Wikipedia, and anyone else, can use under the CC0 public domain license ...

and Wikipedia have a growing set of information related to national energy systems, such as descriptions of individual power stations. The following

table Table may refer to: * Table (furniture), a piece of furniture with a flat surface and one or more legs * Table (landform), a flat area of land * Table (information), a data arrangement with rows and columns * Table (database), how the table data ...

summarizes projects that specifically publish open energy system data. Some are general repositories while others (for instance, oedb) are designed to interact with open energy system models in real-time. Three of the projects listed work with

linked open data In computing, linked data (often capitalized as Linked Data) is structured data which is interlinked with other data so it becomes more useful through semantic queries. It builds upon standard Web technologies such as HTTP, RDF and URIs, but r ...

(LOD), a method of publishing structured data on the web so that it can be networked and subject to semantic queries. The overarching concept is termed the semantic web. Technically, such projects support RESTful APIs, RDF, and the

SPARQL SPARQL (pronounced "sparkle" , a recursive acronym for SPARQL Protocol and RDF Query Language) is an RDF query language—that is, a semantic query language for databases—able to retrieve and manipulate data stored in Resource Description F ...

query language. A 2012 paper reviews the use of LOD in the renewable energy domain.

Climate Compatible Growth starter datasets

The Climate Compatible Growth (CCG) programme provides starter kits for the following 69countries: Algeria, Angola, Argentina, Benin, Botswana, Bolivia, Brazil, Burkina Faso, Burundi, Cambodia, Cameroon, Central African Republic, Chad, Chile, Colombia, Côte d'Ivoire, Democratic Republic of Congo, Djibouti, Ecuador, Egypt, Equatorial Guinea, Eritrea, Eswatini, Ethiopia, Gabon, Gambia, Ghana, Guinea, Guinea-Bissau, Indonesia, Kenya, Laos, Lesotho, Liberia, Libya, Malawi, Malaysia, Mali, Mauritania, Morocco, Mozambique, Myanmar, Namibia, Niger, Nigeria, Papua New Guinea, Paraguay, Peru, Philippines, Republic of Congo, Republic of Korea, Rwanda, Senegal, Sierra Leone, Somalia, South Africa, South Sudan, Sudan, Taiwan, Tanzania, Thailand, Togo, Tunisia, Uganda, Uruguay, Venezuela, VietNam, Zambia, and Zimbabwe. The datasets are hosted on the

Zenodo Zenodo is a general-purpose open repository developed under the European OpenAIRE program and operated by CERN. It allows researchers to deposit research papers, data sets, research software, reports, and any other research related digital artef ...

science archive site, visit that site and search for "ccg starter kit".

Energy Research Data Portal for South Africa

The Energy Research Data Portal for South Africa is being developed by th
Energy Research Centre

University of Cape Town The University of Cape Town (UCT) ( af, Universiteit van Kaapstad, xh, Yunibesithi ya yaseKapa) is a public research university in Cape Town, South Africa. Established in 1829 as the South African College, it was granted full university statu ...

Cape Town Cape Town ( af, Kaapstad; , xh, iKapa) is one of South Africa's three capital cities, serving as the seat of the Parliament of South Africa. It is the legislative capital of the country, the oldest city in the country, and the second largest ...

, South Africa. Coverage includes South Africa and certain other African countries where the Centre undertakes projects. The website uses the

CKAN The Comprehensive Knowledge Archive Network (CKAN) is an open-source open data portal for the storage and distribution of open data. Being initially inspired by the package management capabilities of Debian Linux, CKAN has developed into a powerfu ...

open source data portal software. A number of data formats are supported, including CSV and

XLSX Office Open XML (also informally known as OOXML) is a zipped, XML-based file format developed by Microsoft for representing spreadsheets, charts, presentations and word processing documents. Ecma International standardized the initial version ...

. The site also offers an

API An application programming interface (API) is a way for two or more computer programs to communicate with each other. It is a type of software interface, offering a service to other pieces of software. A document or standard that describes how ...

for automated downloads. , the portal contained 65datasets.

energydata.info

The energydata.info project from the

World Bank Group The World Bank Group (WBG) is a family of five international organizations that make leveraged loans to developing countries. It is the largest and best-known development bank in the world and an observer at the United Nations Development Grou ...

Washington, DC ) , image_skyline = , image_caption = Clockwise from top left: the Washington Monument and Lincoln Memorial on the National Mall, United States Capitol, Logan Circle, Jefferson Memorial, White House, Adams Morgan ...

, USA is an energy database portal designed to support national development by improving public access to energy information. As well as sharing data, the platform also offers tools to visualize and analyze energy data. Although the World Bank Group has made available a number of dataset and apps, external users and organizations are encouraged to contribute. The concepts of open data and open source development are central to the project. energydata.info uses its own

fork In cutlery or kitchenware, a fork (from la, furca 'pitchfork') is a utensil, now usually made of metal, whose long handle terminates in a head that branches into several narrow and often slightly curved tine (structural), tines with which one ...

of the

open source data portal as its web-based platform. The Creative Commons

CC BY 4.0 A Creative Commons (CC) license is one of several public copyright licenses that enable the free distribution of an otherwise copyrighted "work".A "work" is any creative material made by a person. A painting, a graphic, a book, a song/lyric ...

license is preferred for data but other open licenses can be deployed. Users are also bound by the terms of use for the site. , the database held 131datasets, the great majority related to

developing countries A developing country is a sovereign state with a lesser developed industrial base and a lower Human Development Index (HDI) relative to other countries. However, this definition is not universally agreed upon. There is also no clear agreem ...

. The datasets are tagged and can be easily filtered. A number of download formats, including

files, are supported: CSV, XLS,

ArcGIS ArcGIS is a family of client, server and online geographic information system (GIS) software developed and maintained by Esri. ArcGIS was first released in 1999 and originally was released as ARC/INFO, a command line based GIS system for manipul ...

Esri Esri (; Environmental Systems Research Institute) is an American multinational geographic information system (GIS) software company. It is best known for its ArcGIS products. With a 43% market share, Esri is the world's leading supplier of GIS ...

GeoJSON GeoJSON is an open standard format designed for representing simple geographical features, along with their non-spatial attributes. It is based on the JSON format. The features include points (therefore addresses and locations), line strings ( ...

KML Keyhole Markup Language (KML) is an XML notation for expressing geographic annotation and visualization within two-dimensional maps and three-dimensional Earth browsers. KML was developed for use with Google Earth, which was originally named Key ...

, and SHP. Some datasets are also offered as

HTML The HyperText Markup Language or HTML is the standard markup language for documents designed to be displayed in a web browser. It can be assisted by technologies such as Cascading Style Sheets (CSS) and scripting languages such as JavaScri ...

. Again, , four apps are available. Some are web-based and run from a browser.

Enipedia

The semantic wiki-site and

database In computing, a database is an organized collection of data stored and accessed electronically. Small databases can be stored on a file system, while large databases are hosted on computer clusters or cloud storage. The design of databases sp ...

Enipedia lists energy systems data worldwide. Chapter9 discusses in depth the initial development of Enipedia. Enipedia is maintained by th
Energy and Industry Group
Faculty of Technology, Policy and Management,

Delft University of Technology Delft University of Technology ( nl, Technische Universiteit Delft), also known as TU Delft, is the oldest and largest Dutch public technical university, located in Delft, Netherlands. As of 2022 it is ranked by QS World University Rankings among ...

Delft Delft () is a List of cities in the Netherlands by province, city and Municipalities of the Netherlands, municipality in the Provinces of the Netherlands, province of South Holland, Netherlands. It is located between Rotterdam, to the southeast, ...

, the Netherlands. A key tenet of Enipedia is that data displayed on the wiki is not trapped within the wiki, but can be extracted via

queries and used to populate new tools. Any programming environment that can download content from a URL can be used to obtain data. Enipedia went live in March 2011, judging by traffic figures quoted by Davis. A 2010 study describes how community driven data collection, processing, curation, and sharing is revolutionizing the data needs of

industrial ecology Industrial ecology (IE) is the study of material and energy flows through industrial systems. The global industrial economy can be modelled as a network of industrial processes that extract resources from the Earth and transform those resource ...

and energy system analysis. A 2012 chapter introduces a

system of systems engineering System of systems engineering (SoSE) is a set of developing processes, tools, and methods for designing, re-designing and deploying solutions to system-of-systems challenges. Overview System of Systems Engineering (SoSE) methodology is heavily u ...

(SoSE) perspective and outlines how agent-based models and crowdsourced data can contribute to the solving of global issues. , the site has gone offline pending a move to the domain.

Open Energy Platform

The Open Energy Platform (OEP) is a collaborative versioned

repository for storing open energy system model datasets. A dataset is presumed to be in the form of a

database table A table is a collection of related data held in a table format within a database. It consists of columns and rows. In relational databases, and flat file databases, a ''table'' is a set of data elements (values) using a model of vertical colum ...

, together with

. Registered users can upload and download datasets manually using a web-interface or programmatically via an

using

HTTP The Hypertext Transfer Protocol (HTTP) is an application layer protocol in the Internet protocol suite model for distributed, collaborative, hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web, ...

POST Post or POST commonly refers to: *Mail, the postal system, especially in Commonwealth of Nations countries **An Post, the Irish national postal service **Canada Post, Canadian postal service **Deutsche Post, German postal service **Iraqi Post, Ira ...

calls. Uploaded datasets are screened for integrity using deterministic rules and then subject to confirmation by a moderator. The use of versioning means that any prior state of the database can be accessed (as recommended in this 2012 paper). Hence, the repository is specifically designed to interoperate with energy system models. The backend is a

PostgreSQL PostgreSQL (, ), also known as Postgres, is a free and open-source relational database management system (RDBMS) emphasizing extensibility and SQL compliance. It was originally named POSTGRES, referring to its origins as a successor to the In ...

object-relational database under subversion

. Open-data licenses are specific to each dataset. Unlike other database projects, users can download the current version (the public tables) of the entire

database or any previous version. The development is being led by a cross-project community.

Open Data Energy Networks

The Open Data Energy Networks ( or ) portal is run by eight partners, led by the French national

(TSO)

Réseau de Transport d'Électricité Réseau de Transport d'Électricité ("Electricity Transmission Network"), usually known as RTE, is the electricity transmission system operator of France. It is responsible for the operation, maintenance and development of the French high-voltage ...

(RTE). The portal was previously known as Open Data RTE. The site offers electricity system datasets under a Creative Commons compatible license, with

, an

RSS RSS ( RDF Site Summary or Really Simple Syndication) is a web feed that allows users and applications to access updates to websites in a standardized, computer-readable format. Subscribing to RSS feeds can allow a user to keep track of many di ...

feed for notifying updates, and an interface for submitting questions. of information obtained from the site can also register third-party URLs (be they publications or webpages) against specific datasets. The portal uses the French Government Licence Ouverte license and this is explicitly compatible with the United Kingdom

Open Government Licence The Open Government Licence is a copyright licence for Crown copyright works published by the UK government. Other UK public sector bodies may apply it to their publications. It was developed and is maintained by The National Archives. It is co ...

(OGL), the Creative Commons license (and thereby later versions), and the Open Data Commons license. The site hosts electricity, gas, and weather information related to France.

UK Power Networks Open Data Portal

The Open Data Portal is run by UK Power Networks, a GB

Distribution Network Operator A distribution network operator (DNO), also known as a distribution system operator (DSO), is the operator of the electric power distribution system which delivers electricity to most end users. Each country may have many local distribution networ ...

(DNO), hosted on the OpenDataSoft platform. The Portal offers electricity network datasets under a Creative Commons compatible license, with

, a newsfeed, and a data request form. of information obtained from the site can also register third-party URLs (be they publications or webpages) against specific datasets. A number of download formats, including

files, are supported: CSV, XLS,

, and SHP. The site also offers an API for automated downloads. The portal uses the

License and also hosts datasets from other sources which are licensed under the

(OGL). The site hosts electricity datasets related to UK Power Networks' three license areas in London, the East and South East of England.

Open Power System Data

The Open Power System Data (OPSD) project seeks to characterize the

German German(s) may refer to: * Germany (of or related to) ** Germania (historical use) * Germans, citizens of Germany, people of German ancestry, or native speakers of the German language ** For citizens of Germany, see also German nationality law **Ge ...

and

western Europe Western Europe is the western region of Europe. The region's countries and territories vary depending on context. The concept of "the West" appeared in Europe in juxtaposition to "the East" and originally applied to the ancient Mediterranean ...

power plant A power station, also referred to as a power plant and sometimes generating station or generating plant, is an industrial facility for the generation of electric power. Power stations are generally connected to an electrical grid. Many pow ...

fleets, their associated

transmission network Electric power transmission is the bulk movement of electrical energy from a generating site, such as a power plant, to an electrical substation. The interconnected lines that facilitate this movement form a ''transmission network''. This is d ...

, and related information and to make that data available to energy modelers and analysts. The platform was originally implemented by the

University of Flensburg The University of Flensburg (''Europa-Universität Flensburg'') is a university in the city of Flensburg, Germany. It was founded in 1994 and is the northernmost university in Germany. Although having full university status and the right to award ...

, DIW Berlin, the

Technical University of Berlin The Technical University of Berlin (official name both in English and german: link=no, Technische Universität Berlin, also known as TU Berlin and Berlin Institute of Technology) is a public research university located in Berlin, Germany. It was ...

, and the energy economics consultancy Neon Neue Energieökonomik, all from Germany. The first phase of the project, from August 2015 to July 2017, was funded by the

Federal Ministry for Economic Affairs and Energy The Federal Ministry for Economic Affairs and Climate Action (german: Bundesministerium für Wirtschaft und Klimaschutz, ), abbreviated BMWK (was BMWi), is a cabinet-level ministry of the Federal Republic of Germany. It was previously known as ...

(BMWi) for . The project later received funding for a second phase, from January 2018 to December 2020, with

ETH Zurich (colloquially) , former_name = eidgenössische polytechnische Schule , image = ETHZ.JPG , image_size = , established = , type = Public , budget = CHF 1.896 billion (2021) , rector = Günther Dissertori , president = Joël Mesot , ac ...

replacing Flensburg University as a partner. Developers collate and harmonize data from a range of government, regulatory, and industry sources throughout Europe. The website and the metadata utilize English, whereas the original material can be in any one of 24languages. Datasets follow the emerging frictionless data package standard being developed by

Open Knowledge Foundation Open Knowledge Foundation (OKF) is a global, non-profit network that promotes and shares information at no charge, including both content and data. It was founded by Rufus Pollock on 20 May 2004 in Cambridge, UK. It is incorporated in England a ...

(OKF). The website was launched on 28October 2016. , the project offers the following primary packages, for Germany and other European countries: * details, including

geolocation Geopositioning, also known as geotracking, geolocalization, geolocating, geolocation, or geoposition fixing, is the process of determining or estimating the geographic position of an object. Geopositioning yields a set of Geographic coordinate s ...

, of conventional power plants and renewable energy power plants * aggregated generation capacity by technology and country * hourly

covering electrical load, day-ahead electricity spot prices, and wind and solar resources * a script to filter and download

NASA The National Aeronautics and Space Administration (NASA ) is an independent agency of the US federal government responsible for the civil space program, aeronautics research, and space research. NASA was established in 1958, succeeding t ...

MERRA-2 satellite weather data In addition, the project hosts selected contributed packages: * electricity demand and self-generation time series for representative south German households * simulated PV and wind generation

capacity factor The net capacity factor is the unitless ratio of actual electrical energy output over a given period of time to the theoretical maximum electrical energy output over that period. The theoretical maximum energy output of a given installation is def ...

time series for Europe, generated by the Renewables.ninja project To facilitate analysis, the data is aggregated into large structured files (in CSV format) and loaded into data packages with standardized machine-readable metadata (in

JSON JSON (JavaScript Object Notation, pronounced ; also ) is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute–value pairs and arrays (or other ser ...

format). The same data is usually also provided as

(

Excel ExCeL London (an abbreviation for Exhibition Centre London) is an exhibition centre, international convention centre and former hospital in the Custom House area of Newham, East London. It is situated on a site on the northern quay of the ...

) and

SQLite SQLite (, ) is a database engine written in the C programming language. It is not a standalone app; rather, it is a library that software developers embed in their apps. As such, it belongs to the family of embedded databases. It is the m ...

files. The datasets can be accessed in real-time using stable

URLs A Uniform Resource Locator (URL), colloquially termed as a web address, is a reference to a web resource that specifies its location on a computer network and a mechanism for retrieving it. A URL is a specific type of Uniform Resource Identifi ...

. The

Python Python may refer to: Snakes * Pythonidae, a family of nonvenomous snakes found in Africa, Asia, and Australia ** ''Python'' (genus), a genus of Pythonidae found in Africa and Asia * Python (mythology), a mythical serpent Computing * Python (pro ...

scripts deployed for data processing are available on

GitHub GitHub, Inc. () is an Internet hosting service for software development and version control using Git. It provides the distributed version control of Git plus access control, bug tracking, software feature requests, task management, continuous ...

and carry an

MIT license The MIT License is a permissive free software license originating at the Massachusetts Institute of Technology (MIT) in the late 1980s. As a permissive license, it puts only very limited restriction on reuse and has, therefore, high license comp ...

. The licensing conditions for the data itself depends on the source and varies in terms of openness. Previous versions of the datasets and scripts can be recovered in order to track changes or replicate earlier studies. The project also engages with energy data providers, such as

s (TSO) and

ENTSO-E ENTSO-E, the European Network of Transmission System Operators, represents 39 electricity transmission system operators (TSOs) from 35 countries across Europe, thus extending beyond EU borders. ENTSO-E was established and given legal mandates b ...

, to encourage them to make their data available under

open licenses A free license or open license is a license which allows others to reuse another creator’s work as they wish. Without a special license, these uses are normally prohibited by copyright, patent or commercial license. Most free licenses are wo ...

(for instance,

and

ODbL The Open Database License (ODbL) is a copyleft license agreement intended to allow users to freely share, modify, and use a database while maintaining this same freedom for others. ODbL is published by Open Data Commons, which is part of Open Kn ...

licenses). In a 2019 publication, OPSD developers describe their design choices, implementation, and provisioning.
Postprint
Information integrity remains key, with each data package having traceable provenance, curation, and packing. From October 2018, each new or revised data package is assigned a unique DOI to ensure that external references to current and prior versions remain stable. A number of published electricity market modeling analyses are based on OPSD data. In 2017, the Open Power System Data project won the Schleswig-Holstein Open Science Award and the Germany Land of Ideas award.

OpenEI

Open Energy Information (OpenEI) is a collaborative

website A website (also written as a web site) is a collection of web pages and related content that is identified by a common domain name and published on at least one web server. Examples of notable websites are Google Search, Google, Facebook, Amaz ...

, run by the US government, providing

open Open or OPEN may refer to: Music * Open (band), Australian pop/rock band * The Open (band), English indie rock band * ''Open'' (Blues Image album), 1969 * ''Open'' (Gotthard album), 1999 * ''Open'' (Cowboy Junkies album), 2001 * ''Open'' (YF ...

energy data to software developers, analysts, users, consumers, and policymakers. The platform is sponsored by the

United States Department of Energy The United States Department of Energy (DOE) is an executive department of the U.S. federal government that oversees U.S. national energy policy and manages the research and development of nuclear power and nuclear weapons in the United Stat ...

(DOE) and is being developed by the

National Renewable Energy Laboratory The National Renewable Energy Laboratory (NREL) in the US specializes in the research and development of renewable energy, energy efficiency, energy systems integration, and sustainable transportation. NREL is a federally funded research and d ...

(NREL). OpenEI launched on 9December 2009. While much of its data is from US government sources, the platform is intended to be open and global in scope. OpenEI provides two mechanisms for contributing structured information: a

semantic wiki A semantic wiki is a wiki that has an underlying model of the knowledge described in its pages. Regular, or syntactic, wikis have structured text and untyped hyperlinks. Semantic wikis, on the other hand, provide the ability to capture or identif ...

(using

MediaWiki MediaWiki is a free and open-source wiki software. It is used on Wikipedia and almost all other Wikimedia websites, including Wiktionary, Wikimedia Commons and Wikidata; these sites define a large part of the requirement set for MediaWiki ...

and the

Semantic MediaWiki Semantic MediaWiki (SMW) is an extension to MediaWiki that allows for annotating semantic data within wiki pages, thus turning a wiki that incorporates the extension into a semantic wiki. Data that has been encoded can be used in semantic search ...

extension) for collaboratively-managed resources and a

upload facility for contributor-controlled resources. US government data is distributed under a

public domain The public domain (PD) consists of all the creative work A creative work is a manifestation of creative effort including fine artwork (sculpture, paintings, drawing, sketching, performance art), dance, writing (literature), filmmaking, ...

dedication, whereas other contributors are free to select an open data license of their choice. Users can rate data using a five-star system, based on accessibility, adaptability, usefulness, and general quality. Individual datasets can be manually downloaded in an appropriate format, often as CSV files.

Scripts Script may refer to: Writing systems * Script, a distinctive writing system, based on a repertoire of specific elements or symbols, or that repertoire * Script (styles of handwriting) ** Script typeface, a typeface with characteristics of handw ...

for processing data can also be shared through the site. In order to build a community around the platform, a number of forums are offered covering energy system data and related topics. Most of the data on OpenEI is exposed as

(LOD) (described elsewhere on this page). OpenEI also uses LOD methods to populate its definitions throughout the wiki with real-time connections to

DBPedia DBpedia (from "DB" for " database") is a project aiming to extract structured content from the information created in the Wikipedia project. This structured information is made available on the World Wide Web. DBpedia allows users to semanti ...

, and

Wikipedia Wikipedia is a multilingual free online encyclopedia written and maintained by a community of volunteers, known as Wikipedians, through open collaboration and using a wiki-based editing system. Wikipedia is the largest and most-read refer ...

. OpenEI has been used to classify geothermal resources in the United States. And to publicize municipal utility rates, again within the US.

OpenGridMap

OpenGridMap employs

techniques to gather detailed data on electricity network components and then infer a realistic network structure using methods from

statistics Statistics (from German language, German: ''wikt:Statistik#German, Statistik'', "description of a State (polity), state, a country") is the discipline that concerns the collection, organization, analysis, interpretation, and presentation of ...

and

graph theory In mathematics, graph theory is the study of ''graphs'', which are mathematical structures used to model pairwise relations between objects. A graph in this context is made up of '' vertices'' (also called ''nodes'' or ''points'') which are conne ...

. The scope of the project is worldwide and both distribution and transmission networks can be

reverse engineered Reverse engineering (also known as backwards engineering or back engineering) is a process or method through which one attempts to understand through deductive reasoning how a previously made device, process, system, or piece of software accompli ...

. The project is managed by the Chair of Business Information Systems
TUM Department of Informatics

Technical University of Munich The Technical University of Munich (TUM or TU Munich; german: Technische Universität München) is a public research university in Munich, Germany. It specializes in engineering, technology, medicine, and applied and natural sciences. Establis ...

Munich Munich ( ; german: München ; bar, Minga ) is the capital and most populous city of the States of Germany, German state of Bavaria. With a population of 1,558,395 inhabitants as of 31 July 2020, it is the List of cities in Germany by popu ...

, Germany. The project maintains a website and a

Facebook Facebook is an online social media and social networking service owned by American company Meta Platforms. Founded in 2004 by Mark Zuckerberg with fellow Harvard College students and roommates Eduardo Saverin, Andrew McCollum, Dustin M ...

page and provides an Android

mobile app A mobile application or app is a computer program or software application designed to run on a mobile device such as a phone, tablet, or watch. Mobile applications often stand in contrast to desktop applications which are designed to run on d ...

to help the public document electrical devices, such as

transformer A transformer is a passive component that transfers electrical energy from one electrical circuit to another circuit, or multiple circuits. A varying current in any coil of the transformer produces a varying magnetic flux in the transformer' ...

s and substations. The bulk of the data is being made available under a

license. Terms of use last amended 25November 2016. The processing software is written primarily in

and

MATLAB MATLAB (an abbreviation of "MATrix LABoratory") is a proprietary multi-paradigm programming language and numeric computing environment developed by MathWorks. MATLAB allows matrix manipulations, plotting of functions and data, implementation ...

and is hosted on

. OpenGridMap provides a tailored

web application, layered on

, which contributors can use to upload and edit information directly. The same database automatically stores field recordings submitted by the mobile app. Subsequent classification by experts allows normal citizens to document and photograph electrical components and have them correctly identified. The project is experimenting with the use of hobby drones to obtain better information on associated facilities, such as

photovoltaic Photovoltaics (PV) is the conversion of light into electricity using semiconducting materials that exhibit the photovoltaic effect, a phenomenon studied in physics, photochemistry, and electrochemistry. The photovoltaic effect is commercially us ...

installations. Transmission line data is also sourced from and shared with OpenStreetMap. Each component record is verified by a moderator. Once sufficient data is available, the transnet software is run to produce a likely network, using

statistical correlation In statistics, correlation or dependence is any statistical relationship, whether causal or not, between two random variables or bivariate data. Although in the broadest sense, "correlation" may indicate any type of association, in statistics ...

, Voronoi partitioning, and

minimum spanning tree A minimum spanning tree (MST) or minimum weight spanning tree is a subset of the edges of a connected, edge-weighted undirected graph that connects all the vertices together, without any cycles and with the minimum possible total edge weight. T ...

(MST) algorithms. The resulting network can be exported in CSV (separate files for nodes and lines),

XML Extensible Markup Language (XML) is a markup language and file format for storing, transmitting, and reconstructing arbitrary data. It defines a set of rules for encoding documents in a format that is both human-readable and machine-readable ...

, and CIM formats. CIM models are well suited for translation into software-specific data formats for further analysis, including power grid simulation. Transnet also displays

descriptive statistics A descriptive statistic (in the count noun sense) is a summary statistic that quantitatively describes or summarizes features from a collection of information, while descriptive statistics (in the mass noun sense) is the process of using and an ...

about the resulting network for visual confirmation. The project is motivated by the need to provide datasets for high-resolution energy system models, so that energy system transitions (like the German ''

Energiewende The ''Energiewende'' (; ) is the ongoing transition by Germany to a low carbon, environmentally sound, reliable, and affordable energy supply. The new system intends to rely heavily on renewable energy (particularly wind, photovoltaics, and ...

'') can be better managed, both technically and policy-wise. The rapid expansion of renewable generation and the anticipated uptake of

electric vehicle An electric vehicle (EV) is a vehicle that uses one or more electric motors for propulsion. It can be powered by a collector system, with electricity from extravehicular sources, or it can be powered autonomously by a battery (sometimes cha ...

s means that electricity system models must increasingly represent distribution and transmission networks in some detail. , OpenGridMap techniques have been used to estimate the low voltage network in the German city of Garching and to estimate the high voltage grids in several other countries.

Power Explorer

The Power Explorer portal is a part of the larger Resource Watch platform, hosted by the

World Resources Institute The World Resources Institute (WRI) is a global research non-profit organization established in 1982 with funding from the MacArthur Foundation under the leadership of James Gustave Speth. WRI's activities are focused on seven areas: food, for ...

. The initial Global Power Plant Database, an open source database of the power plants globally, was released in April 2018. Download page. Newer versions available via same link. , the portal itself is still under development. Power Explorer is also supported by

Google Google LLC () is an American multinational technology company focusing on search engine technology, online advertising, cloud computing, computer software, quantum computing, e-commerce, artificial intelligence, and consumer electronics. ...

with various research partners, including

KTH KTH may refer to: * Keat Hong LRT station, Singapore, LRT station abbreviation * Kent House railway station, London, National Rail station code * KTH Royal Institute of Technology, a university in Sweden * KTH Krynica, a Polish ice hockey team * Khy ...

, Global Energy Observatory, Enipedia, and OPSD.

PowerGenome

The PowerGenome project aims to provide a coherent dataset covering the United States electricity system. PowerGenome was initially designed to service the

GenX The General Electric GEnx ("General Electric Next-generation") is an advanced dual rotor, axial flow, high-bypass turbofan jet engine in production by GE Aviation for the Boeing 787 and 747-8. The GEnx is intended to succeed the CF6 in GE's pro ...

model, MITEI‑WP‑2017‑10. but support for other modeling frameworks is in planning. The PowerGenome utility also pulls from upstream datasets hosted by the Public Utility Data Liberation project (PUDL) and the

EIA Eia or EIA may refer to: Medicine * Enzyme immunoassay * Equine infectious anemia * Exercise-induced anaphylaxis * Exercise-induced asthma * External iliac artery Transport * Edmonton International Airport, in Alberta, Canada * Erbil Internation ...

, so those dependencies need to be met by users. Datasets are occasionally archived on

. Snapshot. A video describing the project is available. MP4 webcast 00:10:55.

reegle

is a

clean energy Clean may refer to: * Cleaning, the process of removing unwanted substances, such as dirt, infectious agents, and other impurities, from an object or environment * Cleanliness, the state of being clean and free from dirt Arts and media Music A ...

information

portal Portal often refers to: * Portal (architecture), an opening in a wall of a building, gate or fortification, or the extremities (ends) of a tunnel Portal may also refer to: Arts and entertainment Gaming * ''Portal'' (series), two video games ...

covering

renewable energy Renewable energy is energy that is collected from renewable resources that are naturally replenished on a human timescale. It includes sources such as sunlight, wind, the movement of water, and geothermal heat. Although most renewable energy ...

, and climate compatible development topics. reegle was launched in 2006 by REEEP and

REN21 REN21 (Renewable Energy Policy Network for the 21st Century) is a think tank and a multistakeholder governance group which is focused on renewable energy policy. REN21's goal is to facilitate policy development, knowledge exchange, and joint ac ...

with funding from the Dutch (

VROM The Ministry of Housing, Spatial Planning and the Environment ( nl, Ministerie van Volkshuisvesting, Ruimtelijke Ordening en Milieu or ''VROM'') was a Dutch ministry. It was responsible for policies on public housing, spatial planning, the envi ...

), German ( BMU), and UK (

Defra DEFRA may refer to: * Deficit Reduction Act of 1984, United States law * Department for Environment, Food and Rural Affairs, United Kingdom government department {{Disambiguation ...

) environment ministries. Originally released as a specialized

internet search engine A search engine is a software system designed to carry out web searches. They search the World Wide Web in a systematic way for particular information specified in a textual web search query. The search results are generally presented in a l ...

, reegle was relaunched in 2011 as an information portal. reegle offers and utilizes

(LOD) (described elsewhere on this page). Sources of data include UN and

World Bank The World Bank is an international financial institution that provides loans and grants to the governments of low- and middle-income countries for the purpose of pursuing capital projects. The World Bank is the collective name for the Interna ...

databases, as well as dedicated partners around the world. reegle maintains a comprehensive structured

glossary A glossary (from grc, γλῶσσα, ''glossa''; language, speech, wording) also known as a vocabulary or clavis, is an alphabetical list of Term (language), terms in a particular domain of knowledge with the definitions for those terms. Tradi ...

(driven by an LOD-compliant

thesaurus A thesaurus (plural ''thesauri'' or ''thesauruses'') or synonym dictionary is a reference work for finding synonyms and sometimes antonyms of words. They are often used by writers to help find the best word to express an idea: Synonym diction ...

) of energy and climate compatible development terms to assist with the tagging of datasets. The glossary also facilitates intelligent web searches. reegle offers country profiles which collate and display energy data on a per-country basis for most of the world. These profiles are kept current automatically using LOD techniques. As of 2021, the portal is no longer active.

Renewables.ninja

Renewables.ninja is a website that can calculate the hourly power output from

solar photovoltaic A photovoltaic system, also PV system or solar power system, is an electric power system designed to supply usable solar power by means of photovoltaics. It consists of an arrangement of several components, including solar panels to absorb and c ...

installations and

wind farm A wind farm or wind park, also called a wind power station or wind power plant, is a group of wind turbines in the same location used Wind power, to produce electricity. Wind farms vary in size from a small number of turbines to several hundre ...

s located anywhere in the world. The website is a joint project between th
Department of Environmental Systems Science

Zürich Zürich () is the list of cities in Switzerland, largest city in Switzerland and the capital of the canton of Zürich. It is located in north-central Switzerland, at the northwestern tip of Lake Zürich. As of January 2020, the municipality has 43 ...

, Switzerland and th
Centre for Environmental Policy

Imperial College London Imperial College London (legally Imperial College of Science, Technology and Medicine) is a public research university in London, United Kingdom. Its history began with Prince Albert, consort of Queen Victoria, who developed his vision for a cu ...

London London is the capital and largest city of England and the United Kingdom, with a population of just under 9 million. It stands on the River Thames in south-east England at the head of a estuary down to the North Sea, and has been a majo ...

, United Kingdom. The website went live during September 2016. The resulting time series are provided under a Creative Commons license (which is unfortunately not open data conformant) and the underlying power plant models are published using a BSD-new license. , only the solar model, written in

, has been released. Renewables ninja photovoltaics and wind energy database screenshot showing simulated pv output

The project relies on weather data derived from

meteorological reanalysis An atmospheric reanalysis (also: meteorological reanalysis and climate reanalysis) is a meteorological and climate data assimilation project which aims to assimilate historical atmospheric observational data spanning an extended period, using a s ...

models and weather

satellite images Satellite images (also Earth observation imagery, spaceborne photography, or simply satellite photo) are images of Earth collected by imaging satellites operated by governments and businesses around the world. Satellite imaging companies sell ima ...

. More specifically, it uses the 2016 MERRA-2 reanalysis dataset from

and satellite images from CM-SAF SARAH. Contains datasets from 1983 to 2013. File size . For locations in Europe, this weather data is further "corrected" by country so that it better fits with the output from known PV installations and windfarms. Two 2016 papers describe the methods used in detail in relation to Europe. The first covers the calculation of PV power. And the second covers the calculation of wind power. The website displays an interactive world map to aid the selection of a site. Users can then choose a plant type and enter some technical characteristics. , only year 2014 data can be served, due to technical restrictions. The results are automatically plotted and are available for download in hourly CSV format with or without the associated weather information. The site offers an

for programmatic dataset recovery using token-based authorization. Examples deploying cURL and

are provided. A number of studies have been undertaking using the power production datasets underpinning the website (these studies predate the launch of the website), with the bulk focusing on energy options for

Great Britain Great Britain is an island in the North Atlantic Ocean off the northwest coast of continental Europe. With an area of , it is the largest of the British Isles, the largest European island and the ninth-largest island in the world. It is ...

. .

SMARD

The SMARD site (pronounced "smart") serves electricity market data from

Germany Germany,, officially the Federal Republic of Germany, is a country in Central Europe. It is the second most populous country in Europe after Russia, and the most populous member state of the European Union. Germany is situated betwe ...

Austria Austria, , bar, Östareich officially the Republic of Austria, is a country in the southern part of Central Europe, lying in the Eastern Alps. It is a federation of nine states, one of which is the capital, Vienna, the most populous ...

, and

Luxembourg Luxembourg ( ; lb, Lëtzebuerg ; french: link=no, Luxembourg; german: link=no, Luxemburg), officially the Grand Duchy of Luxembourg, ; french: link=no, Grand-Duché de Luxembourg ; german: link=no, Großherzogtum Luxemburg is a small lan ...

and also provides visual information. The electricity market plots and their underlying

are released under a permissive

license. Terms of use for the BNetzA SMARD data portal. The site itself was launched on 3July 2017 in German and an English translation followed shortly. The data portal is mandated under the German Energy Industry Act ('' Energiewirtschaftsgesetz'' or ''EnWG'') section §111d, introduced as an amendment on 13October 2016. Four table formats are offered: CSV, XLS,

, and

PDF Portable Document Format (PDF), standardized as ISO 32000, is a file format developed by Adobe in 1992 to present documents, including text formatting and images, in a manner independent of application software, hardware, and operating systems. ...

. The maximum sampling resolution is . Market data visuals or plots can be downloaded in PDF, SVG, PNG, and JPG formats. Representative output is shown in the thumbnail (on the left), in this case mid-winter dispatch over two days for the whole of Germany. The horizontal ordering by generation type is first split into

renewable A renewable resource, also known as a flow resource, is a natural resource which will replenish to replace the portion depleted by usage and consumption, either through natural reproduction or other recurring processes in a finite amount of ti ...

and conventional generation and then based on

merit Merit may refer to: Religion * Merit (Christianity) * Merit (Buddhism) * Punya (Hinduism) * Imputed righteousness in Reformed Christianity Companies and brands * Merit (cigarette), a brand of cigarettes made by Altria * Merit Energy Company, ...

. Auser guide is updated as required.

Notes

References

Further information

Open energy data wiki
maintained by the

* The list is under a Creative Commons CC‑BY‑4.0 license and many of the datasets cited are similarly licensed.

External links

De-risking Energy Efficiency Platform
(DEEP) – an open energy efficiency data platform for Europe
European Climatic Energy Mixes project
(ECEM) — the role that climate change may play on future energy systems
OpenEnergy Database
(oedb) – an open energy system database being developed in Germany
OpenEnergyMonitor
– an open source energy use monitoring project
Domain‑wide data projects
– a list of data related projects designed to support open energy system modeling {{Energy modeling Energy Energy models Energy policy Open data Open science Systems theory

General considerations

Background

Database design

Dataset copyright and database rights

Energy statistics

Published standards

Non-open data

Open energy system database projects

Climate Compatible Growth starter datasets

Energy Research Data Portal for South Africa

energydata.info

Enipedia

Open Energy Platform

Open Data Energy Networks

UK Power Networks Open Data Portal

Open Power System Data

OpenEI

OpenGridMap

Power Explorer

PowerGenome

reegle

Renewables.ninja

SMARD

See also

Notes

References

Further information

External links