Publishing Open Data

The purpose of open data is to allow others to analyse and find patterns that may or may not be useful. Central to this is the ability to automate. Especially interesting in this case, because the data is distributed around the World.

The structures defined in this specification allow the construction of automated tests of carbon budgets, and they refernce documents that play a role in the way the budgets can develop over time. This works much better than a pile of related documents that happen to contain data. Especially when the documents are written in a diversity of languages! It is much more practical to see invalid applications of accounting rules trigger an alert, and then dig up the corresponding documentation for explanation.

The use of PGP signatures over JSON data, together with fingerprints of PGP keys, allows the buildup of a chain of trust that starts with the local government, or the country, or perhaps the UN, or a combination of those. And then to browse around the linked pieces of carbon accounting data, see them develop over time, and so on. Without knowing the Brazilian PGP key, a Dutch developer may find them through a Chinese fingerprint that is validated by the Dutch government, for instance.

More likely, by chasing the parent links all the way up and then down, an almost-complete view of the data can be found and all the elements can be securely aligned. Peer links and specials from each part may add information to finish off the data set. And all this data can be validated, thanks to the PGP signing system which can work in both directions, up and down but also left and right; it is not strictly hierarchical.

The publication mechanisms described here use HTTP for pulling in data when it is needed; for instance to traverse a tree, or perhaps to dive into a historic trace of the data. In addition, there is a subscription mechanism with MQTT, where new data is instantly sent by way of a notification. This assures constantly up-to-date information, and the ability to live monitor trends and accounting problems. Especially when daily data is made available with the ESA program CO2M, it should be interesting to see the budgets align on a daily basis and be aligned with expediency.