Jul 2023
Rethinking the Impact of Open Data – A summary of data.europa.eu’s report
data.europa.eu recently published the first report on the series of open data impact: “Rethinking the impact of open data – A first step towards a European impact assessment for open data“.
This report is the first in a series of four that aims to establish a standard methodology for open data impact assessments that can be used across Europe. This exercise is key because a consistent definition of the impact of open data does not exist. The lack of a robust, conceptual foundation has made it more difficult for data portals to demonstrate their value through empirical evidence. It also challenges the EU’s ability to understand and compare performance across Member States.
In this article, we highlight some of the key findings and the relevant information about Ireland’s National Open Data Portal and the local Dublin’s Open Data Portal (Dublinked).
Key findings across national open data portals
Countries assessed in this report include Cyprus, Germany, Estonia, Spain, Finland, France, Croatia, Ireland, Lithuania, Netherlands, Poland and Ukraine
Use case repository
Key findings from the analysis of national open data portals reveal that different strategies are employed by national administrations to monitor the impact of open data. The most common mechanism used is a use case repository, although the content and scope of these repositories vary significantly.
Use cases primarily provide insights into how open data can have an impact, although some repositories also present information on the long-term effects. Effective use case sections typically have the following characteristics:
- They display a wide variety of reuse applications across types, platforms and subject areas
- They solicit new submissions through forms that encourage detailed description and consistent tagging and formatting
- They categorise reuse cases appropriately and consistently
- They allow users to filter cases in accordance with these categorisations
- Where keyword-based systems are implemented to allow for more specific filtering, they reduce discrepancies and redundancies to a minimum
- Intended benefits are specified, thereby giving insight into the expected benefits
- Exemplary or highly impactful reuse cases are featured in dedicated articles that clearly explain the reason(s) for highlighting particular cases
Statistics dashboard
Most national portals publish use and user statistics on dedicated dashboards. However, the tracked metrics and types of data vary. Strong statistical dashboards typically provide:
- A curated range of up-to-date statistics
- A clear specification of the temporal range and update frequency
- An option for visitors to select a date range from within which to view statistics
- Metrics on dataset popularity by number of individual downloads
- Metrics on the thematic distribution of downloads
- Metrics on the thematic distribution of reuse cases
Impact studies
Five countries assessed conduct dedicated studies to assess the impact of open data through surveys or interviews. The conclusions of studies tend to be somewhat limited by the expertise of the respondents. In addition, soliciting responses posed a challenge for nearly all portal teams.
Ireland Open Data Portal
The Irish open data portal offers two methods to assess impact: a reuse case repository and portal statistics.
In the showcases section of the portal, you can find 24 cases that are categorised using a keyword-based tagging system. This system is illustrated in the screenshot below. The keywords used can either describe the nature of the service, such as an “android app,” or the subject matter, like “agriculture.”
Reuse case section on the Irish national open data portal
When a visitor filters the reuse cases using a keyword, they have the option to further narrow down the selection by applying additional tags to the initial search results. As it occurs in other portals, when tags are suggested by the submitter it might cause variations that could complicate the filtering process.
Something to improve is the consistency of the information provided as some use cases provide comprehensive details including purpose, development, technical specifications, etc. and links to the data used while others only offer a brief summary.
The portal also includes a statistics dashboard that tracks the portal’s usage and user statistics. It monitors changes in the number of users and datasets added from 2016 to the present. Visitors can also access information on the 10 most-viewed datasets, most-downloaded datasets, and frequently searched keywords.
Statistics section on the Irish national open data portal
Additionally, the portal lists suggested datasets that have not yet been approved for inclusion under a “Suggested datasets” section. It also tracks the number of new and resolved suggestions.
Furthermore, the portal provides monthly website statistics that can be accessed or downloaded in CSV format. These metrics include the number of visitors, number of sessions, and average session duration.
Key findings across local open data portals
The cities/areas assessed for this report include Barcelona, Berlin, Bordeaux, Dublin, Eindhoven, Florence, Helsinki region, Lisbon, Madrid, Paris, Vienna and Zaragoza.
Use case repository
When analysing local open data portals, it becomes evident that municipal administrations utilise various methods to evaluate the impact of open data. Among the assessed portals, compiling examples of reuse cases was the most common approach for impact assessment, although the number of applications listed and the level of detail provided varied significantly.
Similar to national portals, many reuse case repositories in local portals employ categorisation systems, typically fixed theme and application typologies or broader keyword-based tagging. While descriptions of reuse applications are generally provided consistently, links to source data are relatively less common at the local level.
Statistics dashboard
Compared to national portals, local portals tend to provide a narrower range of use and user statistics. Commonly included statistics are dataset popularity rankings based on downloads and visits. Additionally, several portals track the distribution of reuse cases by thematic category.
Impact studies
At the local level, there is a lack of impact studies.
Dublin Open Data Portal – Dublinked
The city of Dublin evaluates the impact of its open data portal through a statistics dashboard. This dashboard includes metrics such as the number of datasets added each month, datasets added by each local authority per month, and the compliance of datasets with formatting and metadata requirements. It also provides information on the total number of users, average number of pageviews per session, and the proportion of datasets accessed via APIs.
Statistics dashboard on the open data portal of the city of Dublin
The most relevant statistics for assessing impact are the rankings of the top 10 most-searched keywords and most-viewed datasets, which offer some insight into the interests of potential reusers.
It would be interesting to see more detailed metrics related to data categories or types of users as well as number of downloads, which would help determine the potential for reuse.
Conclusion
Measuring the impact of open data is no easy task. With its vast influence on the European economy and society, and data’s nature as an intangible asset, even defining areas of impact can be tricky. The lack of a common framework complicates matters, with different definitions and approaches used by stakeholders.
To overcome these hurdles, we must establish a shared definition of open data that encompasses its multi-layered impact. Also define impact along from the social, to the economic, to the environmental, according to established guidelines.
While quantitative data, like user statistics, provide insights into data usage, they don’t perfectly measure impact. Qualitative information on open data use cases is needed for a deeper understanding. However, connecting the quantitative with the qualitative for an overall picture will be somewhat of a challenge.
Open data research frameworks, like the ODM, emphasise explaining how impact is created through use cases, not just providing examples of reuse. Combining consistent impact assessment methods with user statistics creates a solid foundation this evaluation of open data impact.If you wish to hear more on our thoughts and plans on this topic, please do get in contact.
You might also be interested in:
data.gov.ie Customer Success Story
Derilinx provides support to the Department of Public Expenditure and Reform on the national Open Data Initiative including technical support for the national Open Data portal, data.gov.ie.
Latest posts on this area
-
Everything you need to know about High Value Datasets (HVDs) – Part 1
In January 2023, the European Commission published a list of categories of high-value datasets that public sector bodies will have to make available for re-use, free of charge, by June 2024. This arti...
-
Rethinking the Impact of Open Data – A summary of data.europa.eu’s report
data.europa.eu recently published the report 'Rethinking the impact of open data – A first step towards a European impact assessment for open data'. In this article, we highlight some of the key findi...
-
Derilinx delivered a full Data Strategy for the Competition and Consumer Protection Commission – OGP in Action!
Last year we posted about the start of a data audit for the Competition and Consumer Protection Commission in Ireland (CCPC). This was the first step of a full data strategy that has now been delivere...