How ETH Library uses Apigee to make its open data holdings available to the public

Editor's note: The ETH Library aims to advance knowledge, support research, and teaching and as a trustworthy institution to make the world of yesterday, today, and tomorrow more comprehensible to users. As the central university library of ETH Zurich, the ETH Library facilitates access of its data with information from other areas of society and knowledge. In this article, Germano Giuliani explains how Apigee helps the ETH Library to make its diverse data easily discoverable, accessible, and reusable through API management in the context of Open API, opening up new value creation potentials. 

The mission of the ETH Library is to ensure the access of information to employees and students of ETH Zurich and other universities worldwide, to the public, and to companies engaged in research and development. At the ETH Library, we position ourselves as a customer-centric driving force for the generation and development of knowledge and the transfer of this knowledge in society.

Libraries are caught between changing expectations of users, technological developments, budget constraints, and rising costs for content and infrastructure. To be able to cope with those challenges, libraries must reinvent themselves. One way to do this is to embrace the paradigm of openness, which includes making their core products openly available to everyone following applicable web standards. To achieve this, new modern technology stacks must be adopted and new alliances must be formed.

The exchange of data and knowledge on the internet has become commonplace. Information is obtained online, research data is processed digitally, findings are published electronically, services are more connected than ever, and people wish to gain access to said data at any given time.

Our vision at the ETH Library is to create personal workspaces in the physical and digital environments through forward-looking information services as well as tools that are developed together with the users to meet their needs. As the largest public natural science and technology library in Switzerland and a national center for natural science and engineering information, we adapt in line with the constantly changing requirements and opportunities presented by the digital library and will continue to do so in the future as well.

Our goal is to continually make the ETH Library a networked, integrated, customer-centric, and needs-oriented library of the future.

The ETH Library owns and provides access to a large amount of diverse and valuable data (more than 30 million books, series, journals, and non-book materials as well as hundreds of terabytes of digital replica or digital-born images). This data is available in the form of descriptive metadata and very extensively also in the form of binary files such as images, PDF documents, and other file formats.

The management of such data is distributed across a wide range of proprietary applications using different technology stacks as well as diverse storage solutions. The current application landscape is rather isolated. This is due on the one hand to the different business and functional requirements of the individual applications and on the other hand to the historical development processes of the software products used.

The goal of every application is to fulfill the requirements of the business case or the service processes. For this purpose, data storage is selected in such a way that it optimally serves the functionality of the software used. Further use or reuse of the data, e.g., via export functionalities or interfaces, is often not considered by design. At best, applications have interfaces through which the data can be queried in the respective application-specific format and different data structures. The data silos on which the applications are based thus make it difficult to reuse the data across different applications or to make it available in a consistent form.

Providing a single point of access for our customers

To break down this silo view for customers, the ETH Library successfully introduced the approach of a single point of access for customers with the launch of the Knowledge Portal Discovery-Solution in 2010. This approach of a single point of access for customers has proven itself and is also a central architectural approach of the new cloud-based ETH Library @ swisscovery discovery solution which has replaced the Knowledge Portal in December 2020.

Data as a Service

The Data as a Service (DaaS) initiative is intended to enable the cross-application provision of data and digital resources from the ETH Library for further processing and subsequent use by the public. This will enable a single point of access also for developers in the Open Data community who will have easy access to the data and resources of the ETH Library via an API gateway and associated developer portal.

Within the context of the DaaS initiative, it is easier to develop new, innovative products and services with the involvement of our customers, who can become co-creators of said services.

A key feature of digital transformation is the constant availability of digital services at any given time, as well as the increasing networking of services and data with one another. Interoperability via application programming interfaces (APIs) is a key foundation of digital transformation, enabling the rise of new organizational and communication relationships and the establishment of new value networks. 

Open API as a concept stands for open interfaces, unfiltered and machine-readable electronic data which is accessible to everyone. Those APIs are offered free of charge for further unrestricted use by anyone.

Open API as a new distribution channel is making our previously invisible, little-known, or difficult-to-access data available as well as Linked Open Data networking functionalities findable, accessible and reusable for internal and external partners. With the establishment of a developer portal, specifically targeted at developers and the introduction of API management processes, a sustainable Open API ecosystem can be established. This will foster innovation, collaboration, and co-creation in the Open Data community.

Digital transformation is at the heart of a modern and future-oriented library. When looking at all the areas of activities that we want to innovate and advance digitally, we recognized that APIs are essential to our digital transformation. By adopting API management best practices, we can make our digital services more stable and accessible to customers and developers (inside and outside our organization), connect with our partner ecosystems more effectively, and quickly deliver new innovative services. 

Apigee provides APIs as products serving the needs of users

The ETH Library chose Apigee as the core platform for managing APIs throughout its application landscape. Herby, Google for Education and Google Cloud Partner Digital Schooling facilitated simple and affordable access to Apigee. Apigee acts as the technical foundation for integrating data into applications and connecting different types of data. The development and expansion of this platform enable our data resources to be made available via interfaces and thus enable the development of new applications as well as the expansion of existing customer-specific applications.

Apigee provides us with capabilities that are essential for connecting, extending, and integrating systems. By connecting systems, APIs also connect organizations to other organizations, organizations to their products, services to products, or products directly to other products. Apigee’s built-in Developer Portal and a Google Firebase hosted OpenAPI Swagger documentation for our public APIs allow a simple and frictionless onboarding process for internal and external developers. In this sense, the concept of Open Data acts as an enabler for the development of measures in the context of the Data as a Service initiative and has strategic importance for the ETH Library.

Having our API management solution in production since June 2022, we will continue to make more data available. Apigee currently provides API proxies for our Discovery API, which holds over 30 million books, image series, journals, and other materials and APIs for persons and places with linked open data enrichments from Wikidata, metagrid.ch, DNB Entityfacts, as well as beacon.findbuch by QID and GND IDs. Also ETHorama, a service providing a Google Maps based access to digitized documents from the ETH Library platforms such as E-Pics, e-rara, E-Periodica, e-manuscripta, and Research Collection is acting as a Client App to API proxies developed in Apigee.

Check out our new platform and re-use our data in your own applications! The ETH Library API Team can be contacted for questions and further information at api@library.ethz.ch.