by Mihai Lupu
We are now approaching the first anniversary, and it is time to take a look back and see all that has happened over the past year. From the onset, I can say that it is thoroughly exciting to be part of this effort and I admire all our partners for their work and commitment.
Startup Phase
On November 3rd, 2016, at the T-Centre Vienna, home to both T-Mobile and T-Systems Austria, 17 Austrian organizations got together to kick-off the most ambitious project in the area of information and data management, not only at national, but also at European level. These 17 organizations, of which 6 research centres, 3 large enterprises and 8 small and medium enterprises, committed themselves to the development of a national platform of exchanging data and digital services, with the support of the Austrian Research Promotion Agency (FFG) and the Federal Ministry for Transport, Innovation, and Technology (bmvit).
The work started in parallel across two primary directions: social and technical. By social here I’m referring to all activities that involve persons, be they physical or legal. This includes both community involvement in gathering requirements for the future data market, as well as business and legal aspects, to make sure that those requirements lead to an ecosystem that is both financially sustainable and legally defensible.
Over the first 6 months, requirements were collected from over 100 actors in the Austrian industry landscape. One of the main instruments for collecting these requirements was a series of five workshops, on five topics:
- Industry 4.0
- Active Assisted Living (AAL)
- earth observation/space
- mobility
- energy.
These efforts resulted in a comprehensive requirements documentation. We reported more about the growth of the DMA community in our blog earlier this year.
Technical Architecture
At the same time, the technology work packages started by creating an outline of their future technology development plan. The ambitious objective of a federated cloud, where any registered actor can provide or acquire data, services, and infrastructure, mean that a large number of technologies had to be assessed and their fit to purpose measured. In the end, it became clear that the main technology building block would be Docker containers, orchestrated by an OpenShift Origin Container Application Platform. Docker containers are supported by essentially all cloud platforms, and consequently partners agreed that all components, including core services such as authentication or Blockchain management will be developed around this technology platform. Talking about Blockchain – this was another area where recent years have seen the appearance of numerous alternatives. From the original Bitcoin Blockchain, to the more niche blockchains like GameCredits, DMA partners have assessed the widest range of possible options, both open source and commercial, and, based on four criteria (license type, maturity, community support, and smart contract support) have identified Ethereum as the Blockchain technology platform on which to build the future DMA Blockchain. This does not mean that DMA will be part of the global Ethereum blockchain, but rather that we will be using the open source technology behind it. In fact, it is not the intention of DMA to incentivize participants to mine coins for monetary gain, but rather to mine coins only for the purpose of performing transactions in the data market. Such transactions are omnipresent in the market, from simply logging in, to actual data and services being exchanged in the marketplace. The blockchain ensures that all of these are recorded for provenance and billing.
The fundamental operations for the future data market are already available as individual components, and are now being integrated.
Data Ingestion Pipeline
For instance, the Data Ingestion Pipeline manages how data, from a variety of possible sources, is made available via the data market. This may involve actually copying the data to one of the infrastructure providers inside the Market, or making it available from its original location and only uploading metadata. This process includes a metadata quality check, to ensure that all necessary information for the future retrieval of this data is stored, as well as recording a hash of the data in the blockchain, in order to make sure that future users of this data can verify that they obtain the original data and not a modified version thereof.
Providing services is also now being supported by the Data Market, based on an Open API v3 standard. DMA will support service developers by not only providing standards, but also a code generator to make sure that all services implement the necessary API endpoints and provide all the required metadata. Figure 2 shows how the API editor is embedded first in a webserver layer and then in a Docker container, ready to be deployed in the Market.
Metadata
Service Bundle
We have by now mentioned Metadata a couple of times and it is worth taking a moment to further describe this. As any standardization and integration effort, the Data Market Austria needs a common language in order for data and services to be found by others and to interact with each other. This is why a comprehensive metadata standard has been developed by the technical work packages, divided into a so-called Core Metadata (i.e. a set of required fields for a service or data asset to be in the Market) and Additional Metadata (i.e. non essential information). Both the Data Ingestion Pipeline and the DMA Service API mentioned above are based on this metadata layer. Additionally, this layer provides the necessary information to the DMA Brokers – the automatic or human actors who will create the links between data assets, services, and infrastructure.
Next: Call for Startups
All of these components are now being integrated behind the DMA Portal, which will be available by March 2018, when the first call for Startups and SMEs to participate in the Market will start. In the first phase, we aim to finance 5-7 projects to start by September 2018. At that time, a second call will start, for another 10-12 projects that will begin by March 2019.
These calls, and the participation of the first Startups and SMEs in the platform will mark a new, vital stage in the development of the Austrian Data Market and we look forward to the challenges that this presents.