Fraunhofer IDMT was invited for an archive-technology panel at the FIAT/IFTA World Conference 2016 in Warsaw (Oct 14-15).
The event included many interesting sessions with relevance for automatic metadata extraction and MICO aspects:
On Oct 14, there were interactive sessions on next-generation MAM, which highlighted e.g. the need for automatic metadata extraction, semantic search and decentralized infrastructure, and also on cloud aspects for A/V archives, in which it became clear that security and data protection issues are key obstacles for A/V archive usage of cloud storage and processing.
Later on the same day, there were several presentations by A/V archives regarding the use of automatic metadata extraction which confirmed that major goals of automatic metadata extraction in this domain are the detection of persons, locations, subject, and segments, and that automatic extraction typically starts with the use of ASR, NER, face and speaker recognition. It also became clear that in order to support concrete archive workflows, it does not suffice to integrate technologies of the shelf, but that the key is to combine different extractors and to complement them with manual annotation to address the (often very specific) requirements of an individual workflow to be supported.
Another presentation demonstrated the use of statistical analysis to find anomalies in metadata, to speed up metadata curation and troubleshooting, and to predict effects of changing metadata structures.
On Oct 15, the event included a panel for an exchange between archive and technology representative, chaired by SRG SSR, which we were invited to. It included discussions on
- the growing importance of recommendation, the need for semantic linking and the possibilities of using MPEG-21 UDI for standardization by RAI
- an introduction and demonstration of applying deep learning for visual classification by 4Quant
- multi-modal annotation and related challenges by Fraunhofer IDMT, including an outline of the MICO project
This panel was especially interesting in that it evolved into a discussion on why such technologies, despite having great potential for A/V archives, are not yet commonly applied within the domain. Judging from this discussion, main obstacles include
- the need to address data protection issues to exploit recommendation based on collaborative filtering
- the need to combine automatic extraction with manual metadata in a reasonable way, customized to the specific needs of a use case
- the challenge that automatic extraction is considered a challenge and even a threat by some archivists, and that its benefits (and limitations, which will always require a “human in the loop”) need to be explained better
- the challenge to not to put too high expectations into extraction – for instance, even ASR extractors which are far from working “perfectly” can be used as a support tool to speed up manual annotation, or to support key word search
- the need to provide extractors with modular, service-based approaches based on OAIS
- the need to allow archivists to train extractors, based on their own data
- the challenge that long-term cooperation in this domain are required, but that startups cannot always provide that long-term perspective
- the challenge that archivists and technologies have different perspectives, and should communicate more, which helps both sides
All in all, it was a very interesting event in which many MICO-related issues came up, which has resulted in several ongoing follow-up discussions with A/V archives.