Workshop: Application Profiles in Practice, 6 May 2009

This was an event in two parts: firstly, an introduction to the user testing methodology being developed by the AP Support project in collaboration with the IEMSR and the IE Demonstrator project; secondly, an iteration of the paper prototyping element of the user testing. On this occasion the audience was comprised largely of experts rather than an especially representative group of typical users – quite understandably, given the nature of the meeting. (While it is very helpful to engage repository managers in user testing, it is more difficult to involve entirely non-specialist users, so there is a need for further work in facilitating this.) The session proved to be a success in raising considerable interest in current developments in application profiles.

It was always the intention to use this particular event as a platform for consulting colleagues in the repositories community about the usefulness of the approach. In this respect, the workshop was highly successful: attendees responded positively to the intention of engaging users in order to analyse and address the strengths and weaknesses of the various application profiles, raising some insightful questions and contributing to an animated debate. Rachel Bruce of JISC commended the workshop in her speech closing the Programme Meeting on the following day.

“Working with the Repositories Community: WRAP Project” (Jenny Delasalle, Warwick University), 6 May 2009

Jenny Delasalle referred to the difficulties faced in pioneering an implementation of SWAP in an institutional repository based on EPrints 3.0. Unlike in its successor EPrints 3.1, versioning was unsupported at the time, which to a great extent hampered the SWAP effort in WRAP at Warwick. She considered that in its present form, SWAP represents too complex a metadata model for adoption by the typical IR. But since there is not necessarily a need to employ all of the SWAP metadata terms (any more than one would necessarily need to employ all of the terms in DC Simple or Qualified DC), it must be presumed that the FRBR structure and the lack of automated means to populate fields with structural metadata represent a significant part of the problem. It would be useful to get a clarification from Jenny on this.

That the feasibility of complex metadata schemas could be radically improved by the use of text mining to autopopulate metadata fields, thus requiring far less input and/or correction from the user, was raised later in the Forum in the discussion “How can text mining support repository tasks?”, convened by James Farnhill of JISC and led principally by Brian Rea of NaCTeM, University of Manchester. This would be of obvious and immediate relevance to the liklihood of SWAP being more widely implemented, whether in its present form or following the recommendations from the user testing effort.

Repositories Roadmap Session (Rachel Heery, external consultant for JISC), 7 May 2009

Rachel Heery gave a summary of her Digital Repositories Roadmap Review, revised from the original version by herself and Andy Powell in 2006.  Recommendation 11 referred to SWAP specifically, proposing a cut-down version without the FRBR entity-relationship model and a re-analysis of the sort undertaken in the current user testing programme; Recommendation 12 made an interesting reference to OAI-ORE in the context of SWAP.

Recommendation 11: Explore deployment of a cut down version of SWAP, possibly at the copy level, retaining the cataloguing rules to ensure a consistent approach to linking to full text. Evaluate whether use of SWAP is consistent with a Web architecture approach to repositories.

Recommendation 12: Explore use of OAI-ORE to enable applications to handle complex objects. Demonstrate how OAI-ORE facilitates the re-use of research outputs and research data. Clarify different roles of OAI-ORE and SWAP.

Outcomes

There was considerable discussion of SWAP on Twitter among colleagues at Eduserv, UKOLN and elsewhere on both days of the meeting, focussing on both the structure and implementation of SWAP as it was originally intended, and in response to Rachel Heery’s recommendations. The need to solve the lack of implementation of the Dublin Core Application Profiles appears to have regained significant impetus from the interest in the series of user testing events planned by UKOLN. In particular, new impetus has been given to the SWAP implementation effort, in which expectations had previously subsided. Given Rachel Heery’s review, it is clear that SWAP may need to be considered once more as an ongoing project rather than a past product that failed to gain support, and one that may need substantial revision in future iterations. It is important to keep an open mind about the nature of those revisions, which should be conditioned by the results of the ongoing user testing effort.

In collaboration with the IEMSR and IE Demonstrator projects, which also have an interest in developing and implementing application profiles in repositories, we are very interested in developing methodologies for the evaluation of the Dublin Core Application Profiles (DCAPs) funded by JISC. Of these, SWAP has the best developed online presence and content in institutional repositories, as well as a strong and developed user community focussed on developing Open Access content on the Web.

Our current work is therefore focussed on SWAP in the first instance, but we naturally intend to develop the process of practical user testing for the other DCAPs. We are of course aware that the needs of different resource types and repository communities will differ very widely. This is the reason that we are interested in practical user testing within those communities, to ensure that theoretical approaches to constructing application profiles actually fill the needs and requirements than underpin the development of such content in repositories and other related services on the Web.

In many ways, it is fair to say that SWAP is the “lowest hanging fruit” for this endeavour, but the impact of getting application profiles right for such a large and growing proportion of repository content should not be underestimated. Being largely textual resources, scholarly works are likely to be an area where significant lessons can be learnt for other resource types with more specific constraints and requirements. It is intended that we conduct user testing of several other DCAPs during the summer of 2009, if possible, following initial work on SWAP.

It is perhaps worth remembering that developments in repository technology have come a long way since SWAP was first developed, the example of which the other DCAPs have tended to follow, especially in the matter of using the FRBR structure. It is by no means certain that this is the only way, or the best way, to create relationships in so-called complex objects, which is to say sets of resources that relate to each other as versions. In particular, OAI-ORE is an exciting development that may provide an alternative, although its relevance for this purpose needs to be carefully evaluated and compared to existing approaches and technologies. It will not do to simply adopt the newest, coolest approach without a careful analysis of how the needs of users relate to the functionality that is presently available. If these do not correlate well within the software contexts currently in use in the community, the application profiles will fail accordingly.

It has become obvious that implementation of the DCAPs has been slow. In the case of SWAP, which has been around the longest, that lag has become a profound apathy towards efforts to implement the application profile widely in repositories. It is not even clear how best this should be done, as neither methods nor benefits have been convincingly demonstrated. It would be a great shame if the investment of expertise in improving the metadata vocabulary were wasted because the structure has not been successfully integrated into repositories. This mismatch must be understood and resolved if the situation is to be turned round and the expected gains of SWAP and the other DCAPs are to be realised.

There are a variety of technological approaches to application profiles that will need careful study, once the user testing brings a better understanding of how users need to relate particular types of resources together in repositories. These may include the Description Set Profile approach, traditional XML with XML Schema and OAI-ORE. But more importantly, it must be shown beyond doubt how the DCAPs fit the applications that users need them for, and which changes may be required, before software developers will have the motivation to address those needs by implementing the DCAPs in the major repository platforms.