Oops re. BL. Have corrected to the British Museum.

Re. “EAD is challenging because at heart it’s not really a “data-centric” XML format”, yeah, that is of course a good point, and one perhaps I’ve tended not to fully appreciate myself, even though I’m on the project :). I’ll have a look at those slides you mention.

I was aware when writing that section of the post, that it might sound as if I’m saying we’ve found it really hard with the Locah experience, which isn’t what I meant, so I guess I should have tried to rephrase it. The point was more that John did kind of give the impression that it’s very easy (unless I was misunderstanding him), which I think is in danger of giving the wrong impression to people. Maybe within the government sector they are finding it that easy, in which case it would be great to learn more from them, hence my questions.

I think we should be a bit careful about generalising, or at least we need to bear in mind some of the particular characteristics of the source data with which Locah is dealing when doing so.

Working with EAD is challenging because at heart it’s not really a “data-centric” XML format, for the sorts of reasons Mark Matienzo has talked about e.g. in http://www.slideshare.net/anarchivist/archives-the-semantic-web – and it is by design a fairly complex XML format that allows for a lot of structural variation.

And there’s further complexity arising from the nature of the Archives Hub collection, where data creation is decentralised, and has been carried out by multiple independent parties, using different tools and approaches, over an extended period of time, and as consequence there’s a good deal of variation in that aggregated data.

I’d argue that that is a challenge in any processing across that dataset, even staying within the world of XML/XPath etc.

So given this background, I still – despite my occasionally tearing my hair out on Twitter on finding some new pattern in the data that I hadn’t anticipated! – think the process has been relatively straightforward, really.

From my perspective, most of the effort has been in managing that complexity, rather than because of any difficulty associated with the process of generating linked data in particular. And I think the fact that we did process a “controlled” subset of the data – one where we knew the variation was limited – quite quickly reflects that.