Digital Curation Appraisal
A Review of Current Research on Appraisal Strategies in Digital Curation Environments
Annotator: Nelle Butler
Introduction
The research reflected in this project was conducted to identify the contemporary state of affairs regarding appraisal in digital curation environments. The field encompassing digital curation is an expanding arena in which solutions for the access to information that gives data its relevance are needed, and so the importance of tracking new practices associated with its interpretation cannot be underestimated. Strategies associated with appraisal for digital materials, are for the most part, all relatively new, and one comment that surfaced frequently among many of the authors of the articles chosen for this project, was on the scarcity of research related to their topic. Therefore, the challenge for this project was primarily associated with locating articles on the subject in order to deliver an overview of the activity regarding it in research circles.
Annotations
Anderson, B., Eaton, F., & Schwartz, S. (2015). Archival appraisal and the digital record: Applying past tradition for future practice. New Review of Information Networking, 20(1-2), 3-15.
This article proposes a strategy for the practice of archival appraisal for born-digital records, termed macro-appraisal. Implementation of the method begins with an evaluation of top-level policies in an organization, and how they are reflected in decision-making processes, to determine how digital records should be appraised. It proceeds downward, to specific collections of documents which show evidence of the administrative functions identified in the initial step of the process. The authors argue that traditional appraisal approaches move in the opposite direction, proceeding from the bottom-up, to examine the creation of records according to their administrative function. The application of the macro-appraisal approach is surveyed in three organizations: the National Archives, the University of Illinois, and the Sousa Archives. Records of different classes are covered in each location, including email, science, and music records. The survey will be useful to digital curators who are looking for pragmatic information on new approaches to appraisal of their archives.
Baldwin, D. A. (2004). Toward an appraisal model for audio materials in regional archives & historical societies. ARSC Journal, 35(2), 179-196.
This article introduces a new model for appraisal of audio materials to be applied in context of the policies and missions of organizations. The model establishes three levels of analysis in the classification of sound materials: provenance, modality, and form. Archival appraisal is governed according to value, cost, and control categories, which are further subdivided, for a total of fourteen criteria with which to assess acquisitions. It is therefore, a template—one which furnishes a highly-defined road map for the appraisal of audio materials in a field that currently offers limited resources to guide the archivist in their appraisal decisions.
Cocciolo, A. (2016). Email as cultural heritage resource: Appraisal solutions from an art museum context. Records Management Journal, 26(1), 68-82.
The case study presented in this article examines appraisal strategies for selection of relevant email correspondence through a process which does not capture personal or inconsequential emails. The study began with a question asking what method of appraisal would work to isolate significant emails only, and in an effort to identify an effective process for this purpose, the emails of individuals at an important cultural institution in the Northeastern United States were subjected to a manual appraisal, using a rubric to ascertain their worthiness for retention. Information gleaned from the appraisal was used to locate potential strategies for facilitating the process, with one useful outcome showing that built-in mail clients, such as Microsoft Outlook, offer the best available tools for complex searching, sorting, and categorization” (p. 79). According to the authors, research on this topic is scarce, making this study a valuable resource on methods for evaluating email content for preservation purposes.
Eastwood, T. (2006). Appraising digital records for long-term preservation. Data Science Journal, 3, 202-208.
This article presents the model for an archival selection function developed for electronic records by the Appraisal Task Force of the The International Research on Permanent Authentic Records in Electronic Systems (InterPARES). The model assumes the identity of the preserver, which conducts selection, including appraisal and custody transfer of records chosen for preservation. Eastwood claims the model holds potential for preservation of digital objects beyond records, including scientific data. The appraisal methods, drawn from those traditionally used in archival environments, involve four specific actions: “compiling and analyzing information about the records and their contexts; assessing their capacity to serve the needs of their creator and society; determining the feasibility of preserving them; and, on the basis of the foregoing, making the appraisal decision" (p. 204). Discussion of these four activities forms the body of the article, and furnishes methods for curators who are looking for practical advice based on solid research for developing the means with which to launch an appraisal program for selection of digital records.
Gutmann, M., Schürer, K., Donakowski, D., & Beedham, H. (2006). The selection, appraisal, and retention of social science data. Data Science Journal, 3, 209-221.
This article reviews the methods used by two large organizations to manage vast stores of digital data archives: the Inter-university Consortium for Political and Social Research (ICPSR) in the United States, and the UK Data Archive (UKDA) in the United Kingdom. Specific guidelines are provided for appraisal, including requirements for the technical documentation needed to support social science data collections, and in a similar vein, on distinct issues associated with confidentiality. Levels of processing, which are more extensive for social science collections are covered, as is the quality and richness of metadata that will insure viability over the long term. This paper will be of interest to data curators who are looking for new methods of archiving which originate in social sciences environments, and is recommended for researchers who may find benefit in knowing of the information required in order to see their work accepted by archival institutions.
Kim, Y. and Ross, S. (2013), Closing the loop: Assisting archival appraisal and information retrieval in one sweep. Proceedings of the 76th Annual Meeting of the American Society for Information Science and Technology, November 1-6, 2013, Montreal, Quebec, Canada., 1-10.
The study discussed in this paper looks at similarities between the selection criteria used in archives settings, and the relevance criteria applied in information retrieval environments, under the premise that the relevance judgment and relevance criteria used to assign value in information areas can be applied to current practices in archival appraisal and selection for digital materials. The purpose: to develop a semi-automated selection and appraisal system. To support this assertion, Kim and Ross present three selection scenarios which display the characteristics shared between the two frames of reference, in spite of their differing end-goals, such as "scope, equality, uniqueness, usability, and resource consumption" (p. 1). Selection scenarios include archival appraisal, research data management, and relevance judgment and literature-based discovery; all are accompanied by detailed explanations, and in some cases, as with archives, include specific methods to assist the appraiser in their decision-making process. This paper informs of a multi-step approach to archives appraisal, which calls for more preparation at the outset, but appears to offer substantial opportunity for information needed for ongoing evaluation purposes.
Niu, J. (2014). Appraisal and selection for digital curation. International Journal of Digital Curation, 9(2), 65-82.
This paper provides a valuable starting point for digital curators who seek analysis of appraisal and selection theories and policies generally used in museums, archives, libraries, and also social science and science data centers. A generic framework is presented for appraisal and selection of digital materials. Aspects of the framework covered in detail involve three methods for selection of digital resources (statistical sampling, risk analysis, and appraisal), objects of appraisal (including metadata and documentation), criteria for appraisal, appraisal decisions, and importantly, implementation of these practices. The information supplied here functions as an introduction to the categories of appraisal to be considered for digital curators, and can be used to begin development of an appraisal program.
Niu, J. (2012). Appraisal and custody of electronic records: Findings from four national archives. Archival Issues, 34(2), 117-130.
The appraisal and custody methodologies commonly employed at four national archives are analyzed in this survey, in context of four principal changes proposed by Luciana Duranti, regarding electronic records. Although appraisal practices for electronic records derive from traditional theoretical perspectives, Duranti, a professor of archival science and director of InterPARES, argues that the methodologies required for electronic records are fundamentally different, despite the fact that the "values and criteria remain the same" (p. 117) in both settings. The four changes relate to the appraising of authenticity, the feasibility of preservation, the importance of early appraisal, and the monitoring of appraisal decisions. Niu's research revealed that the essential changes proposed by Duranti are in fact not unique to electronic records, and she also discovered that the characteristics particular to electronic records require new appraisal methods, which must be created. This article provides important updates on the state of appraisal practices as they continue to develop in digital environments.
Pienta, A., Akmon, D., Noble, J., Hoelter, L., & Jekielek, S. (2017). A data-driven approach to appraisal and selection at a domain data repository. International Journal of Digital Curation, 12(2), 362-375.
For this study, the search activity of users at the Inter-university Consortium for Political and Social Research (ICPSR) a large social science data repository, was surveyed in an effort to address questions related to appraisal and selection issues which impact collection development. Data repositories are required to understand the interests of user communities, to identify the information most relevant to them, to insure that finite resources available for data curation can be properly utilized. Google Analytics was used to collect the data for the study, which centered on use of the search-box function on the ICPSR website, where over 500,000 searches occur on an annual basis. Findings disclosed that most searches are keyword, as opposed to study or researcher names; the top 500 search terms accounted for over twenty percent of all searches; and that search-to-study ratios appear to have direct bearing on implications for collection development. This study provides critical Information for digital curators who seek greater control of organizational resources, including return on investment.
Vellino, A., & Alberts, I. (2016). Assisting the appraisal of e-mail records with automatic classification. Records Management Journal, 26(3), 293-313.
The objective of the research presented in this paper was to demonstrate the effective use of automated email classification for appraisal and records management purposes in organizations. The methodology involved a qualitative analysis of the appraisal practices utilized to assess the value of email records by eight records management experts, with the results used to train a number of support vector machine classifiers to reproduce the decision-making procedures for locating emails of value to the organization. Experiments using automated classification were conducted on a collection of 846 emails, and despite the contextuality associated with records, were found to exhibit a high level of accuracy. Implications for the automation of email records identification look promising for organizations, and the research affirms the use of new technologies to contend with recordkeeping duties.
