» Control Charts as a Tool in Data Quality Improvement
Host: www-nrd.nhtsa.dot.gov
approach using control charts that can
help improve data quality ... Statistical
quality control has received much
attention from industry in the last ...
Home » Management » Quality Control
Quality Control in Large Annotation Projects Involving Multiple ...
Source: papers.ldc.upenn.edu
Topic: Quality Control
Sort Desciption: quality control measures adopted by the LDC during the creation of the TDT corpora presents techniques that were utilized to ...
Content Inside: Quality Control in Large Annotation Projects Involving Multiple Judges: The Case of the TDT Corpora Stephanie Strassel David Graff Nii Martey Christopher Cieri Linguistic Data Consortium 3615 Market Street Philadelphia PA 19104 USA {strassel graff nmartey ccieri}@ldc.upenn.edu Abstract The Linguistic Data Consortium at the University of Pennsylvania has recently been engaged in the creation of large-scale annotated corpora of broadcast news materials in support of the ongoing Topic Detection and Tracking (TDT) research project. The TDT corpora were designed to support three basic research tasks: segmentation topic detection and topic tracking in newswire television and radio sources from English and Mandarin Chinese. The most recent TDT corpus TDT3 added two tasks story link and first story detection. Annotation of the TDT corpora involved a large staff of annotators who produced millions of human judgements. As with any large corpus creation effort quality assurance and inter-annotator consistency were a major concern. This paper reports the quality control measures adopted by the LDC during the creation of the TDT corpora presents techniques that were utilized to evaluate and improve the consistency of human annotators for all annotation tasks and discusses aspects of project administration that were designed to enhance annotation consistency. 1. Introduction This paper will review the annotation effort that supported the development of the TDT2 and TDT3 corpora at the Linguistic Data Consortium with particular emphasis on the quality control measures that were employed for each annotation task. In addition the paper reviews general approaches to quality control that were adopted as part of the overall project design and project administration. 2. The TDT corpora This section provides a brief overview of the content of the Topic Detection and Tracking (TDT) corpora. For a more detailed report see Cieri et al. (2000). A discussion of the TDT resear ...
Related PDF Files:
» QUALITY CONTROL PROGRAM
Host: www.ctdol.state.ct.us
QUALITY CONTROL PROGRAM. What is it?
Quality Control is a management ...
essential areas the Quality. Control staff
must consider to ...
» Contractor Quality Control Plans Contractor Guidelines and Example ...
Host: www.cflhd.gov
A contractor quality control plan (CQCP)
is the documentation of the contractors
... requirements dealing with quality
control and quality assurance. ...