Error Rates in Users of Automatic Face Recognition Software
Publication Date
October 14, 2015
Journal
PLOS ONE
Authors
David White, James D. Dunn, Alexandra C. Schmid & Richard I. Kemp
Volume
10
Issue
10
Pages
e0139827
DOI
https://dx.plos.org/10.1371/journal.pone.0139827
Publisher URL
http://journals.plos.org/plosone/article?id=10.1371%2Fjournal.pone.0139827
PubMed
http://www.ncbi.nlm.nih.gov/pubmed/26465631
PubMed Central
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4605725
Europe PMC
http://europepmc.org/abstract/MED/26465631
Web of Science
000363183100061
Scopus
84949035148
Mendeley
http://www.mendeley.com/research/error-rates-users-automatic-face-recognition-software-5
Events
Loading … Spinner

Mendeley | Further Information

{"title"=>"Error rates in users of automatic face recognition software", "type"=>"journal", "authors"=>[{"first_name"=>"David", "last_name"=>"White", "scopus_author_id"=>"55456557200"}, {"first_name"=>"James D.", "last_name"=>"Dunn", "scopus_author_id"=>"56988419500"}, {"first_name"=>"Alexandra C.", "last_name"=>"Schmid", "scopus_author_id"=>"55376896800"}, {"first_name"=>"Richard I.", "last_name"=>"Kemp", "scopus_author_id"=>"7201632549"}], "year"=>2015, "source"=>"PLoS ONE", "identifiers"=>{"sgr"=>"84949035148", "doi"=>"10.1371/journal.pone.0139827", "pui"=>"607111619", "pmid"=>"26465631", "scopus"=>"2-s2.0-84949035148", "issn"=>"19326203"}, "id"=>"1ba1d2f2-e2bb-3ae0-b643-6f59421eae15", "abstract"=>"© 2015 White et al.This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. In recent years, wide deployment of automatic face recognition systems has been accompanied by substantial gains in algorithm performance. However, benchmarking tests designed to evaluate these systems do not account for the errors of human operators, who are often an integral part of face recognition solutions in forensic and security settings. This causes a mismatch between evaluation tests and operational accuracy. We address this by measuring user performance in a face recognition system used to screen passport applications for identity fraud. Experiment 1 measured target detection accuracy in algorithm-generated 'candidate lists' selected from a large database of passport images. Accuracy was notably poorer than in previous studies of unfamiliar face matching: participants made over 50%errors for adult target faces, and over 60%when matching images of children. Experiment 2 then compared performance of student participants to trained passport officers-who use the system in their daily work-and found equivalent performance in these groups. Encouragingly, a group of highly trained and experienced \"facial examiners\" outperformed these groups by 20 percentage points.We conclude that human performance curtails accuracy of face recognition systems-potentially reducing benchmark estimates by 50%in operational settings. Mere practise does not attenuate these limits, but superior performance of trained examiners suggests that recruitment and selection of human operators, in combination with effective training and mentorship, can improve the operational accuracy of face recognition systems.", "link"=>"http://www.mendeley.com/research/error-rates-users-automatic-face-recognition-software-5", "reader_count"=>32, "reader_count_by_academic_status"=>{"Unspecified"=>1, "Professor > Associate Professor"=>1, "Researcher"=>3, "Student > Ph. D. Student"=>7, "Student > Master"=>5, "Student > Bachelor"=>7, "Lecturer"=>4, "Lecturer > Senior Lecturer"=>1, "Professor"=>3}, "reader_count_by_user_role"=>{"Unspecified"=>1, "Professor > Associate Professor"=>1, "Researcher"=>3, "Student > Ph. D. Student"=>7, "Student > Master"=>5, "Student > Bachelor"=>7, "Lecturer"=>4, "Lecturer > Senior Lecturer"=>1, "Professor"=>3}, "reader_count_by_subject_area"=>{"Unspecified"=>3, "Engineering"=>1, "Medicine and Dentistry"=>3, "Psychology"=>17, "Chemistry"=>1, "Social Sciences"=>1, "Computer Science"=>5, "Decision Sciences"=>1}, "reader_count_by_subdiscipline"=>{"Engineering"=>{"Engineering"=>1}, "Medicine and Dentistry"=>{"Medicine and Dentistry"=>3}, "Chemistry"=>{"Chemistry"=>1}, "Social Sciences"=>{"Social Sciences"=>1}, "Decision Sciences"=>{"Decision Sciences"=>1}, "Psychology"=>{"Psychology"=>17}, "Computer Science"=>{"Computer Science"=>5}, "Unspecified"=>{"Unspecified"=>3}}, "reader_count_by_country"=>{"New Zealand"=>1, "Japan"=>1, "United Kingdom"=>1}, "group_count"=>1}

CrossRef

Scopus | Further Information

{"@_fa"=>"true", "link"=>[{"@_fa"=>"true", "@ref"=>"self", "@href"=>"https://api.elsevier.com/content/abstract/scopus_id/84949035148"}, {"@_fa"=>"true", "@ref"=>"author-affiliation", "@href"=>"https://api.elsevier.com/content/abstract/scopus_id/84949035148?field=author,affiliation"}, {"@_fa"=>"true", "@ref"=>"scopus", "@href"=>"https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=84949035148&origin=inward"}, {"@_fa"=>"true", "@ref"=>"scopus-citedby", "@href"=>"https://www.scopus.com/inward/citedby.uri?partnerID=HzOxMe3b&scp=84949035148&origin=inward"}], "prism:url"=>"https://api.elsevier.com/content/abstract/scopus_id/84949035148", "dc:identifier"=>"SCOPUS_ID:84949035148", "eid"=>"2-s2.0-84949035148", "dc:title"=>"Error rates in users of automatic face recognition software", "dc:creator"=>"White D.", "prism:publicationName"=>"PLoS ONE", "prism:eIssn"=>"19326203", "prism:volume"=>"10", "prism:issueIdentifier"=>"10", "prism:pageRange"=>nil, "prism:coverDate"=>"2015-10-14", "prism:coverDisplayDate"=>"14 October 2015", "prism:doi"=>"10.1371/journal.pone.0139827", "citedby-count"=>"52", "affiliation"=>[{"@_fa"=>"true", "affilname"=>"University of New South Wales (UNSW) Australia", "affiliation-city"=>"Sydney", "affiliation-country"=>"Australia"}], "pubmed-id"=>"26465631", "prism:aggregationType"=>"Journal", "subtype"=>"ar", "subtypeDescription"=>"Article", "article-number"=>"e0139827", "source-id"=>"10600153309", "openaccess"=>"1", "openaccessFlag"=>true}

Facebook

  • {"url"=>"http%3A%2F%2Fjournals.plos.org%2Fplosone%2Farticle%3Fid%3D10.1371%252Fjournal.pone.0139827", "share_count"=>0, "like_count"=>0, "comment_count"=>0, "click_count"=>0, "total_count"=>0}

Twitter

Counter

  • {"month"=>"10", "year"=>"2015", "pdf_views"=>"95", "xml_views"=>"3", "html_views"=>"444"}
  • {"month"=>"11", "year"=>"2015", "pdf_views"=>"15", "xml_views"=>"1", "html_views"=>"180"}
  • {"month"=>"12", "year"=>"2015", "pdf_views"=>"12", "xml_views"=>"1", "html_views"=>"91"}
  • {"month"=>"1", "year"=>"2016", "pdf_views"=>"23", "xml_views"=>"0", "html_views"=>"142"}
  • {"month"=>"2", "year"=>"2016", "pdf_views"=>"5", "xml_views"=>"0", "html_views"=>"125"}
  • {"month"=>"3", "year"=>"2016", "pdf_views"=>"19", "xml_views"=>"0", "html_views"=>"139"}
  • {"month"=>"4", "year"=>"2016", "pdf_views"=>"13", "xml_views"=>"0", "html_views"=>"119"}
  • {"month"=>"5", "year"=>"2016", "pdf_views"=>"13", "xml_views"=>"0", "html_views"=>"104"}
  • {"month"=>"6", "year"=>"2016", "pdf_views"=>"27", "xml_views"=>"0", "html_views"=>"124"}
  • {"month"=>"7", "year"=>"2016", "pdf_views"=>"13", "xml_views"=>"0", "html_views"=>"123"}
  • {"month"=>"8", "year"=>"2016", "pdf_views"=>"14", "xml_views"=>"0", "html_views"=>"118"}
  • {"month"=>"9", "year"=>"2016", "pdf_views"=>"17", "xml_views"=>"0", "html_views"=>"138"}
  • {"month"=>"10", "year"=>"2016", "pdf_views"=>"29", "xml_views"=>"0", "html_views"=>"310"}
  • {"month"=>"11", "year"=>"2016", "pdf_views"=>"17", "xml_views"=>"0", "html_views"=>"136"}
  • {"month"=>"12", "year"=>"2016", "pdf_views"=>"4", "xml_views"=>"1", "html_views"=>"171"}
  • {"month"=>"1", "year"=>"2017", "pdf_views"=>"17", "xml_views"=>"3", "html_views"=>"232"}
  • {"month"=>"2", "year"=>"2017", "pdf_views"=>"26", "xml_views"=>"2", "html_views"=>"273"}
  • {"month"=>"3", "year"=>"2017", "pdf_views"=>"20", "xml_views"=>"0", "html_views"=>"569"}
  • {"month"=>"4", "year"=>"2017", "pdf_views"=>"15", "xml_views"=>"1", "html_views"=>"331"}
  • {"month"=>"5", "year"=>"2017", "pdf_views"=>"32", "xml_views"=>"0", "html_views"=>"246"}
  • {"month"=>"6", "year"=>"2017", "pdf_views"=>"23", "xml_views"=>"2", "html_views"=>"249"}
  • {"month"=>"7", "year"=>"2017", "pdf_views"=>"8", "xml_views"=>"1", "html_views"=>"166"}
  • {"month"=>"8", "year"=>"2017", "pdf_views"=>"20", "xml_views"=>"1", "html_views"=>"376"}
  • {"month"=>"9", "year"=>"2017", "pdf_views"=>"21", "xml_views"=>"1", "html_views"=>"294"}
  • {"month"=>"10", "year"=>"2017", "pdf_views"=>"36", "xml_views"=>"1", "html_views"=>"294"}
  • {"month"=>"11", "year"=>"2017", "pdf_views"=>"14", "xml_views"=>"0", "html_views"=>"257"}
  • {"month"=>"12", "year"=>"2017", "pdf_views"=>"18", "xml_views"=>"2", "html_views"=>"250"}
  • {"month"=>"1", "year"=>"2018", "pdf_views"=>"21", "xml_views"=>"0", "html_views"=>"132"}
  • {"month"=>"2", "year"=>"2018", "pdf_views"=>"22", "xml_views"=>"0", "html_views"=>"100"}
  • {"month"=>"3", "year"=>"2018", "pdf_views"=>"19", "xml_views"=>"1", "html_views"=>"137"}
  • {"month"=>"4", "year"=>"2018", "pdf_views"=>"20", "xml_views"=>"0", "html_views"=>"156"}
  • {"month"=>"5", "year"=>"2018", "pdf_views"=>"20", "xml_views"=>"5", "html_views"=>"138"}
  • {"month"=>"6", "year"=>"2018", "pdf_views"=>"40", "xml_views"=>"5", "html_views"=>"102"}
  • {"month"=>"7", "year"=>"2018", "pdf_views"=>"25", "xml_views"=>"6", "html_views"=>"90"}
  • {"month"=>"8", "year"=>"2018", "pdf_views"=>"35", "xml_views"=>"2", "html_views"=>"107"}
  • {"month"=>"9", "year"=>"2018", "pdf_views"=>"19", "xml_views"=>"0", "html_views"=>"111"}
  • {"month"=>"10", "year"=>"2018", "pdf_views"=>"33", "xml_views"=>"2", "html_views"=>"134"}
  • {"month"=>"11", "year"=>"2018", "pdf_views"=>"31", "xml_views"=>"1", "html_views"=>"126"}
  • {"month"=>"12", "year"=>"2018", "pdf_views"=>"26", "xml_views"=>"0", "html_views"=>"96"}
  • {"month"=>"1", "year"=>"2019", "pdf_views"=>"20", "xml_views"=>"0", "html_views"=>"81"}
  • {"month"=>"2", "year"=>"2019", "pdf_views"=>"25", "xml_views"=>"2", "html_views"=>"113"}
  • {"month"=>"3", "year"=>"2019", "pdf_views"=>"41", "xml_views"=>"0", "html_views"=>"143"}
  • {"month"=>"4", "year"=>"2019", "pdf_views"=>"26", "xml_views"=>"1", "html_views"=>"140"}
  • {"month"=>"5", "year"=>"2019", "pdf_views"=>"50", "xml_views"=>"2", "html_views"=>"119"}
  • {"month"=>"6", "year"=>"2019", "pdf_views"=>"74", "xml_views"=>"0", "html_views"=>"111"}
  • {"month"=>"7", "year"=>"2019", "pdf_views"=>"131", "xml_views"=>"2", "html_views"=>"73"}
  • {"month"=>"8", "year"=>"2019", "pdf_views"=>"93", "xml_views"=>"0", "html_views"=>"98"}
  • {"month"=>"9", "year"=>"2019", "pdf_views"=>"84", "xml_views"=>"28", "html_views"=>"110"}
  • {"month"=>"10", "year"=>"2019", "pdf_views"=>"31", "xml_views"=>"0", "html_views"=>"92"}
  • {"month"=>"11", "year"=>"2019", "pdf_views"=>"32", "xml_views"=>"0", "html_views"=>"116"}
  • {"month"=>"12", "year"=>"2019", "pdf_views"=>"19", "xml_views"=>"4", "html_views"=>"113"}
  • {"month"=>"1", "year"=>"2020", "pdf_views"=>"22", "xml_views"=>"1", "html_views"=>"176"}
  • {"month"=>"2", "year"=>"2020", "pdf_views"=>"31", "xml_views"=>"1", "html_views"=>"141"}
  • {"month"=>"3", "year"=>"2020", "pdf_views"=>"28", "xml_views"=>"0", "html_views"=>"118"}
  • {"month"=>"4", "year"=>"2020", "pdf_views"=>"63", "xml_views"=>"1", "html_views"=>"111"}
  • {"month"=>"5", "year"=>"2020", "pdf_views"=>"42", "xml_views"=>"1", "html_views"=>"95"}
  • {"month"=>"6", "year"=>"2020", "pdf_views"=>"27", "xml_views"=>"1", "html_views"=>"140"}
  • {"month"=>"7", "year"=>"2020", "pdf_views"=>"25", "xml_views"=>"0", "html_views"=>"153"}
  • {"month"=>"8", "year"=>"2020", "pdf_views"=>"16", "xml_views"=>"0", "html_views"=>"74"}
  • {"month"=>"9", "year"=>"2020", "pdf_views"=>"23", "xml_views"=>"0", "html_views"=>"90"}
  • {"month"=>"10", "year"=>"2020", "pdf_views"=>"30", "xml_views"=>"1", "html_views"=>"133"}
  • {"month"=>"11", "year"=>"2020", "pdf_views"=>"24", "xml_views"=>"0", "html_views"=>"100"}

Figshare

  • {"files"=>["https://ndownloader.figshare.com/files/2359492"], "description"=>"<p>Participants had to decide whether the target (top image) was present in the gallery below, and if so, select the matching identity (for solution to these examples, see Acknowledgements). Images are representative of the stimuli used in the matching task, but for reasons of privacy we are not able to provide examples of the passport images used in our studies.</p>", "links"=>[], "tags"=>["face recognition system", "student participants", "adult target", "passport images", "integral part", "algorithm performance", "benchmark estimates", "face recognition systems", "evaluation tests", "benchmarking tests", "target detection accuracy", "screen passport applications", "face recognition solutions", "identity fraud", "error rates", "equivalent performance", "Automatic Face Recognition Software", "Mere practise", "20 percentage points", "user performance", "security settings", "Experiment 1", "Experiment 2"], "article_id"=>1575231, "categories"=>["Uncategorised"], "users"=>["David White", "James D. Dunn", "Alexandra C. Schmid", "Richard I. Kemp"], "doi"=>["https://dx.doi.org/10.1371/journal.pone.0139827.g001"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Example_one_to_eight_face_matching_arrays_used_to_simulate_FR_8216_candidate_lists_8217_in_Experiments_1_and_2_/1575231", "title"=>"Example one-to-eight face matching arrays used to simulate FR ‘candidate lists’ in Experiments 1 and 2.", "pos_in_sequence"=>0, "defined_type"=>1, "published_date"=>"2015-10-14 04:18:27"}
  • {"files"=>["https://ndownloader.figshare.com/files/2359493"], "description"=>"<p>Performance is expressed as correct rejection (CR), hit, miss and misidentification (MISID) rates. Error bars denote standard error.</p>", "links"=>[], "tags"=>["face recognition system", "student participants", "adult target", "passport images", "integral part", "algorithm performance", "benchmark estimates", "face recognition systems", "evaluation tests", "benchmarking tests", "target detection accuracy", "screen passport applications", "face recognition solutions", "identity fraud", "error rates", "equivalent performance", "Automatic Face Recognition Software", "Mere practise", "20 percentage points", "user performance", "security settings", "Experiment 1", "Experiment 2"], "article_id"=>1575232, "categories"=>["Uncategorised"], "users"=>["David White", "James D. Dunn", "Alexandra C. Schmid", "Richard I. Kemp"], "doi"=>["https://dx.doi.org/10.1371/journal.pone.0139827.g002"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Face_matching_performance_by_age_of_target_face_in_Experiment_1_/1575232", "title"=>"Face matching performance by age of target face in Experiment 1.", "pos_in_sequence"=>0, "defined_type"=>1, "published_date"=>"2015-10-14 04:18:27"}
  • {"files"=>["https://ndownloader.figshare.com/files/2359494"], "description"=>"<p>Performance is expressed as correct rejection (CR), hit, miss and misidentification (MISID) rates. Error bars denote standard error.</p>", "links"=>[], "tags"=>["face recognition system", "student participants", "adult target", "passport images", "integral part", "algorithm performance", "benchmark estimates", "face recognition systems", "evaluation tests", "benchmarking tests", "target detection accuracy", "screen passport applications", "face recognition solutions", "identity fraud", "error rates", "equivalent performance", "Automatic Face Recognition Software", "Mere practise", "20 percentage points", "user performance", "security settings", "Experiment 1", "Experiment 2"], "article_id"=>1575233, "categories"=>["Uncategorised"], "users"=>["David White", "James D. Dunn", "Alexandra C. Schmid", "Richard I. Kemp"], "doi"=>["https://dx.doi.org/10.1371/journal.pone.0139827.g003"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Face_matching_performance_of_participant_groups_in_Experiment_2_/1575233", "title"=>"Face matching performance of participant groups in Experiment 2.", "pos_in_sequence"=>0, "defined_type"=>1, "published_date"=>"2015-10-14 04:18:27"}
  • {"files"=>["https://ndownloader.figshare.com/files/2359495"], "description"=>"<p>The graph does not differentiate facial review staff from facial examiners to protect participant anonymity.</p>", "links"=>[], "tags"=>["face recognition system", "student participants", "adult target", "passport images", "integral part", "algorithm performance", "benchmark estimates", "face recognition systems", "evaluation tests", "benchmarking tests", "target detection accuracy", "screen passport applications", "face recognition solutions", "identity fraud", "error rates", "equivalent performance", "Automatic Face Recognition Software", "Mere practise", "20 percentage points", "user performance", "security settings", "Experiment 1", "Experiment 2"], "article_id"=>1575234, "categories"=>["Uncategorised"], "users"=>["David White", "James D. Dunn", "Alexandra C. Schmid", "Richard I. Kemp"], "doi"=>["https://dx.doi.org/10.1371/journal.pone.0139827.g004"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Scatterplot_showing_non_significant_correlation_between_Passport_Officers_employment_duration_and_overall_face_matching_accuracy_/1575234", "title"=>"Scatterplot showing non-significant correlation between Passport Officers' employment duration and overall face matching accuracy.", "pos_in_sequence"=>0, "defined_type"=>1, "published_date"=>"2015-10-14 04:18:27"}
  • {"files"=>["https://ndownloader.figshare.com/files/2359496", "https://ndownloader.figshare.com/files/2359497"], "description"=>"<div><p>In recent years, wide deployment of automatic face recognition systems has been accompanied by substantial gains in algorithm performance. However, benchmarking tests designed to evaluate these systems do not account for the errors of human operators, who are often an integral part of face recognition solutions in forensic and security settings. This causes a mismatch between evaluation tests and operational accuracy. We address this by measuring user performance in a face recognition system used to screen passport applications for identity fraud. Experiment 1 measured target detection accuracy in algorithm-generated ‘candidate lists’ selected from a large database of passport images. Accuracy was notably poorer than in previous studies of unfamiliar face matching: participants made over 50% errors for adult target faces, and over 60% when matching images of children. Experiment 2 then compared performance of student participants to trained passport officers–who use the system in their daily work–and found equivalent performance in these groups. Encouragingly, a group of highly trained and experienced “facial examiners” outperformed these groups by 20 percentage points. We conclude that human performance curtails accuracy of face recognition systems–potentially reducing benchmark estimates by 50% in operational settings. Mere practise does not attenuate these limits, but superior performance of trained examiners suggests that recruitment and selection of human operators, in combination with effective training and mentorship, can improve the operational accuracy of face recognition systems.</p></div>", "links"=>[], "tags"=>["face recognition system", "student participants", "adult target", "passport images", "integral part", "algorithm performance", "benchmark estimates", "face recognition systems", "evaluation tests", "benchmarking tests", "target detection accuracy", "screen passport applications", "face recognition solutions", "identity fraud", "error rates", "equivalent performance", "Automatic Face Recognition Software", "Mere practise", "20 percentage points", "user performance", "security settings", "Experiment 1", "Experiment 2"], "article_id"=>1575235, "categories"=>["Uncategorised"], "users"=>["David White", "James D. Dunn", "Alexandra C. Schmid", "Richard I. Kemp"], "doi"=>["https://dx.doi.org/10.1371/journal.pone.0139827.s001", "https://dx.doi.org/10.1371/journal.pone.0139827.s002"], "stats"=>{"downloads"=>0, "page_views"=>0, "likes"=>0}, "figshare_url"=>"https://figshare.com/articles/_Error_Rates_in_Users_of_Automatic_Face_Recognition_Software_/1575235", "title"=>"Error Rates in Users of Automatic Face Recognition Software", "pos_in_sequence"=>0, "defined_type"=>4, "published_date"=>"2015-10-14 04:18:27"}

PMC Usage Stats | Further Information

  • {"unique-ip"=>"2", "full-text"=>"3", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"10"}
  • {"unique-ip"=>"9", "full-text"=>"8", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"2"}
  • {"unique-ip"=>"22", "full-text"=>"19", "pdf"=>"12", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"5", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"11"}
  • {"unique-ip"=>"10", "full-text"=>"10", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2015", "month"=>"12"}
  • {"unique-ip"=>"3", "full-text"=>"2", "pdf"=>"0", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"1"}
  • {"unique-ip"=>"7", "full-text"=>"6", "pdf"=>"0", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"6", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"3"}
  • {"unique-ip"=>"17", "full-text"=>"17", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"4"}
  • {"unique-ip"=>"13", "full-text"=>"20", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"5"}
  • {"unique-ip"=>"28", "full-text"=>"30", "pdf"=>"8", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"2", "year"=>"2016", "month"=>"6"}
  • {"unique-ip"=>"26", "full-text"=>"29", "pdf"=>"0", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"2", "year"=>"2016", "month"=>"7"}
  • {"unique-ip"=>"23", "full-text"=>"24", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"1", "cited-by"=>"0", "year"=>"2016", "month"=>"8"}
  • {"unique-ip"=>"32", "full-text"=>"38", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"6", "supp-data"=>"0", "cited-by"=>"1", "year"=>"2016", "month"=>"9"}
  • {"unique-ip"=>"42", "full-text"=>"53", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"2", "cited-by"=>"0", "year"=>"2016", "month"=>"10"}
  • {"unique-ip"=>"48", "full-text"=>"59", "pdf"=>"5", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"11"}
  • {"unique-ip"=>"54", "full-text"=>"59", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"4", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2016", "month"=>"12"}
  • {"unique-ip"=>"51", "full-text"=>"57", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"1"}
  • {"unique-ip"=>"54", "full-text"=>"60", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"7", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"2"}
  • {"unique-ip"=>"64", "full-text"=>"79", "pdf"=>"3", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"6", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"3"}
  • {"unique-ip"=>"58", "full-text"=>"64", "pdf"=>"12", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"0", "cited-by"=>"1", "year"=>"2017", "month"=>"4"}
  • {"unique-ip"=>"54", "full-text"=>"59", "pdf"=>"7", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"0", "cited-by"=>"1", "year"=>"2017", "month"=>"5"}
  • {"unique-ip"=>"41", "full-text"=>"47", "pdf"=>"0", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"4", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"6"}
  • {"unique-ip"=>"38", "full-text"=>"47", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"1", "cited-by"=>"0", "year"=>"2017", "month"=>"7"}
  • {"unique-ip"=>"51", "full-text"=>"54", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"8"}
  • {"unique-ip"=>"62", "full-text"=>"65", "pdf"=>"3", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"11", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2017", "month"=>"9"}
  • {"unique-ip"=>"72", "full-text"=>"80", "pdf"=>"4", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"0", "cited-by"=>"2", "year"=>"2017", "month"=>"10"}
  • {"unique-ip"=>"71", "full-text"=>"80", "pdf"=>"2", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"8", "supp-data"=>"1", "cited-by"=>"0", "year"=>"2017", "month"=>"11"}
  • {"unique-ip"=>"75", "full-text"=>"83", "pdf"=>"6", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"1", "cited-by"=>"0", "year"=>"2017", "month"=>"12"}
  • {"unique-ip"=>"52", "full-text"=>"56", "pdf"=>"1", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"8", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"1"}
  • {"unique-ip"=>"5", "full-text"=>"7", "pdf"=>"0", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"2"}
  • {"unique-ip"=>"102", "full-text"=>"110", "pdf"=>"8", "abstract"=>"0", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"3"}
  • {"unique-ip"=>"65", "full-text"=>"76", "pdf"=>"1", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"1", "year"=>"2019", "month"=>"1"}
  • {"unique-ip"=>"89", "full-text"=>"100", "pdf"=>"3", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"2", "cited-by"=>"0", "year"=>"2018", "month"=>"11"}
  • {"unique-ip"=>"77", "full-text"=>"79", "pdf"=>"6", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"9"}
  • {"unique-ip"=>"132", "full-text"=>"149", "pdf"=>"4", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"1", "cited-by"=>"1", "year"=>"2018", "month"=>"4"}
  • {"unique-ip"=>"154", "full-text"=>"199", "pdf"=>"4", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"5"}
  • {"unique-ip"=>"71", "full-text"=>"84", "pdf"=>"4", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"2", "cited-by"=>"1", "year"=>"2018", "month"=>"6"}
  • {"unique-ip"=>"59", "full-text"=>"63", "pdf"=>"5", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"2", "cited-by"=>"1", "year"=>"2018", "month"=>"7"}
  • {"unique-ip"=>"111", "full-text"=>"114", "pdf"=>"14", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"1", "cited-by"=>"1", "year"=>"2018", "month"=>"10"}
  • {"unique-ip"=>"80", "full-text"=>"83", "pdf"=>"2", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"1", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"8"}
  • {"unique-ip"=>"48", "full-text"=>"52", "pdf"=>"4", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2018", "month"=>"12"}
  • {"unique-ip"=>"48", "full-text"=>"60", "pdf"=>"1", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"2"}
  • {"unique-ip"=>"57", "full-text"=>"60", "pdf"=>"2", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"3"}
  • {"unique-ip"=>"82", "full-text"=>"95", "pdf"=>"4", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"4", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"4"}
  • {"unique-ip"=>"92", "full-text"=>"108", "pdf"=>"6", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"2", "cited-by"=>"1", "year"=>"2019", "month"=>"5"}
  • {"unique-ip"=>"57", "full-text"=>"63", "pdf"=>"4", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"1", "year"=>"2019", "month"=>"8"}
  • {"unique-ip"=>"46", "full-text"=>"54", "pdf"=>"2", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"7", "supp-data"=>"0", "cited-by"=>"2", "year"=>"2019", "month"=>"9"}
  • {"unique-ip"=>"58", "full-text"=>"63", "pdf"=>"5", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2019", "month"=>"10"}
  • {"unique-ip"=>"64", "full-text"=>"66", "pdf"=>"7", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"3", "supp-data"=>"2", "cited-by"=>"1", "year"=>"2019", "month"=>"12"}
  • {"unique-ip"=>"124", "full-text"=>"151", "pdf"=>"19", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"12", "supp-data"=>"1", "cited-by"=>"3", "year"=>"2020", "month"=>"2"}
  • {"unique-ip"=>"102", "full-text"=>"119", "pdf"=>"6", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"5", "supp-data"=>"5", "cited-by"=>"0", "year"=>"2020", "month"=>"3"}
  • {"unique-ip"=>"97", "full-text"=>"129", "pdf"=>"9", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"4", "supp-data"=>"3", "cited-by"=>"0", "year"=>"2020", "month"=>"4"}
  • {"unique-ip"=>"87", "full-text"=>"106", "pdf"=>"3", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"2", "cited-by"=>"0", "year"=>"2020", "month"=>"5"}
  • {"unique-ip"=>"84", "full-text"=>"112", "pdf"=>"9", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2020", "month"=>"6"}
  • {"unique-ip"=>"53", "full-text"=>"54", "pdf"=>"3", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"2", "supp-data"=>"0", "cited-by"=>"0", "year"=>"2020", "month"=>"7"}
  • {"unique-ip"=>"48", "full-text"=>"58", "pdf"=>"2", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"3", "cited-by"=>"1", "year"=>"2020", "month"=>"8"}
  • {"unique-ip"=>"55", "full-text"=>"61", "pdf"=>"10", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"2", "cited-by"=>"0", "year"=>"2020", "month"=>"9"}
  • {"unique-ip"=>"71", "full-text"=>"97", "pdf"=>"2", "scanned-summary"=>"0", "scanned-page-browse"=>"0", "figure"=>"0", "supp-data"=>"0", "cited-by"=>"2", "year"=>"2020", "month"=>"10"}

Relative Metric

{"start_date"=>"2015-01-01T00:00:00Z", "end_date"=>"2015-12-31T00:00:00Z", "subject_areas"=>[]}
Loading … Spinner
There are currently no alerts