Cristin-resultat-ID: 1971592
Sist endret: 21. mars 2022, 15:50
NVI-rapporteringsår: 2021
Resultat
Vitenskapelig artikkel
2021

Machine learning in systematic reviews: comparing automated text clustering with Lingo3G and human researcher categorization in a rapid review

Bidragsytere:
  • Ashley Elizabeth Muller
  • Heather Melanie R Ames
  • Patricia Sofia Jacobsen Jardim og
  • Christopher James Rose

Tidsskrift

Research Synthesis Methods
ISSN 1759-2879
e-ISSN 1759-2887
NVI-nivå 1

Om resultatet

Vitenskapelig artikkel
Publiseringsår: 2021
Publisert online: 2021
Volum: 13
Hefte: 2
Sider: 229 - 241
Open Access

Importkilder

Scopus-ID: 2-s2.0-85121494725

Beskrivelse Beskrivelse

Tittel

Machine learning in systematic reviews: comparing automated text clustering with Lingo3G and human researcher categorization in a rapid review

Sammendrag

Systematic reviews are resource-intensive. The machine learning tools beingdeveloped mostly focus on the study identification process, but tools to assistin analysis and categorization are also needed. One possibility is to useunsupervised automatic text clustering, in which each study is automaticallyassigned to one or more meaningful clusters. Our main aim was to assess theusefulness of an automated clustering method, Lingo3G, in categorizing stud-ies in a simplified rapid review, then compare performance (precision andrecall) of this method compared to manual categorization. We randomlyassigned all 128 studies in a review to be coded by a human researcher blindedto cluster assignment (mimicking two independent researchers) or by a humanresearcher non-blinded to cluster assignment (mimicking one researcherchecking another's work). We compared time use, precision and recall of man-ual categorization versus automated clustering. Automated clustering andmanual categorization organized studies by population and intervention/con-text. Automated clustering failed to identify two manually identified categoriesbut identified one additional category not identified by the human researcher.We estimate that automated clustering has similar precision to both blindedand non-blinded researchers (e.g., 88% vs. 89%), but higher recall (e.g., 89%vs. 84%). Manual categorization required 49% more time than automated clus-tering. Using a specific clustering algorithm, automated clustering can be help-ful with categorization of and identifying patterns across studies in simplersystematic reviews. We found that the clustering was sensitive enough togroup studies according to linguistic differences that often corresponded to themanual categories

Bidragsytere

Ashley Elizabeth Muller

  • Tilknyttet:
    Forfatter
    ved Avdeling for vurdering av tiltak ved Folkehelseinstituttet

Heather Melanie R Ames

  • Tilknyttet:
    Forfatter
    ved Avdeling for vurdering av tiltak ved Folkehelseinstituttet
  • Tilknyttet:
    Forfatter
    ved La Trobe University

Patricia Sofia Jacobsen Jardim

  • Tilknyttet:
    Forfatter
    ved Avdeling for vurdering av tiltak ved Folkehelseinstituttet

Christopher James Rose

  • Tilknyttet:
    Forfatter
    ved Helsetjenester ved Folkehelseinstituttet
1 - 4 av 4