Search Engines Provide Higher Quality Information on HIV PrEP Compared to Virtual Assistants

Google yielded higher quality information on pre-exposure prophylaxis (PrEP) for HIV more frequently compared to other search engines, but all search engines performed better than virtual assistants, according to the results of a study published in JMIR Pediatrics and Parenting.

mrmohock - stock.adobe.com

Adolescents and young adults affected by HIV more likely to turn to internet resources for PrEP than physicians, according to the study authors. Investigators noted that efforts to expand information regarding HIV prevention methods and PrEP should be tailored to adolescents and young adults who may express discomfort discussing their sexual health with physicians.

The aim of the study was to determine whether the accuracy of web-based information regarding PrEP, comparing web search engines and virtual assistants, as well as readability.

In the study, clinical experts on adolescent HIV prevention developed a list of 23 prevention-related questions. Investigators used search engines, including Ask.com, Bing, Google, and Yahoo, and virtual assistants, including Amazon Alexa, Microsoft Cortana, Google Assistant, and Apple Siri.

Investigators recorded the first 3 results from search engines and the virtual assistant web references, as well as the virtual assistant verbal responses.They coded the responses using a 6-tiered scale to help assess the quality of the information provided from corresponding searches.

From all search media, 244 results were used and assigned a score. Investigators also used the Flesch-Kincaid Grade Level scale to assess the readability of the results used per search question.

Investigators found that Google and Google Assistant provided higher quality information more frequently, ranging from 3.4 to 3.7 and 2.8 to 3.3 out of 4, respectively, compared to other companies. Additionally, all search engines provided higher quality results compared to virtual assistants.

Bing ranged from 2.7 to 3.7, Yahoo ranged from 2.8 to 3.4, and Ask.com ranged from 0.5 to 1.8. Investigators noted that Ask.com provided the lowest quality information and directed users to websites that had no information about PrEP. Apple Siri ranged from 1.8 to 3.3, Microsoft Cortana ranged from 2.0 to 2.5, and Amazon Alexa ranged from 0.9 to 1.4.

Investigators noted that Amazon Alexa generally provided the lowest quality information, including irrelevant responses or a “Sorry, I don’t know that one” response.

When investigators used the readability assessment, they found that the search engines aligned with a reading level of an eighth, ninth, and tenth grade student, whereas the virtual assistant results ranged from a second grade to eighth grade level.

Investigators noted that the literacy level of both search methods correlated with the quality of information, with the search engines providing higher quality information more frequently.

Limitations of the study included the variability of results produced by the various algorithms. Investigators performed searches on a single day for all search engines and on a separate day within the same week for the virtual assistants.

Investigators suggested that further studies should be conducted on a time-based analysis in which searches are completed at different points over a year or comparing the top results to the same question over time. Further, injectable PrEP had not been approved yet, therefore, investigators did not include it in the search results.

Reference

Darien K, Lee S, Knowles K, Wood S, et al. Health information from web search engines and virtual assistants about pre-exposure prophylaxis for HIV prevention in adolescents and young adults: content analysis. JMIR Pediatr Parent. 2023;6:e41806. doi:10.2196/41806