Post published on Twitter/X on 26 Aug 2022 07:42

Abstract

Importance: The integrity of the grant application process is important to the success of the entire research enterprise. However, little information is available concerning the prevalence and evolution of subjective or promotional language ("hype") that has the potential to undermine objectivity in the writing and evaluation of grant applications. Objective: To assess changes over time in the use of hype in abstracts of National Institutes of Health (NIH) grant applications. Design, Setting, and Participants: This cross-sectional study assessed the prevalence of promotional adjectives in abstracts in the NIH archive from 1985 to 2020. Main Outcomes and Measures: From all abstracts in the NIH RePORTER (Research Portfolio Online Reporting Tools: Expenditures and Results) archive, adjectives were automatically extracted, and their frequencies in the most recent year (2020) were assessed relative to the start year (1985). Adjectives that shifted significantly in frequency and that carried a promotional sense (ie, hype) were retained, and patterns of change were assessed by plotting yearly frequencies (1985-2020). By grouping the adjectives based on shared semantic properties, broad meanings commonly expressed by hype were identified. Absolute change was measured as the difference in normalized frequency between 1985 and 2020. Relative change was measured as the percentage change in normalized frequency in 2020 relative to 1985, or the first year of occurrence. Results: In total, 901 717 abstracts were analyzed and 139 adjective forms were identified as hype. Among these 139 adjective forms, 130 hype adjectives increased in frequency by 7690 words per million (wpm) (mean [SD] relative increase, 1378% [3132%]), while 9 hype adjectives decreased in frequency by 686 wpm (mean [SD] relative decrease, 44% [18%]). The largest absolute increases were for the terms novel (1054 wpm), critical (555 wpm), and key (461 wpm), while the largest relative increases were for the terms sustainable (25 157%), actionable (16 114%), and scalable (13 029%). Hype most often serves to promote the significance, novelty, scale, and rigor of a project; the utility of the expected outcomes; the qualities of the investigators and research environment; and the gravity of the problem; as well as conveying the personal attitudes of the applicants. Conclusions and Relevance: Levels of hype in successful NIH grant applications have increased over time from 1985 to 2020. The findings in this study should serve to sensitize applicants, reviewers, and funding agencies to the increasing prevalence of subjective, promotional language in funding applications.