Abbreviations
BMJ = The BMJ
cat = categorical
chars = characters
CI = confidence interval
DOAJ = Directory of Open Access Journals
ext = extended
IQR = interquartile range
JAMA = Journal of the American Medical Association
New Engl J Med = New England Journal of Medicine OAD = open access designation
PLoS Med = PLoS Medicine
PMCID = PubMed Central identifier
RCT = randomized controlled trial
RE = random effect
SD = standard deviation
SE = standard error
WoS = Web of Science
Notes
Meta-analyses: The logit transformation was used for the meta-analysis of binary variables for estimating the pooled proportions, standard errors were not back transformed. For some variables, meta-analyses were performed with and without articles published in 2020 because the relevant information was completed in the Web of Science (WoS) data base after our data export date.
Regression models: For all regression models, the journal BMJ was used as reference category. Linear mixed models with journal as fixed effect and time as random effect were performed. Linear regression models with journal and time as fixed effects were used for sensitivity analyses.
Citation analyses: For citation analyses, only articles published between 2011 and 2018 were considered to analyze articles with a citation period of at least 2 years. The number of total citations were investigated using quasi Poisson regression models. Quantile regression on the log number of total citations shifted by 1 was used for sensitivity analyses.
Number of articles
The following table gives an overview of original research articles per year and per journal after the data management of the PubMed and WoS export data.
Characteristic |
BMJ, N = 1,398 |
JAMA, N = 1,627 |
Lancet, N = 1,605 |
New Engl J Med, N = 2,153 |
PLoS Med, N = 1,313 |
Publication Year |
|
|
|
|
|
2011 |
177 |
170 |
151 |
215 |
86 |
2012 |
204 |
180 |
151 |
222 |
79 |
2013 |
148 |
175 |
153 |
222 |
79 |
2014 |
133 |
173 |
141 |
210 |
80 |
2015 |
141 |
161 |
170 |
221 |
70 |
2016 |
103 |
149 |
191 |
204 |
138 |
2017 |
114 |
170 |
166 |
204 |
157 |
2018 |
129 |
147 |
161 |
207 |
158 |
2019 |
127 |
153 |
169 |
209 |
204 |
2020 |
122 |
149 |
152 |
239 |
262 |
Absolute frequency of research articles. |
Descriptive statistics
Criteria
Continuous variables
Characteristic |
BMJ, N = 1,398 |
JAMA, N = 1,627 |
Lancet, N = 1,605 |
New Engl J Med, N = 2,153 |
PLoS Med, N = 1,313 |
Title length (chars) |
|
|
|
|
|
Mean (SD) |
130.3 (34.3) |
126.6 (39.4) |
154.6 (43.4) |
69.4 (9.0) |
140.2 (33.1) |
Median (IQR) |
127.0 (107.0, 151.0) |
127.0 (96.0, 153.0) |
153.0 (124.0, 183.0) |
71.0 (65.0, 75.0) |
138.0 (118.0, 161.0) |
Range |
46.0, 304.0 |
24.0, 279.0 |
38.0, 345.0 |
32.0, 119.0 |
58.0, 258.0 |
Title length (words) |
|
|
|
|
|
Mean (SD) |
17.9 (4.8) |
17.6 (5.6) |
21.6 (6.2) |
9.7 (1.8) |
20.1 (4.9) |
Median (IQR) |
18.0 (15.0, 21.0) |
17.0 (13.0, 21.0) |
21.0 (17.0, 25.0) |
10.0 (8.0, 11.0) |
20.0 (17.0, 23.0) |
Range |
6.0, 39.0 |
2.0, 40.0 |
4.0, 55.0 |
4.0, 17.0 |
8.0, 38.0 |
Length of article |
|
|
|
|
|
Mean (SD) |
10.5 (2.6) |
9.7 (1.9) |
11.0 (7.1) |
10.7 (1.4) |
17.8 (5.0) |
Median (IQR) |
10.0 (9.0, 12.0) |
10.0 (8.2, 11.0) |
10.0 (9.0, 11.0) |
11.0 (10.0, 12.0) |
18.0 (15.0, 20.0) |
Range |
5.0, 27.0 |
4.0, 29.0 |
5.0, 86.0 |
5.0, 18.0 |
6.0, 41.0 |
Unknown |
176 |
9 |
23 |
44 |
17 |
Number of references |
|
|
|
|
|
Mean (SD) |
40.5 (14.6) |
32.8 (8.9) |
34.3 (14.0) |
29.9 (8.7) |
46.4 (20.2) |
Median (IQR) |
38.0 (30.0, 48.0) |
33.0 (27.0, 38.0) |
31.0 (28.0, 37.0) |
30.0 (24.0, 37.0) |
43.0 (34.0, 55.0) |
Range |
11.0, 130.0 |
7.0, 89.0 |
10.0, 192.0 |
5.0, 76.0 |
12.0, 341.0 |
Unknown |
176 |
9 |
23 |
44 |
17 |
Number of authors |
|
|
|
|
|
Mean (SD) |
9.0 (7.2) |
15.0 (12.5) |
17.5 (14.1) |
20.9 (13.7) |
12.0 (10.7) |
Median (IQR) |
7.0 (5.0, 11.0) |
12.0 (7.0, 19.0) |
14.0 (9.0, 22.0) |
18.0 (12.0, 26.0) |
9.0 (6.0, 14.0) |
Range |
1.0, 101.0 |
1.0, 102.0 |
1.0, 101.0 |
1.0, 102.0 |
1.0, 101.0 |
Categorical variables
Articles published 2011-2020
Characteristic |
BMJ, N = 1,398 |
JAMA, N = 1,627 |
Lancet, N = 1,605 |
New Engl J Med, N = 2,153 |
PLoS Med, N = 1,313 |
Less than 10 words (Albert) |
|
|
|
|
|
0 |
1,370.0 (98.0%) |
1,539.0 (94.6%) |
1,587.0 (98.9%) |
1,165.0 (54.1%) |
1,311.0 (99.8%) |
1 |
28.0 (2.0%) |
88.0 (5.4%) |
18.0 (1.1%) |
988.0 (45.9%) |
2.0 (0.2%) |
At least 20 words (Albert) |
|
|
|
|
|
0 |
910.0 (65.1%) |
1,058.0 (65.0%) |
629.0 (39.2%) |
2,153.0 (100.0%) |
639.0 (48.7%) |
1 |
488.0 (34.9%) |
569.0 (35.0%) |
976.0 (60.8%) |
0.0 (0.0%) |
674.0 (51.3%) |
Absolute verbs (Goodman) |
|
|
|
|
|
0 |
1,391.0 (99.5%) |
1,627.0 (100.0%) |
1,590.0 (99.1%) |
2,125.0 (98.7%) |
1,302.0 (99.2%) |
1 |
7.0 (0.5%) |
0.0 (0.0%) |
15.0 (0.9%) |
28.0 (1.3%) |
11.0 (0.8%) |
Relative verbs (Goodman) |
|
|
|
|
|
0 |
1,362.0 (97.4%) |
1,623.0 (99.8%) |
1,582.0 (98.6%) |
2,146.0 (99.7%) |
1,290.0 (98.2%) |
1 |
36.0 (2.6%) |
4.0 (0.2%) |
23.0 (1.4%) |
7.0 (0.3%) |
23.0 (1.8%) |
Nounal verbs (Goodman) |
|
|
|
|
|
0 |
1,358.0 (97.1%) |
1,608.0 (98.8%) |
1,579.0 (98.4%) |
2,146.0 (99.7%) |
1,287.0 (98.0%) |
1 |
40.0 (2.9%) |
19.0 (1.2%) |
26.0 (1.6%) |
7.0 (0.3%) |
26.0 (2.0%) |
Active verbs (Goodman) |
|
|
|
|
|
0 |
1,315.0 (94.1%) |
1,604.0 (98.6%) |
1,542.0 (96.1%) |
2,111.0 (98.0%) |
1,254.0 (95.5%) |
1 |
83.0 (5.9%) |
23.0 (1.4%) |
63.0 (3.9%) |
42.0 (2.0%) |
59.0 (4.5%) |
Corporate authors |
|
|
|
|
|
0 |
1,251.0 (89.5%) |
1,209.0 (74.3%) |
995.0 (62.0%) |
1,199.0 (55.7%) |
1,170.0 (89.1%) |
1 |
147.0 (10.5%) |
418.0 (25.7%) |
610.0 (38.0%) |
954.0 (44.3%) |
143.0 (10.9%) |
RCT (title) |
|
|
|
|
|
0 |
1,104.0 (79.0%) |
939.0 (57.7%) |
630.0 (39.3%) |
2,038.0 (94.7%) |
1,066.0 (81.2%) |
1 |
294.0 (21.0%) |
688.0 (42.3%) |
975.0 (60.7%) |
115.0 (5.3%) |
247.0 (18.8%) |
RCT (abstract) |
|
|
|
|
|
0 |
848.0 (69.4%) |
803.0 (49.6%) |
543.0 (34.3%) |
773.0 (36.7%) |
867.0 (66.9%) |
1 |
374.0 (30.6%) |
815.0 (50.4%) |
1,039.0 (65.7%) |
1,336.0 (63.3%) |
429.0 (33.1%) |
Unknown |
176 |
9 |
23 |
44 |
17 |
RCT (title or abstract) |
|
|
|
|
|
0 |
1,001.0 (71.6%) |
804.0 (49.4%) |
544.0 (33.9%) |
813.0 (37.8%) |
879.0 (66.9%) |
1 |
397.0 (28.4%) |
823.0 (50.6%) |
1,061.0 (66.1%) |
1,340.0 (62.2%) |
434.0 (33.1%) |
Open access (PubMed) |
|
|
|
|
|
0 |
76.0 (5.4%) |
603.0 (37.1%) |
1,047.0 (65.2%) |
1,297.0 (60.2%) |
6.0 (0.5%) |
1 |
1,322.0 (94.6%) |
1,024.0 (62.9%) |
558.0 (34.8%) |
856.0 (39.8%) |
1,307.0 (99.5%) |
Open access (WoS binary) |
|
|
|
|
|
0 |
185.0 (13.2%) |
112.0 (6.9%) |
883.0 (55.0%) |
520.0 (24.2%) |
17.0 (1.3%) |
1 |
1,213.0 (86.8%) |
1,515.0 (93.1%) |
722.0 (45.0%) |
1,633.0 (75.8%) |
1,296.0 (98.7%) |
Open access (WoS & PubMed cat.) |
|
|
|
|
|
Bronze or Green Published |
192.0 (13.8%) |
1,523.0 (96.8%) |
218.0 (29.9%) |
1,292.0 (77.7%) |
12.0 (0.9%) |
Gold |
1,197.0 (86.2%) |
4.0 (0.3%) |
296.0 (40.6%) |
1.0 (0.1%) |
1,296.0 (99.1%) |
Green Accepted |
0.0 (0.0%) |
47.0 (3.0%) |
215.0 (29.5%) |
369.0 (22.2%) |
0.0 (0.0%) |
Unknown |
9 |
53 |
876 |
491 |
5 |
Open access (WoS & PubMed cat. ext.) |
|
|
|
|
|
Bronze or Green Published |
192.0 (13.8%) |
1,523.0 (96.8%) |
218.0 (29.9%) |
1,292.0 (77.7%) |
12.0 (0.9%) |
DOAJ Gold |
0.0 (0.0%) |
0.0 (0.0%) |
0.0 (0.0%) |
0.0 (0.0%) |
1,296.0 (99.1%) |
Green Accepted |
0.0 (0.0%) |
47.0 (3.0%) |
215.0 (29.5%) |
369.0 (22.2%) |
0.0 (0.0%) |
Other Gold |
1,197.0 (86.2%) |
4.0 (0.3%) |
296.0 (40.6%) |
1.0 (0.1%) |
0.0 (0.0%) |
Unknown |
9 |
53 |
876 |
491 |
5 |
Open access (PubMed & WoS) |
|
|
|
|
|
0 |
9.0 (0.6%) |
53.0 (3.3%) |
876.0 (54.6%) |
491.0 (22.8%) |
5.0 (0.4%) |
1 |
1,389.0 (99.4%) |
1,574.0 (96.7%) |
729.0 (45.4%) |
1,662.0 (77.2%) |
1,308.0 (99.6%) |
Country |
|
|
|
|
|
0 |
1,226.0 (87.7%) |
1,571.0 (96.6%) |
1,418.0 (88.3%) |
2,093.0 (97.2%) |
914.0 (69.6%) |
1 |
172.0 (12.3%) |
56.0 (3.4%) |
187.0 (11.7%) |
60.0 (2.8%) |
399.0 (30.4%) |
City |
|
|
|
|
|
0 |
1,369.0 (97.9%) |
1,611.0 (99.0%) |
1,582.0 (98.6%) |
2,142.0 (99.5%) |
1,258.0 (95.8%) |
1 |
29.0 (2.1%) |
16.0 (1.0%) |
23.0 (1.4%) |
11.0 (0.5%) |
55.0 (4.2%) |
Geography |
|
|
|
|
|
0 |
1,207.0 (86.3%) |
1,563.0 (96.1%) |
1,410.0 (87.9%) |
2,087.0 (96.9%) |
894.0 (68.1%) |
1 |
191.0 (13.7%) |
64.0 (3.9%) |
195.0 (12.1%) |
66.0 (3.1%) |
419.0 (31.9%) |
Colon |
|
|
|
|
|
0 |
14.0 (1.0%) |
923.0 (56.7%) |
25.0 (1.6%) |
2,153.0 (100.0%) |
37.0 (2.8%) |
1 |
1,384.0 (99.0%) |
704.0 (43.3%) |
1,580.0 (98.4%) |
0.0 (0.0%) |
1,276.0 (97.2%) |
Articles published 2011-2019
Characteristic |
BMJ, N = 1,276 |
JAMA, N = 1,478 |
Lancet, N = 1,453 |
New Engl J Med, N = 1,914 |
PLoS Med, N = 1,051 |
Corporate authors |
|
|
|
|
|
0 |
1,141.0 (89.4%) |
1,104.0 (74.7%) |
883.0 (60.8%) |
1,047.0 (54.7%) |
930.0 (88.5%) |
1 |
135.0 (10.6%) |
374.0 (25.3%) |
570.0 (39.2%) |
867.0 (45.3%) |
121.0 (11.5%) |
Open access (PMCID) |
|
|
|
|
|
0 |
73.0 (5.7%) |
597.0 (40.4%) |
968.0 (66.6%) |
1,142.0 (59.7%) |
0.0 (0.0%) |
1 |
1,203.0 (94.3%) |
881.0 (59.6%) |
485.0 (33.4%) |
772.0 (40.3%) |
1,051.0 (100.0%) |
Open access (WoS binary) |
|
|
|
|
|
0 |
169.0 (13.2%) |
52.0 (3.5%) |
813.0 (56.0%) |
377.0 (19.7%) |
7.0 (0.7%) |
1 |
1,107.0 (86.8%) |
1,426.0 (96.5%) |
640.0 (44.0%) |
1,537.0 (80.3%) |
1,044.0 (99.3%) |
Open access (WoS & PubMed cat.) |
|
|
|
|
|
Bronze or Green Published |
176.0 (13.9%) |
1,380.0 (96.5%) |
177.0 (27.6%) |
1,178.0 (76.6%) |
7.0 (0.7%) |
Gold |
1,092.0 (86.1%) |
4.0 (0.3%) |
254.0 (39.6%) |
1.0 (0.1%) |
1,044.0 (99.3%) |
Green Accepted |
0.0 (0.0%) |
46.0 (3.2%) |
211.0 (32.9%) |
358.0 (23.3%) |
0.0 (0.0%) |
Unknown |
8 |
48 |
811 |
377 |
0 |
Open access (WoS & PubMed cat. ext.) |
|
|
|
|
|
Bronze or Green Published |
176.0 (13.9%) |
1,380.0 (96.5%) |
177.0 (27.6%) |
1,178.0 (76.6%) |
7.0 (0.7%) |
DOAJ Gold |
0.0 (0.0%) |
0.0 (0.0%) |
0.0 (0.0%) |
0.0 (0.0%) |
1,044.0 (99.3%) |
Green Accepted |
0.0 (0.0%) |
46.0 (3.2%) |
211.0 (32.9%) |
358.0 (23.3%) |
0.0 (0.0%) |
Other Gold |
1,092.0 (86.1%) |
4.0 (0.3%) |
254.0 (39.6%) |
1.0 (0.1%) |
0.0 (0.0%) |
Unknown |
8 |
48 |
811 |
377 |
0 |
Open access (PubMed & WoS) |
|
|
|
|
|
0 |
8.0 (0.6%) |
48.0 (3.2%) |
811.0 (55.8%) |
377.0 (19.7%) |
0.0 (0.0%) |
1 |
1,268.0 (99.4%) |
1,430.0 (96.8%) |
642.0 (44.2%) |
1,537.0 (80.3%) |
1,051.0 (100.0%) |
Citations
Mean (SD) of cumulative citation frequency per journal for articles published between 2011 and 2018
2011 |
3.39 (4.34) |
6.16 (11.19) |
10.42 (13.39) |
14.62 (19.55) |
2.81 (4.15) |
2012 |
8.41 (12.13) |
17.57 (27.26) |
27.35 (37.83) |
47.85 (74.73) |
7.87 (10.43) |
2013 |
14.50 (19.14) |
32.36 (56.41) |
52.07 (70.08) |
86.95 (134.55) |
13.53 (16.85) |
2014 |
22.03 (27.09) |
48.76 (87.93) |
80.17 (126.02) |
129.09 (196.60) |
19.16 (22.43) |
2015 |
28.67 (35.06) |
66.10 (119.79) |
104.15 (189.01) |
171.25 (255.42) |
25.78 (30.30) |
2016 |
36.18 (42.43) |
83.78 (155.04) |
125.68 (246.00) |
216.91 (315.17) |
28.33 (38.92) |
2017 |
42.57 (49.61) |
99.22 (183.26) |
150.61 (300.87) |
259.11 (379.61) |
30.75 (47.01) |
2018 |
47.92 (56.34) |
116.05 (207.53) |
176.62 (348.09) |
298.61 (439.59) |
33.98 (54.73) |
2019 |
59.90 (63.38) |
146.90 (236.57) |
228.60 (418.82) |
378.74 (518.58) |
45.18 (65.89) |
2020 |
71.14 (70.71) |
175.98 (263.50) |
278.69 (485.70) |
454.96 (601.07) |
56.01 (76.66) |
Distribution of citation frequency over the years
Raw distribution:
Log transformed distribution:
Hypotheses for the title length
Characters
All journals
Linear mixed models
The ANOVA revealed a p-value of < 0.001 for the variable journal.
Linear mixed model - title length in characters
Intercept |
131.01 |
127.31 |
134.70 |
< 0.001 |
JAMA |
-4.11 |
-6.44 |
-1.78 |
< 0.001 |
Lancet |
23.37 |
21.03 |
25.72 |
< 0.001 |
New Engl J Med |
-61.60 |
-63.79 |
-59.40 |
< 0.001 |
PLoS Med |
7.31 |
4.82 |
9.80 |
< 0.001 |
Multiple comparisons linear mixed model - title length in characters
JAMA - BMJ |
-4.11 |
-6.44 |
-1.78 |
0.005 |
Lancet - BMJ |
23.37 |
21.03 |
25.72 |
< 0.001 |
New Engl J Med - BMJ |
-61.60 |
-63.79 |
-59.40 |
< 0.001 |
PLoS Med - BMJ |
7.31 |
4.82 |
9.80 |
< 0.001 |
Lancet - JAMA |
27.48 |
25.23 |
29.73 |
< 0.001 |
New Engl J Med - JAMA |
-57.49 |
-59.59 |
-55.39 |
< 0.001 |
PLoS Med - JAMA |
11.42 |
9.02 |
13.81 |
< 0.001 |
New Engl J Med - Lancet |
-84.97 |
-87.08 |
-82.86 |
< 0.001 |
PLoS Med - Lancet |
-16.06 |
-18.46 |
-13.67 |
< 0.001 |
PLoS Med - New Engl J Med |
68.90 |
66.64 |
71.16 |
< 0.001 |
Linear regression
The ANOVA revealed a p-value of < 0.001 for the variable journal.
Linear regression model - title length in characters
Intercept |
123.37 |
121.39 |
125.35 |
<0.001 |
Time |
1.70 |
1.46 |
1.95 |
<0.001 |
JAMA |
-4.11 |
-6.44 |
-1.77 |
<0.001 |
Lancet |
23.38 |
21.04 |
25.72 |
<0.001 |
New Engl J Med |
-61.65 |
-63.85 |
-59.45 |
<0.001 |
PLoS Med |
7.10 |
4.61 |
9.59 |
<0.001 |
Multiple comparisons - title length in characters
JAMA - BMJ |
-4.11 |
-6.44 |
-1.77 |
0.005 |
Lancet - BMJ |
23.38 |
21.04 |
25.72 |
< 0.001 |
New Engl J Med - BMJ |
-61.65 |
-63.85 |
-59.45 |
< 0.001 |
PLoS Med - BMJ |
7.10 |
4.61 |
9.59 |
< 0.001 |
Lancet - JAMA |
27.49 |
25.24 |
29.74 |
< 0.001 |
New Engl J Med - JAMA |
-57.55 |
-59.65 |
-55.45 |
< 0.001 |
PLoS Med - JAMA |
11.21 |
8.81 |
13.60 |
< 0.001 |
New Engl J Med - Lancet |
-85.03 |
-87.14 |
-82.93 |
< 0.001 |
PLoS Med - Lancet |
-16.28 |
-18.68 |
-13.89 |
< 0.001 |
PLoS Med - New Engl J Med |
68.75 |
66.50 |
71.01 |
< 0.001 |
Journal-wise
Linear mixed model
Linear mixed model with time as RE and title length in characters as dependent variable in BMJ articles
130.29 |
128.34 |
132.23 |
< 0.001 |
Linear mixed model with time as RE and title length in characters as dependent variable in JAMA articles
127.28 |
118.59 |
135.97 |
< 0.001 |
Linear mixed model with time as RE and title length in characters as dependent variable in Lancet articles
154.27 |
149.43 |
159.11 |
< 0.001 |
Linear mixed model with time as RE and title length in characters as dependent variable in New Engl J Med articles
69.40 |
68.67 |
70.12 |
< 0.001 |
Linear mixed model with time as RE and title length in characters as dependent variable in PLoS Med articles
138.31 |
134.07 |
142.56 |
< 0.001 |
Linear regression
Linear regression model with time as covariate and title length in characters as dependent variable in BMJ articles
Intercept |
129.24 |
126.18 |
132.30 |
<0.001 |
Time |
0.26 |
-0.35 |
0.87 |
0.409 |
Linear regression model with time as covariate and title length in characters as dependent variable in JAMA articles
Intercept |
107.57 |
104.28 |
110.86 |
<0.001 |
Time |
4.40 |
3.77 |
5.03 |
<0.001 |
Linear regression model with time as covariate and title length in characters as dependent variable in Lancet articles
Intercept |
144.81 |
140.80 |
148.82 |
<0.001 |
Time |
2.13 |
1.38 |
2.87 |
<0.001 |
Linear regression model with time as covariate and title length in characters as dependent variable in New Engl J Med articles
Intercept |
68.01 |
67.31 |
68.71 |
<0.001 |
Time |
0.31 |
0.18 |
0.44 |
<0.001 |
Linear regression model with time as covariate and title length in characters as dependent variable in PLoS Med articles
Intercept |
129.61 |
125.66 |
133.55 |
<0.001 |
Time |
1.85 |
1.23 |
2.47 |
<0.001 |
Randomized controlled trials (all journals)
Title
Linear mixed model - title length in characters adjusted for RCTs (title)
Intercept |
124.89 |
121.55 |
128.24 |
< 0.001 |
RCT (title) |
28.50 |
26.85 |
30.15 |
< 0.001 |
JAMA |
-10.08 |
-12.30 |
-7.87 |
< 0.001 |
Lancet |
12.20 |
9.91 |
14.48 |
< 0.001 |
New Engl J Med |
-57.02 |
-59.09 |
-54.94 |
< 0.001 |
PLoS Med |
8.24 |
5.91 |
10.57 |
< 0.001 |
Multiple comparisons linear mixed model - title length in characters adjusted for RCTs (title)
JAMA - BMJ |
-10.08 |
-12.30 |
-7.87 |
< 0.001 |
Lancet - BMJ |
12.20 |
9.91 |
14.48 |
< 0.001 |
New Engl J Med - BMJ |
-57.02 |
-59.09 |
-54.94 |
< 0.001 |
PLoS Med - BMJ |
8.24 |
5.91 |
10.57 |
< 0.001 |
Lancet - JAMA |
22.28 |
20.15 |
24.41 |
< 0.001 |
New Engl J Med - JAMA |
-46.93 |
-48.99 |
-44.87 |
< 0.001 |
PLoS Med - JAMA |
18.32 |
16.04 |
20.60 |
< 0.001 |
New Engl J Med - Lancet |
-69.21 |
-71.39 |
-67.04 |
< 0.001 |
PLoS Med - Lancet |
-3.96 |
-6.31 |
-1.61 |
0.008 |
PLoS Med - New Engl J Med |
65.25 |
63.13 |
67.38 |
< 0.001 |
Abstract
Linear mixed model - title length in characters adjusted for RCTs (abstract)
Intercept |
126.53 |
123.03 |
130.02 |
< 0.001 |
RCT (abstract) |
18.87 |
17.42 |
20.32 |
< 0.001 |
JAMA |
-9.17 |
-11.51 |
-6.83 |
< 0.001 |
Lancet |
15.63 |
13.24 |
18.02 |
< 0.001 |
New Engl J Med |
-68.97 |
-71.22 |
-66.71 |
< 0.001 |
PLoS Med |
5.82 |
3.35 |
8.29 |
< 0.001 |
Multiple comparisons linear mixed model - title length in characters adjusted for RCTs (abstract)
JAMA - BMJ |
-9.17 |
-11.51 |
-6.83 |
< 0.001 |
Lancet - BMJ |
15.63 |
13.24 |
18.02 |
< 0.001 |
New Engl J Med - BMJ |
-68.97 |
-71.22 |
-66.71 |
< 0.001 |
PLoS Med - BMJ |
5.82 |
3.35 |
8.29 |
< 0.001 |
Lancet - JAMA |
24.80 |
22.62 |
26.98 |
< 0.001 |
New Engl J Med - JAMA |
-59.79 |
-61.83 |
-57.76 |
< 0.001 |
PLoS Med - JAMA |
15.00 |
12.67 |
17.32 |
< 0.001 |
New Engl J Med - Lancet |
-84.59 |
-86.63 |
-82.55 |
< 0.001 |
PLoS Med - Lancet |
-9.81 |
-12.17 |
-7.44 |
< 0.001 |
PLoS Med - New Engl J Med |
74.79 |
72.55 |
77.02 |
< 0.001 |
Title or abstract
Linear mixed model - title length in characters adjusted for RCTs (title or abstract)
Intercept |
125.40 |
122.01 |
128.79 |
< 0.001 |
RCT (title or abstract) |
19.44 |
18.01 |
20.87 |
< 0.001 |
JAMA |
-8.36 |
-10.62 |
-6.10 |
< 0.001 |
Lancet |
16.14 |
13.83 |
18.45 |
< 0.001 |
New Engl J Med |
-68.11 |
-70.27 |
-65.94 |
< 0.001 |
PLoS Med |
6.62 |
4.23 |
9.01 |
< 0.001 |
Multiple comparisons linear mixed model - title length in characters adjusted for RCTs (title or abstract)
JAMA - BMJ |
-8.36 |
-10.62 |
-6.10 |
< 0.001 |
Lancet - BMJ |
16.14 |
13.83 |
18.45 |
< 0.001 |
New Engl J Med - BMJ |
-68.11 |
-70.27 |
-65.94 |
< 0.001 |
PLoS Med - BMJ |
6.62 |
4.23 |
9.01 |
< 0.001 |
Lancet - JAMA |
24.51 |
22.34 |
26.67 |
< 0.001 |
New Engl J Med - JAMA |
-59.74 |
-61.76 |
-57.72 |
< 0.001 |
PLoS Med - JAMA |
14.98 |
12.67 |
17.29 |
< 0.001 |
New Engl J Med - Lancet |
-84.25 |
-86.27 |
-82.23 |
< 0.001 |
PLoS Med - Lancet |
-9.52 |
-11.87 |
-7.18 |
< 0.001 |
PLoS Med - New Engl J Med |
74.73 |
72.52 |
76.93 |
< 0.001 |
Randomized controlled trials (journal-wise)
Title
Linear mixed model with time as RE, RCT (title) as FE and title length in characters as dependent variable in BMJ articles
Intercept |
126.18 |
123.94 |
128.42 |
< 0.001 |
RCT (title) |
19.66 |
15.36 |
23.96 |
< 0.001 |
Linear mixed model with time as RE, RCT (title) as FE and title length in characters as dependent variable in JAMA articles
Intercept |
109.49 |
102.78 |
116.20 |
< 0.001 |
RCT (title) |
41.69 |
38.55 |
44.83 |
< 0.001 |
Linear mixed model with time as RE, RCT (title) as FE and title length in characters as dependent variable in Lancet articles
Intercept |
136.78 |
132.03 |
141.53 |
< 0.001 |
RCT (title) |
28.91 |
24.82 |
33.00 |
< 0.001 |
Linear mixed model with time as RE, RCT (title) as FE and title length in characters as dependent variable in New Engl J Med articles
Intercept |
69.28 |
68.56 |
70.00 |
< 0.001 |
RCT (title) |
2.19 |
0.51 |
3.88 |
0.011 |
Linear mixed model with time as RE, RCT (title) as FE and title length in characters as dependent variable in PLoS Med articles
Intercept |
134.31 |
130.23 |
138.38 |
< 0.001 |
RCT (title) |
22.27 |
17.89 |
26.66 |
< 0.001 |
Abstract
Linear mixed model with time as RE, RCT (abstract) as FE and title length in characters as dependent variable in BMJ articles
Intercept |
127.76 |
125.18 |
130.34 |
< 0.001 |
RCT (abstract) |
13.49 |
9.39 |
17.58 |
< 0.001 |
Linear mixed model with time as RE, RCT (abstract) as FE and title length in characters as dependent variable in JAMA articles
Intercept |
108.61 |
101.35 |
115.87 |
< 0.001 |
RCT (abstract) |
36.86 |
33.64 |
40.08 |
< 0.001 |
Linear mixed model with time as RE, RCT (abstract) as FE and title length in characters as dependent variable in Lancet articles
Intercept |
135.78 |
130.83 |
140.72 |
< 0.001 |
RCT (abstract) |
28.45 |
24.21 |
32.69 |
< 0.001 |
Linear mixed model with time as RE, RCT (abstract) as FE and title length in characters as dependent variable in New Engl J Med articles
Intercept |
68.11 |
67.28 |
68.93 |
< 0.001 |
RCT (abstract) |
2.04 |
1.25 |
2.84 |
< 0.001 |
Linear mixed model with time as RE, RCT (abstract) as FE and title length in characters as dependent variable in PLoS Med articles
Intercept |
134.04 |
129.75 |
138.32 |
< 0.001 |
RCT (abstract) |
13.64 |
9.92 |
17.37 |
< 0.001 |
Title or abstract
Linear mixed model with time as RE, RCT (title or abstract) as FE and title length in characters as dependent variable in BMJ articles
Intercept |
125.86 |
123.50 |
128.23 |
< 0.001 |
RCT (title or abstract) |
15.66 |
11.75 |
19.57 |
< 0.001 |
Linear mixed model with time as RE, RCT (title or abstract) as FE and title length in characters as dependent variable in JAMA articles
Intercept |
108.25 |
100.97 |
115.52 |
< 0.001 |
RCT (title or abstract) |
37.43 |
34.23 |
40.63 |
< 0.001 |
Linear mixed model with time as RE, RCT (title or abstract) as FE and title length in characters as dependent variable in Lancet articles
Intercept |
134.88 |
129.95 |
139.80 |
< 0.001 |
RCT (title or abstract) |
29.44 |
25.22 |
33.66 |
< 0.001 |
Linear mixed model with time as RE, RCT (title or abstract) as FE and title length in characters as dependent variable in New Engl J Med articles
Intercept |
68.12 |
67.30 |
68.94 |
< 0.001 |
RCT (title or abstract) |
2.04 |
1.26 |
2.82 |
< 0.001 |
Linear mixed model with time as RE, RCT (title or abstract) as FE and title length in characters as dependent variable in PLoS Med articles
Intercept |
133.74 |
129.50 |
137.99 |
< 0.001 |
RCT (title or abstract) |
14.21 |
10.52 |
17.90 |
< 0.001 |
Words
All journals
Linear mixed models
The ANOVA revealed a p-value of < 0.001 for the variable journal.
Linear mixed model - title length in words
Intercept |
18.05 |
17.53 |
18.58 |
< 0.001 |
JAMA |
-0.40 |
-0.74 |
-0.07 |
0.018 |
Lancet |
3.54 |
3.20 |
3.87 |
< 0.001 |
New Engl J Med |
-8.37 |
-8.68 |
-8.05 |
< 0.001 |
PLoS Med |
1.78 |
1.42 |
2.13 |
< 0.001 |
Multiple comparisons linear mixed model - title length in words
JAMA - BMJ |
-0.40 |
-0.74 |
-0.07 |
0.123 |
Lancet - BMJ |
3.54 |
3.20 |
3.87 |
< 0.001 |
New Engl J Med - BMJ |
-8.37 |
-8.68 |
-8.05 |
< 0.001 |
PLoS Med - BMJ |
1.78 |
1.42 |
2.13 |
< 0.001 |
Lancet - JAMA |
3.94 |
3.62 |
4.26 |
< 0.001 |
New Engl J Med - JAMA |
-7.96 |
-8.26 |
-7.66 |
< 0.001 |
PLoS Med - JAMA |
2.18 |
1.84 |
2.53 |
< 0.001 |
New Engl J Med - Lancet |
-11.90 |
-12.20 |
-11.60 |
< 0.001 |
PLoS Med - Lancet |
-1.76 |
-2.10 |
-1.42 |
< 0.001 |
PLoS Med - New Engl J Med |
10.14 |
9.82 |
10.47 |
< 0.001 |
Linear regression
The ANOVA revealed a p-value of < 0.001 for the variable journal.
Linear regression model - title length in words
Intercept |
16.95 |
16.66 |
17.23 |
<0.001 |
Time |
0.25 |
0.21 |
0.28 |
<0.001 |
JAMA |
-0.41 |
-0.74 |
-0.07 |
0.018 |
Lancet |
3.54 |
3.20 |
3.87 |
<0.001 |
N Engl J Med |
-8.37 |
-8.69 |
-8.06 |
<0.001 |
PLoS Med |
1.75 |
1.40 |
2.11 |
<0.001 |
Multiple comparisons - title length in words
JAMA - BMJ |
-0.41 |
-0.74 |
-0.07 |
0.122 |
Lancet - BMJ |
3.54 |
3.20 |
3.87 |
< 0.001 |
New Engl J Med - BMJ |
-8.37 |
-8.69 |
-8.06 |
< 0.001 |
PLoS Med - BMJ |
1.75 |
1.40 |
2.11 |
< 0.001 |
Lancet - JAMA |
3.94 |
3.62 |
4.26 |
< 0.001 |
New Engl J Med - JAMA |
-7.97 |
-8.27 |
-7.67 |
< 0.001 |
PLoS Med - JAMA |
2.16 |
1.82 |
2.50 |
< 0.001 |
New Engl J Med - Lancet |
-11.91 |
-12.21 |
-11.61 |
< 0.001 |
PLoS Med - Lancet |
-1.78 |
-2.13 |
-1.44 |
< 0.001 |
PLoS Med - New Engl J Med |
10.13 |
9.80 |
10.45 |
< 0.001 |
Journal-wise
Linear mixed model
Linear mixed model with time as RE and title length in words as dependent variable in BMJ articles
17.95 |
17.69 |
18.21 |
< 0.001 |
Linear mixed model with time as RE and title length in words as dependent variable in JAMA articles
17.71 |
16.42 |
19.00 |
< 0.001 |
Linear mixed model with time as RE and title length in words as dependent variable in Lancet articles
21.58 |
20.98 |
22.19 |
< 0.001 |
Linear mixed model with time as RE and title length in words as dependent variable in New Engl J Med articles
9.69 |
9.56 |
9.82 |
< 0.001 |
Linear mixed model with time as RE and title length in words as dependent variable in PLoS Med articles
19.84 |
19.23 |
20.44 |
< 0.001 |
Linear regression
Linear regression model with time as covariate and title length in words as dependent variable in BMJ
Intercept |
17.79 |
17.37 |
18.22 |
<0.001 |
Time |
0.04 |
-0.05 |
0.12 |
0.374 |
Linear regression model with time as covariate and title length in words as dependent variable in JAMA
Intercept |
14.77 |
14.30 |
15.23 |
<0.001 |
Time |
0.66 |
0.57 |
0.75 |
<0.001 |
Linear regression model with time as covariate and title length in words as dependent variable in Lancet
Intercept |
20.42 |
19.85 |
21.00 |
<0.001 |
Time |
0.26 |
0.15 |
0.37 |
<0.001 |
Linear regression model with time as covariate and title length in words as dependent variable in New Engl J Med
Intercept |
9.43 |
9.29 |
9.57 |
<0.001 |
Time |
0.06 |
0.03 |
0.08 |
<0.001 |
Linear regression model with time as covariate and title length in words as dependent variable in PLoS Med
Intercept |
18.50 |
17.92 |
19.08 |
<0.001 |
Time |
0.28 |
0.19 |
0.37 |
<0.001 |
Randomized controlled trials (all journals)
Title
Linear mixed model - title length in words adjusted for RCTs (title)
Intercept |
17.32 |
16.83 |
17.81 |
< 0.001 |
RCT (title) |
3.42 |
3.18 |
3.66 |
< 0.001 |
JAMA |
-1.12 |
-1.45 |
-0.80 |
< 0.001 |
Lancet |
2.20 |
1.86 |
2.53 |
< 0.001 |
New Engl J Med |
-7.82 |
-8.12 |
-7.51 |
< 0.001 |
PLoS Med |
1.89 |
1.55 |
2.23 |
< 0.001 |
Multiple comparisons linear mixed model - title length in words adjusted for RCTs (title)
JAMA - BMJ |
-1.12 |
-1.45 |
-0.80 |
< 0.001 |
Lancet - BMJ |
2.20 |
1.86 |
2.53 |
< 0.001 |
New Engl J Med - BMJ |
-7.82 |
-8.12 |
-7.51 |
< 0.001 |
PLoS Med - BMJ |
1.89 |
1.55 |
2.23 |
< 0.001 |
Lancet - JAMA |
3.32 |
3.01 |
3.63 |
< 0.001 |
New Engl J Med - JAMA |
-6.69 |
-7.00 |
-6.39 |
< 0.001 |
PLoS Med - JAMA |
3.01 |
2.68 |
3.34 |
< 0.001 |
New Engl J Med - Lancet |
-10.01 |
-10.33 |
-9.69 |
< 0.001 |
PLoS Med - Lancet |
-0.31 |
-0.65 |
0.04 |
0.401 |
PLoS Med - New Engl J Med |
9.70 |
9.39 |
10.02 |
< 0.001 |
Abstract
Linear mixed model - title length in words adjusted for RCTs (abstract)
Intercept |
17.54 |
17.03 |
18.05 |
< 0.001 |
RCT (abstract) |
2.13 |
1.92 |
2.34 |
< 0.001 |
JAMA |
-0.97 |
-1.31 |
-0.62 |
< 0.001 |
Lancet |
2.67 |
2.32 |
3.01 |
< 0.001 |
New Engl J Med |
-9.19 |
-9.52 |
-8.86 |
< 0.001 |
PLoS Med |
1.61 |
1.25 |
1.97 |
< 0.001 |
Multiple comparisons linear mixed model - title length in words adjusted for RCTs (abstract)
JAMA - BMJ |
-0.97 |
-1.31 |
-0.62 |
< 0.001 |
Lancet - BMJ |
2.67 |
2.32 |
3.01 |
< 0.001 |
New Engl J Med - BMJ |
-9.19 |
-9.52 |
-8.86 |
< 0.001 |
PLoS Med - BMJ |
1.61 |
1.25 |
1.97 |
< 0.001 |
Lancet - JAMA |
3.63 |
3.31 |
3.95 |
< 0.001 |
New Engl J Med - JAMA |
-8.23 |
-8.52 |
-7.93 |
< 0.001 |
PLoS Med - JAMA |
2.58 |
2.24 |
2.92 |
< 0.001 |
New Engl J Med - Lancet |
-11.86 |
-12.16 |
-11.56 |
< 0.001 |
PLoS Med - Lancet |
-1.06 |
-1.40 |
-0.71 |
< 0.001 |
PLoS Med - New Engl J Med |
10.80 |
10.48 |
11.13 |
< 0.001 |
Title or abstract
Linear mixed model - title length in words adjusted for RCTs (title or abstract)
Intercept |
17.42 |
16.92 |
17.91 |
< 0.001 |
RCT (title or abstract) |
2.21 |
2.00 |
2.42 |
< 0.001 |
JAMA |
-0.89 |
-1.22 |
-0.56 |
< 0.001 |
Lancet |
2.72 |
2.38 |
3.05 |
< 0.001 |
New Engl J Med |
-9.10 |
-9.42 |
-8.79 |
< 0.001 |
PLoS Med |
1.70 |
1.35 |
2.05 |
< 0.001 |
Multiple comparisons linear mixed model - title length in words adjusted for RCTs (title or abstract)
JAMA - BMJ |
-0.89 |
-1.22 |
-0.56 |
< 0.001 |
Lancet - BMJ |
2.72 |
2.38 |
3.05 |
< 0.001 |
New Engl J Med - BMJ |
-9.10 |
-9.42 |
-8.79 |
< 0.001 |
PLoS Med - BMJ |
1.70 |
1.35 |
2.05 |
< 0.001 |
Lancet - JAMA |
3.60 |
3.29 |
3.92 |
< 0.001 |
New Engl J Med - JAMA |
-8.22 |
-8.51 |
-7.92 |
< 0.001 |
PLoS Med - JAMA |
2.59 |
2.25 |
2.92 |
< 0.001 |
New Engl J Med - Lancet |
-11.82 |
-12.12 |
-11.53 |
< 0.001 |
PLoS Med - Lancet |
-1.02 |
-1.36 |
-0.67 |
< 0.001 |
PLoS Med - New Engl J Med |
10.80 |
10.48 |
11.13 |
< 0.001 |
Randomized controlled trials (journal-wise)
Title
Linear mixed model with time as RE, RCT (title) as FE and title length in words as dependent variable in BMJ articles
Intercept |
17.57 |
17.27 |
17.87 |
< 0.001 |
RCT (title) |
1.83 |
1.22 |
2.44 |
< 0.001 |
Linear mixed model with time as RE, RCT (title) as FE and title length in words as dependent variable in JAMA articles
Intercept |
15.14 |
14.13 |
16.15 |
< 0.001 |
RCT (title) |
6.03 |
5.59 |
6.46 |
< 0.001 |
Linear mixed model with time as RE, RCT (title) as FE and title length in words as dependent variable in Lancet articles
Intercept |
20.07 |
19.41 |
20.73 |
< 0.001 |
RCT (title) |
2.50 |
1.90 |
3.11 |
< 0.001 |
Linear mixed model with time as RE, RCT (title) as FE and title length in words as dependent variable in New Engl J Med articles
Intercept |
9.65 |
9.52 |
9.78 |
< 0.001 |
RCT (title) |
0.66 |
0.32 |
1.00 |
< 0.001 |
Linear mixed model with time as RE, RCT (title) as FE and title length in words as dependent variable in PLoS Med articles
Intercept |
19.33 |
18.74 |
19.92 |
< 0.001 |
RCT (title) |
2.81 |
2.16 |
3.46 |
< 0.001 |
Abstract
Linear mixed model with time as RE, RCT (abstract) as FE and title length in words as dependent variable in BMJ articles
Intercept |
17.79 |
17.46 |
18.13 |
< 0.001 |
RCT (abstract) |
1.10 |
0.53 |
1.68 |
< 0.001 |
Linear mixed model with time as RE, RCT (abstract) as FE and title length in words as dependent variable in JAMA articles
Intercept |
15.05 |
13.96 |
16.15 |
< 0.001 |
RCT (abstract) |
5.26 |
4.81 |
5.71 |
< 0.001 |
Linear mixed model with time as RE, RCT (abstract) as FE and title length in words as dependent variable in Lancet articles
Intercept |
19.96 |
19.28 |
20.64 |
< 0.001 |
RCT (abstract) |
2.50 |
1.87 |
3.12 |
< 0.001 |
Linear mixed model with time as RE, RCT (abstract) as FE and title length in words as dependent variable in New Engl J Med articles
Intercept |
9.67 |
9.50 |
9.83 |
< 0.001 |
RCT (abstract) |
0.03 |
-0.13 |
0.19 |
0.754 |
Linear mixed model with time as RE, RCT (abstract) as FE and title length in words as dependent variable in PLoS Med articles
Intercept |
19.34 |
18.72 |
19.95 |
< 0.001 |
RCT (abstract) |
1.58 |
1.03 |
2.13 |
< 0.001 |
Title or abstract
Linear mixed model with time as RE, RCT (title or abstract) as FE and title length in words as dependent variable in BMJ articles
Intercept |
17.55 |
17.23 |
17.87 |
< 0.001 |
RCT (title or abstract) |
1.40 |
0.85 |
1.95 |
< 0.001 |
Linear mixed model with time as RE, RCT (title or abstract) as FE and title length in words as dependent variable in JAMA articles
Intercept |
14.99 |
13.90 |
16.09 |
< 0.001 |
RCT (title or abstract) |
5.34 |
4.90 |
5.79 |
< 0.001 |
Linear mixed model with time as RE, RCT (title or abstract) as FE and title length in words as dependent variable in Lancet articles
Intercept |
19.85 |
19.17 |
20.53 |
< 0.001 |
RCT (title or abstract) |
2.63 |
2.01 |
3.25 |
< 0.001 |
Linear mixed model with time as RE, RCT (title or abstract) as FE and title length in words as dependent variable in New Engl J Med articles
Intercept |
9.67 |
9.51 |
9.84 |
< 0.001 |
RCT (title or abstract) |
0.02 |
-0.14 |
0.18 |
0.806 |
Linear mixed model with time as RE, RCT (title or abstract) as FE and title length in words as dependent variable in PLoS Med articles
Intercept |
19.30 |
18.69 |
19.91 |
< 0.001 |
RCT (title or abstract) |
1.66 |
1.11 |
2.21 |
< 0.001 |
Additional aims
Proportion of articles with active verbs
A1: Goodman predicted that 4.4% of all clinical papers in 2010 were going to have an active verb in the title. Our aim was to estimate the proportion of articles with an active verb for the period 2011 to 2020.
Random effect meta-analysis for 2011 - proportion of active verbs (logit transformed proportion)
Active verbs |
0.026 |
0.354 |
0.013 |
0.051 |
Random effect meta-analysis over all years and all journals - proportion of active verbs (logit transformed proportion)
Active verbs |
0.035 |
0.09 |
0.030 |
0.042 |
Number of title characters and title words
A2: One additional aim is to estimate the number of title characters and title words. The descriptive statistics for the number of title characters and title words is provided in section 4.1.1. In addition, journal-specific estimates from regression models with corresponding 95% CIs are provided in sections 6.1.2.1 and 6.2.2.1.
Proportion of free of charge articles
A3: The proportion of free of charge articles increases over time for all journals, except for PLOS, which is a free access journal.
Random effect meta-analysis year-wise (2011-2019) for all journals (except PLoS Med) - proportion of free of charge articles (logit transformed proportion)
2011 |
0.86 |
0.77 |
0.57 |
0.96 |
2012 |
0.87 |
0.91 |
0.52 |
0.97 |
2013 |
0.86 |
0.70 |
0.62 |
0.96 |
2014 |
0.88 |
0.78 |
0.61 |
0.97 |
2015 |
0.88 |
0.82 |
0.60 |
0.97 |
2016 |
0.93 |
1.11 |
0.62 |
0.99 |
2017 |
0.89 |
0.75 |
0.66 |
0.97 |
2018 |
0.95 |
0.92 |
0.76 |
0.99 |
2019 |
0.98 |
1.96 |
0.55 |
1.00 |
Random effect meta-analysis over all years (2011-2019) and all journals (except PLoS Med) - proportion of free of charge articles (logit transformed proportion)
Free of charge |
0.89 |
0.22 |
0.84 |
0.93 |
Increase of free of charge articles over time
Intercept |
0.845 |
0.818 |
0.871 |
< 0.001 |
Time |
0.014 |
0.008 |
0.019 |
0.002 |