User login
I have written before about the COSMOS study and its finding that multivitamins (and chocolate) did not improve brain or cardiovascular health. So I was surprised to read that a “new” study found that vitamins can forestall dementia and age-related cognitive decline.
Upon closer look, the new data are neither new nor convincing, at least to me.
Chocolate and multivitamins for CVD and cancer prevention
The large randomized COSMOS trial was supposed to be the definitive study on chocolate that would establish its heart-health benefits without a doubt. Or, rather, the benefits of a cocoa bean extract in pill form given to healthy, older volunteers. The COSMOS study was negative. Chocolate, or the cocoa bean extract they used, did not reduce cardiovascular events.
And yet for all the prepublication importance attached to COSMOS, it is scarcely mentioned. Had it been positive, rest assured that Mars, the candy bar company that cofunded the research, and other interested parties would have been shouting it from the rooftops. As it is, they’re already spinning it.
Which brings us to the multivitamin component. COSMOS actually had a 2 × 2 design. In other words, there were four groups in this study: chocolate plus multivitamin, chocolate plus placebo, placebo plus multivitamin, and placebo plus placebo. This type of study design allows you to study two different interventions simultaneously, provided that they are independent and do not interact with each other. In addition to the primary cardiovascular endpoint, they also studied a cancer endpoint.
The multivitamin supplement didn’t reduce cardiovascular events either. Nor did it affect cancer outcomes. The main COSMOS study was negative and reinforced what countless other studies have proven: Taking a daily multivitamin does not reduce your risk of having a heart attack or developing cancer.
But wait, there’s more: COSMOS-Mind
But no researcher worth his salt studies just one or two endpoints in a study. The participants also underwent neurologic and memory testing. These results were reported separately in the COSMOS-Mind study.
COSMOS-Mind is often described as a separate (or “new”) study. In reality, it included the same participants from the original COSMOS trial and measured yet another primary outcome of cognitive performance on a series of tests administered by telephone. Although there is nothing inherently wrong with studying multiple outcomes in your patient population (after all, that salami isn’t going to slice itself), they cannot all be primary outcomes. Some, by necessity, must be secondary hypothesis–generating outcomes. If you test enough endpoints, multiple hypothesis testing dictates that eventually you will get a positive result simply by chance.
There was a time when the neurocognitive outcomes of COSMOS would have been reported in the same paper as the cardiovascular outcomes, but that time seems to have passed us by. Researchers live or die by the number of their publications, and there is an inherent advantage to squeezing as many publications as possible from the same dataset. Though, to be fair, the journal would probably have asked them to split up the paper as well.
In brief, the cocoa extract again fell short in COSMOS-Mind, but the multivitamin arm did better on the composite cognitive outcome. It was a fairly small difference – a 0.07-point improvement on the z-score at the 3-year mark (the z-score is the mean divided by the standard deviation). Much was also made of the fact that the improvement seemed to vary by prior history of cardiovascular disease (CVD). Those with a history of CVD had a 0.11-point improvement, whereas those without had a 0.06-point improvement. The authors couldn’t offer a definitive explanation for these findings. Any argument that multivitamins improve cardiovascular health and therefore prevent vascular dementia has to contend with the fact that the main COSMOS study didn’t show a cardiovascular benefit for vitamins. Speculation that you are treating nutritional deficiencies is exactly that: speculation.
A more salient question is: What does a 0.07-point improvement on the z-score mean clinically? This study didn’t assess whether a multivitamin supplement prevented dementia or allowed people to live independently for longer. In fairness, that would have been exceptionally difficult to do and would have required a much longer study.
Their one attempt to quantify the cognitive benefit clinically was a calculation about normal age-related decline. Test scores were 0.045 points lower for every 1-year increase in age among participants (their mean age was 73 years). So the authors contend that a 0.07-point increase, or the 0.083-point increase that they found at year 3, corresponds to 1.8 years of age-related decline forestalled. Whether this is an appropriate assumption, I leave for the reader to decide.
COSMOS-Web and replication
The results of COSMOS-Mind were seemingly bolstered by the recent publication of COSMOS-Web. Although I’ve seen this study described as having replicated the results of COSMOS-Mind, that description is a bit misleading. This was yet another ancillary COSMOS study; more than half of the 2,262 participants in COSMOS-Mind were also included in COSMOS-Web. Replicating results in the same people isn’t true replication.
The main difference between COSMOS-Mind and COSMOS-Web is that the former used a telephone interview to administer the cognitive tests and the latter used the Internet. They also had different endpoints, with COSMOS-Web looking at immediate recall rather than a global test composite.
COSMOS-Web was a positive study in that patients getting the multivitamin supplement did better on the test for immediate memory recall (remembering a list of 20 words), though they didn’t improve on tests of memory retention, executive function, or novel object recognition (basically a test where subjects have to identify matching geometric patterns and then recall them later). They were able to remember an additional 0.71 word on average, compared with 0.44 word in the placebo group. (For the record, it found no benefit for the cocoa extract).
Everybody does better on memory tests the second time around because practice makes perfect, hence the improvement in the placebo group. This benefit at 1 year did not survive to the end of follow-up at 3 years, in contrast to COSMOS-Mind, where the benefit was not apparent at 1 year and seen only at year 3. A history of cardiovascular disease didn’t seem to affect the results in COSMOS-Web as it did in COSMOS-Mind. As far as replications go, COSMOS-Web has some very non-negligible differences, compared with COSMOS-Mind. This incongruity, especially given the overlap in the patient populations is hard to reconcile. If COSMOS-Web was supposed to assuage any doubts that persisted after COSMOS-Mind, it hasn’t for me.
One of these studies is not like the others
Finally, although the COSMOS trial and all its ancillary study analyses suggest a neurocognitive benefit to multivitamin supplementation, it’s not the first study to test the matter. The Age-Related Eye Disease Study looked at vitamin C, vitamin E, beta-carotene, zinc, and copper. There was no benefit on any of the six cognitive tests administered to patients. The Women’s Health Study, the Women’s Antioxidant Cardiovascular Study and PREADViSE have all failed to show any benefit to the various vitamins and minerals they studied. A meta-analysis of 11 trials found no benefit to B vitamins in slowing cognitive aging.
The claim that COSMOS is the “first” study to test the hypothesis hinges on some careful wordplay. Prior studies tested specific vitamins, not a multivitamin. In the discussion of the paper, these other studies are critiqued for being short term. But the Physicians’ Health Study II did in fact study a multivitamin and assessed cognitive performance on average 2.5 years after randomization. It found no benefit. The authors of COSMOS-Web critiqued the 2.5-year wait to perform cognitive testing, saying it would have missed any short-term benefits. Although, given that they simultaneously praised their 3 years of follow-up, the criticism is hard to fully accept or even understand.
Whether follow-up is short or long, uses individual vitamins or a multivitamin, the results excluding COSMOS are uniformly negative.
Do enough tests in the same population, and something will rise above the noise just by chance. When you get a positive result in your research, it’s always exciting. But when a slew of studies that came before you are negative, you aren’t groundbreaking. You’re an outlier.
Dr. Labos is a cardiologist at Hôpital Notre-Dame, Montreal. He has disclosed no relevant financial relationships.
A version of this article appeared on Medscape.com.
I have written before about the COSMOS study and its finding that multivitamins (and chocolate) did not improve brain or cardiovascular health. So I was surprised to read that a “new” study found that vitamins can forestall dementia and age-related cognitive decline.
Upon closer look, the new data are neither new nor convincing, at least to me.
Chocolate and multivitamins for CVD and cancer prevention
The large randomized COSMOS trial was supposed to be the definitive study on chocolate that would establish its heart-health benefits without a doubt. Or, rather, the benefits of a cocoa bean extract in pill form given to healthy, older volunteers. The COSMOS study was negative. Chocolate, or the cocoa bean extract they used, did not reduce cardiovascular events.
And yet for all the prepublication importance attached to COSMOS, it is scarcely mentioned. Had it been positive, rest assured that Mars, the candy bar company that cofunded the research, and other interested parties would have been shouting it from the rooftops. As it is, they’re already spinning it.
Which brings us to the multivitamin component. COSMOS actually had a 2 × 2 design. In other words, there were four groups in this study: chocolate plus multivitamin, chocolate plus placebo, placebo plus multivitamin, and placebo plus placebo. This type of study design allows you to study two different interventions simultaneously, provided that they are independent and do not interact with each other. In addition to the primary cardiovascular endpoint, they also studied a cancer endpoint.
The multivitamin supplement didn’t reduce cardiovascular events either. Nor did it affect cancer outcomes. The main COSMOS study was negative and reinforced what countless other studies have proven: Taking a daily multivitamin does not reduce your risk of having a heart attack or developing cancer.
But wait, there’s more: COSMOS-Mind
But no researcher worth his salt studies just one or two endpoints in a study. The participants also underwent neurologic and memory testing. These results were reported separately in the COSMOS-Mind study.
COSMOS-Mind is often described as a separate (or “new”) study. In reality, it included the same participants from the original COSMOS trial and measured yet another primary outcome of cognitive performance on a series of tests administered by telephone. Although there is nothing inherently wrong with studying multiple outcomes in your patient population (after all, that salami isn’t going to slice itself), they cannot all be primary outcomes. Some, by necessity, must be secondary hypothesis–generating outcomes. If you test enough endpoints, multiple hypothesis testing dictates that eventually you will get a positive result simply by chance.
There was a time when the neurocognitive outcomes of COSMOS would have been reported in the same paper as the cardiovascular outcomes, but that time seems to have passed us by. Researchers live or die by the number of their publications, and there is an inherent advantage to squeezing as many publications as possible from the same dataset. Though, to be fair, the journal would probably have asked them to split up the paper as well.
In brief, the cocoa extract again fell short in COSMOS-Mind, but the multivitamin arm did better on the composite cognitive outcome. It was a fairly small difference – a 0.07-point improvement on the z-score at the 3-year mark (the z-score is the mean divided by the standard deviation). Much was also made of the fact that the improvement seemed to vary by prior history of cardiovascular disease (CVD). Those with a history of CVD had a 0.11-point improvement, whereas those without had a 0.06-point improvement. The authors couldn’t offer a definitive explanation for these findings. Any argument that multivitamins improve cardiovascular health and therefore prevent vascular dementia has to contend with the fact that the main COSMOS study didn’t show a cardiovascular benefit for vitamins. Speculation that you are treating nutritional deficiencies is exactly that: speculation.
A more salient question is: What does a 0.07-point improvement on the z-score mean clinically? This study didn’t assess whether a multivitamin supplement prevented dementia or allowed people to live independently for longer. In fairness, that would have been exceptionally difficult to do and would have required a much longer study.
Their one attempt to quantify the cognitive benefit clinically was a calculation about normal age-related decline. Test scores were 0.045 points lower for every 1-year increase in age among participants (their mean age was 73 years). So the authors contend that a 0.07-point increase, or the 0.083-point increase that they found at year 3, corresponds to 1.8 years of age-related decline forestalled. Whether this is an appropriate assumption, I leave for the reader to decide.
COSMOS-Web and replication
The results of COSMOS-Mind were seemingly bolstered by the recent publication of COSMOS-Web. Although I’ve seen this study described as having replicated the results of COSMOS-Mind, that description is a bit misleading. This was yet another ancillary COSMOS study; more than half of the 2,262 participants in COSMOS-Mind were also included in COSMOS-Web. Replicating results in the same people isn’t true replication.
The main difference between COSMOS-Mind and COSMOS-Web is that the former used a telephone interview to administer the cognitive tests and the latter used the Internet. They also had different endpoints, with COSMOS-Web looking at immediate recall rather than a global test composite.
COSMOS-Web was a positive study in that patients getting the multivitamin supplement did better on the test for immediate memory recall (remembering a list of 20 words), though they didn’t improve on tests of memory retention, executive function, or novel object recognition (basically a test where subjects have to identify matching geometric patterns and then recall them later). They were able to remember an additional 0.71 word on average, compared with 0.44 word in the placebo group. (For the record, it found no benefit for the cocoa extract).
Everybody does better on memory tests the second time around because practice makes perfect, hence the improvement in the placebo group. This benefit at 1 year did not survive to the end of follow-up at 3 years, in contrast to COSMOS-Mind, where the benefit was not apparent at 1 year and seen only at year 3. A history of cardiovascular disease didn’t seem to affect the results in COSMOS-Web as it did in COSMOS-Mind. As far as replications go, COSMOS-Web has some very non-negligible differences, compared with COSMOS-Mind. This incongruity, especially given the overlap in the patient populations is hard to reconcile. If COSMOS-Web was supposed to assuage any doubts that persisted after COSMOS-Mind, it hasn’t for me.
One of these studies is not like the others
Finally, although the COSMOS trial and all its ancillary study analyses suggest a neurocognitive benefit to multivitamin supplementation, it’s not the first study to test the matter. The Age-Related Eye Disease Study looked at vitamin C, vitamin E, beta-carotene, zinc, and copper. There was no benefit on any of the six cognitive tests administered to patients. The Women’s Health Study, the Women’s Antioxidant Cardiovascular Study and PREADViSE have all failed to show any benefit to the various vitamins and minerals they studied. A meta-analysis of 11 trials found no benefit to B vitamins in slowing cognitive aging.
The claim that COSMOS is the “first” study to test the hypothesis hinges on some careful wordplay. Prior studies tested specific vitamins, not a multivitamin. In the discussion of the paper, these other studies are critiqued for being short term. But the Physicians’ Health Study II did in fact study a multivitamin and assessed cognitive performance on average 2.5 years after randomization. It found no benefit. The authors of COSMOS-Web critiqued the 2.5-year wait to perform cognitive testing, saying it would have missed any short-term benefits. Although, given that they simultaneously praised their 3 years of follow-up, the criticism is hard to fully accept or even understand.
Whether follow-up is short or long, uses individual vitamins or a multivitamin, the results excluding COSMOS are uniformly negative.
Do enough tests in the same population, and something will rise above the noise just by chance. When you get a positive result in your research, it’s always exciting. But when a slew of studies that came before you are negative, you aren’t groundbreaking. You’re an outlier.
Dr. Labos is a cardiologist at Hôpital Notre-Dame, Montreal. He has disclosed no relevant financial relationships.
A version of this article appeared on Medscape.com.
I have written before about the COSMOS study and its finding that multivitamins (and chocolate) did not improve brain or cardiovascular health. So I was surprised to read that a “new” study found that vitamins can forestall dementia and age-related cognitive decline.
Upon closer look, the new data are neither new nor convincing, at least to me.
Chocolate and multivitamins for CVD and cancer prevention
The large randomized COSMOS trial was supposed to be the definitive study on chocolate that would establish its heart-health benefits without a doubt. Or, rather, the benefits of a cocoa bean extract in pill form given to healthy, older volunteers. The COSMOS study was negative. Chocolate, or the cocoa bean extract they used, did not reduce cardiovascular events.
And yet for all the prepublication importance attached to COSMOS, it is scarcely mentioned. Had it been positive, rest assured that Mars, the candy bar company that cofunded the research, and other interested parties would have been shouting it from the rooftops. As it is, they’re already spinning it.
Which brings us to the multivitamin component. COSMOS actually had a 2 × 2 design. In other words, there were four groups in this study: chocolate plus multivitamin, chocolate plus placebo, placebo plus multivitamin, and placebo plus placebo. This type of study design allows you to study two different interventions simultaneously, provided that they are independent and do not interact with each other. In addition to the primary cardiovascular endpoint, they also studied a cancer endpoint.
The multivitamin supplement didn’t reduce cardiovascular events either. Nor did it affect cancer outcomes. The main COSMOS study was negative and reinforced what countless other studies have proven: Taking a daily multivitamin does not reduce your risk of having a heart attack or developing cancer.
But wait, there’s more: COSMOS-Mind
But no researcher worth his salt studies just one or two endpoints in a study. The participants also underwent neurologic and memory testing. These results were reported separately in the COSMOS-Mind study.
COSMOS-Mind is often described as a separate (or “new”) study. In reality, it included the same participants from the original COSMOS trial and measured yet another primary outcome of cognitive performance on a series of tests administered by telephone. Although there is nothing inherently wrong with studying multiple outcomes in your patient population (after all, that salami isn’t going to slice itself), they cannot all be primary outcomes. Some, by necessity, must be secondary hypothesis–generating outcomes. If you test enough endpoints, multiple hypothesis testing dictates that eventually you will get a positive result simply by chance.
There was a time when the neurocognitive outcomes of COSMOS would have been reported in the same paper as the cardiovascular outcomes, but that time seems to have passed us by. Researchers live or die by the number of their publications, and there is an inherent advantage to squeezing as many publications as possible from the same dataset. Though, to be fair, the journal would probably have asked them to split up the paper as well.
In brief, the cocoa extract again fell short in COSMOS-Mind, but the multivitamin arm did better on the composite cognitive outcome. It was a fairly small difference – a 0.07-point improvement on the z-score at the 3-year mark (the z-score is the mean divided by the standard deviation). Much was also made of the fact that the improvement seemed to vary by prior history of cardiovascular disease (CVD). Those with a history of CVD had a 0.11-point improvement, whereas those without had a 0.06-point improvement. The authors couldn’t offer a definitive explanation for these findings. Any argument that multivitamins improve cardiovascular health and therefore prevent vascular dementia has to contend with the fact that the main COSMOS study didn’t show a cardiovascular benefit for vitamins. Speculation that you are treating nutritional deficiencies is exactly that: speculation.
A more salient question is: What does a 0.07-point improvement on the z-score mean clinically? This study didn’t assess whether a multivitamin supplement prevented dementia or allowed people to live independently for longer. In fairness, that would have been exceptionally difficult to do and would have required a much longer study.
Their one attempt to quantify the cognitive benefit clinically was a calculation about normal age-related decline. Test scores were 0.045 points lower for every 1-year increase in age among participants (their mean age was 73 years). So the authors contend that a 0.07-point increase, or the 0.083-point increase that they found at year 3, corresponds to 1.8 years of age-related decline forestalled. Whether this is an appropriate assumption, I leave for the reader to decide.
COSMOS-Web and replication
The results of COSMOS-Mind were seemingly bolstered by the recent publication of COSMOS-Web. Although I’ve seen this study described as having replicated the results of COSMOS-Mind, that description is a bit misleading. This was yet another ancillary COSMOS study; more than half of the 2,262 participants in COSMOS-Mind were also included in COSMOS-Web. Replicating results in the same people isn’t true replication.
The main difference between COSMOS-Mind and COSMOS-Web is that the former used a telephone interview to administer the cognitive tests and the latter used the Internet. They also had different endpoints, with COSMOS-Web looking at immediate recall rather than a global test composite.
COSMOS-Web was a positive study in that patients getting the multivitamin supplement did better on the test for immediate memory recall (remembering a list of 20 words), though they didn’t improve on tests of memory retention, executive function, or novel object recognition (basically a test where subjects have to identify matching geometric patterns and then recall them later). They were able to remember an additional 0.71 word on average, compared with 0.44 word in the placebo group. (For the record, it found no benefit for the cocoa extract).
Everybody does better on memory tests the second time around because practice makes perfect, hence the improvement in the placebo group. This benefit at 1 year did not survive to the end of follow-up at 3 years, in contrast to COSMOS-Mind, where the benefit was not apparent at 1 year and seen only at year 3. A history of cardiovascular disease didn’t seem to affect the results in COSMOS-Web as it did in COSMOS-Mind. As far as replications go, COSMOS-Web has some very non-negligible differences, compared with COSMOS-Mind. This incongruity, especially given the overlap in the patient populations is hard to reconcile. If COSMOS-Web was supposed to assuage any doubts that persisted after COSMOS-Mind, it hasn’t for me.
One of these studies is not like the others
Finally, although the COSMOS trial and all its ancillary study analyses suggest a neurocognitive benefit to multivitamin supplementation, it’s not the first study to test the matter. The Age-Related Eye Disease Study looked at vitamin C, vitamin E, beta-carotene, zinc, and copper. There was no benefit on any of the six cognitive tests administered to patients. The Women’s Health Study, the Women’s Antioxidant Cardiovascular Study and PREADViSE have all failed to show any benefit to the various vitamins and minerals they studied. A meta-analysis of 11 trials found no benefit to B vitamins in slowing cognitive aging.
The claim that COSMOS is the “first” study to test the hypothesis hinges on some careful wordplay. Prior studies tested specific vitamins, not a multivitamin. In the discussion of the paper, these other studies are critiqued for being short term. But the Physicians’ Health Study II did in fact study a multivitamin and assessed cognitive performance on average 2.5 years after randomization. It found no benefit. The authors of COSMOS-Web critiqued the 2.5-year wait to perform cognitive testing, saying it would have missed any short-term benefits. Although, given that they simultaneously praised their 3 years of follow-up, the criticism is hard to fully accept or even understand.
Whether follow-up is short or long, uses individual vitamins or a multivitamin, the results excluding COSMOS are uniformly negative.
Do enough tests in the same population, and something will rise above the noise just by chance. When you get a positive result in your research, it’s always exciting. But when a slew of studies that came before you are negative, you aren’t groundbreaking. You’re an outlier.
Dr. Labos is a cardiologist at Hôpital Notre-Dame, Montreal. He has disclosed no relevant financial relationships.
A version of this article appeared on Medscape.com.
