What kinds of disclosure statements would be meaningful? This post describes some disclosure statements that would really tell you something about the authors, and explains why disclosure statements are actually meaningless.
Disclosure statements I would like to see
Study showing that exercise can promote weight loss
Disclosures: the lead author has been a lifelong sports enthusiast and firm believer in the notion that “exercise is medicine”. He is also actively fund raising for a new Institute of Sports Medicine and has made a lucrative side career as motivational speaker with regular media appearances to promote exercise as the cure for all that ails us (including excess weight). The positive message of this paper, which just happens to be very much in line with the author’s thinking on the subject, clearly justifies the lack of a control group and the exclusion of participants who did not lose weight as non-compliant “outliers”.
Study showing that low vitamin D intake causes diabetes
Disclosures: The lead author is on the verge of promotion to tenure but still needs a couple of “big” papers for his CV. Given that a negative study is unlikely to make the cut, extra efforts were taken to massage the data to ensure a positive finding (at least in one subgroup) to increase chances of acceptance in a high impact journal. Never mind that this cross-sectional study cannot actually prove causality, the “provocative” title and the assumptions of causality made in the paper should ensure welcome media attention.
Study showing that weight gain may be linked to lower cognitive function
Disclosures: The senior author has long harboured (in secret) the opinion that fat people cannot be very smart (after all how difficult is it to push away from the plate?). She is also the author of a book on using will-power and positive thinking on conquering obesity and another book on improving cognitive health. She also runs a website where she promotes the use of nutrition supplements to enhance brain function. Never mind that in this study unintentional weight loss had an even bigger effect on cognitive decline – that, after all, is nothing anyone can do much about (also makes for a less sexy paper).
Study showing that early breast cancer screening can save lives
Disclosures: At age 15, the principal investigator lost her mother to breast cancer. This is why the paper chooses to focus on the lives saved rather than on the potential harm caused by frequent false-positives findings or the poor cost-effectiveness of routine screening.
Study showing that individuals with obesity spend food stamps on sugar-sweetened beverages
Disclosures: The author is a card-holding member of the Libertarian party and would like to see government reduce taxes and end hand-outs to folks not willing to work for a living.
Study showing that industry funded research is more likely to show favourable results
Disclosures: The left-leaning senior author has long held the view that Big Pharma, Big Food, Big Energy, Big (take your pick) is the root of all evil and must be kept away from the holy church of independent science, even if much of the research funded by these organisations would never get done (even better!).
Study showing that a widely used cholesterol lowering drug has prevented over 1,000,000 heart attacks
Disclosures: The principle investigator has on several occasions been invited to a cup of coffee by the sales rep of the company that makes this drug. The masters student and post-doc involved in this study have each eaten a slice of pizza (may have been two slices) provided by another (unrelated) drug company at a science fair celebrating young researchers.
Why disclosure statements are meaningless
On a regular basis, whether it is for papers I write, conferences I speak at, or committees I sit on, I am deluged by “Conflict of Interest Declaration Forms” that seemingly require more personal information than my annual tax return.
The goal of all of this, apparently, is to provide “transparency”, so that the respective audience can judge the objectivity (or lack thereof) of my work based on whether or not I may or may not have a percieived “conflict of interest”.
In my humble opinion, this is an entirely irrelevant and useless exercise, which does nothing to actually ensure objectivity in how my work or actions are interpreted or perceived.
[The “disclosure statements I would I would like to see” above are] meant to illustrate how non-declared “conflicts of interest” may be as (if not more) relevant to a real or perceived conflict than whether or not I have consulted for a company or received research funding from industry (or, for that matter, any other interest group – by definition an “interest group” is interested in the outcome of where its money goes – no group that I am aware of is giving away free handouts).Consider the issue of peer review. Although hardly perfect, the whole purpose of the time-honoured peer-review process is to allow knowledgeable peers to evaluate the scientific merit of a paper. It is their job to fairly evaluate the paper to the best of their ability: Is the topic important? Is the hypothesis relevant? Is the methodology valid? Are the proper statistical tests applied? Are the full data presented? Are the findings interpreted cautiously (and not overstated)? Are limitations acknowledged?
These are the questions that count – in fact, they are the only questions that count. Who funded the study, or what the personal relationship of the authors were to the funding source, is entirely irrelevant – all funders pursue goals, whether commercial, political, or ideological – who cares?
If the paper meets the scientific standards required by the journal (and we assume here that higher impact journals have higher standards and do a more thorough job of vetting all of the relevant aspects of a paper), the funding source should be irrelevant – if the study is well conducted, the findings should stand on their own merit. If anyone does not believe the data or findings or interpretations, they are welcome to disagree – but their criticism should be based on scientific arguments – not just by pointing fingers at the funding source (or ad hominem attacks on the authors).
If any serious doubts do arise about any of the above questions (e.g. methodology, analysis, interpretation, etc.), it is up to the reviewers and editors to either request clarification or to reject (or even retract) the paper. After all, that is what the whole notion of peer-review is about.
So what about the argument that industry funded studies are more likely to report positive findings than other research and should therefore be taken with a grain of salt?
I can think of several possible explanations including the simple fact that no industry that wants to stay in business is likely to fund a trial where there is not at least a fighting chance of having a favourable result.
I am therefore not at all surprised that industry often goes to great lengths to perform due diligence regarding what trials to fund (often more so than some peer-review committees I have sat on) in the hope for a “positive” outcome. Studies that don’t stand a fair chance of producing positive findings is not where industry is likely to (or can be reasonably expected to) put its money. This, however, is not the same as saying that the data or the study (or the investigators) are somehow manipulated to produce positive results – that would be outright scientific fraud.
So rather than wondering about why industry funded studies so often tend to be favourable, I am in fact surprised every time this “biased” funding by industry does leads to results that are far from favourable (or even damaging) to the sponsor (for e.g. I just happened to be one of the PIs of a 10,000 patient study on a an anti-obesity drug, which showed this drug to modestly increase the risk for non-fatal cardiovascular events, a finding which led to the drug being taken off all markets worldwide – hardly a result that the sponsor (who footed the cost of almost $200 million for the trial) wanted to see).
Every researcher I know would like to see their study confirm their favourite hypothesis (or rather discard the null-hypothesis) – the funding source has nothing to do with this – the rewards of a positive finding are evident: high-impact publications, peer-recognition, media interest, promotion, tenure, and funding for yet more studies. I have yet to meet a “successful” researcher who has build a career on a track record of “negative” studies.
But peer-review is not the only mechanism that provides checks and balances. Clinical trials have to be registered, study protocols have to be vetted by ethics committees, good clinical practice guidelines need to be followed, sites are monitored (including random and targeted checks by regulators), primary data sources have to be archived, raw data may have to be made available to the reviewers (or even the public), data monitoring boards must ensure participant safety, the list of checks and balances (at least in clinical trials) goes on and on.
None of this will provide 100% protection against fraud or criminal intent – but nor will a disclosure of the funding source or a statement as to what shares my grandkids happen to own in their education funds.
The only consequence that I see resulting from “disclosures gone wild” is the undermining of public trust in the scientific process. Thus, no matter how relevant, precise, accurate, arms-length or important the findings – simply seeing a statement of industry funding on a paper, is often automatically interpreted as tainting the study.
Oddly enough, the same folks who would criticize an industry funded study showing a positive result for a given product, would often have no problem citing that same study if it happened to show an outcome more in line with their own views and thinking on the matter.
So, rather than obsessing about who is funding what, let us allow the science speak for itself. Let us make sure we respect the peer-review process and ensure that all the other checks and balances are in place.
If we do not trust the scientific process, the addition of a disclosure statement will hardly make us trust it more.
Editors note: This was originally published at Dr. Sharma’s Obesity Notes and has been republished here with permission from the author and minimal editing indicated by brackets. Original posts: Disclosure Statements I Would Like to See and Why Disclosure Statements Are Meaningless.
Image may be NSFW.
Clik here to view.
Dr. Arya M. Sharma is the Chair of Obesity Research and Management at the University of Alberta. Follow him on Twitter @DrSharma.
The post Why disclosure statements are meaningless appeared first on Biology Fortified, Inc..