Join the discussion
Write your take first — we'll ask for email only when you're ready to publish.
- Hacker News
- Feels like the minimum standard should be sharing the exact query/design choices and being very explicit about what biases the analysis can and cannot addressby BrtByte
- Some of the examples of bad design mentioned in the article are quite shocking, if we are assuming 'medical student' is not some 18 year old rookie but a person who already has several years of university level study behind them.by harvey9
- admissions and residency matching give a lot of weight to "research output", aka publications.
For residency, the two most important things are: 1) board scores. 2) research output.
It's not uncommon to see 40-50 publications for competitive residencies.
incentives, incentives, incentives.
- >incentives, incentives, incentives.
Exactly. The people at the John Hopkins Camel-Winston Center for Respiratory Research know that Camel and Winston are gonna stop cutting the checks if they don't get something they can trot out to support them when they get hauled in front of congress even if that's only a minority of the work output they funded.
Even if nobody means to do evil the evil will be done just as a result of people factoring that in subconsciously and the pressure that applies systemically.
Where is really gets spicy is when you have dueling funding sources. Where I went to school you had the environmental compliance industry funding the public policy research to say that nobody should be allowed to install a fencepost without paying their way through some hoops while the Kochs (through some indirection) were funding different research in the same department to say no akshually that compliance stuff is making us all poorer.
- Really grateful for people like Wang making an effort to deter this behavior.
Hoping more folks like him defend the guardrails.
ill-incentives have always influenced academia, but I’m hoping we’re able to walk it back a bit
by googaar - Some context to this is recently the big "step 1" board exam that covered the basic science portion of medicine went from being graded to being pass/fail. Since then, all the medical students where I work have started trying to get any kind of research on their CV to stand out during their residency application. New 120 participant "conferences" where med students can present posters are popping up, more med students trying to get into the labs for rotations than ever before. A lot of it is really low quality, but for them, if they don't have any research, and someone else does, they will be ranked lower and get worse jobs. Weird effect of whatever happened to make Step 1 pass/failby bagelbob432
- My assumption is the credibility of a non-PhD-holding medical student’s research is 0, just like (almost) any other inexperienced researcher.by fn-mote
- I guess it depends on who the coauthors and PI are - some academic mentors can be overly trusting and ‘hands-off.’ A lone medical student’s self published paper shouldn’t be worth much though…by aardvark92
- LLVM was a masters thesis project (not medicine related but research by non PhDs should not be disregarded imo)by samuraijack
- a friend of a friend who did a stint in biomedical academia told me that the researchers in their field did not hold research coming from the medicine community in high regard
- In the end it is about personal integrity and idealism, no matter what the titles are.
Totally different if someone's self image is that of a researcher for benefit of humankind or if they pick the career because they want to drive a Porsche.
by bflesch - A med student can absolutely contribute useful work, especially with good supervision. The issue is more that inexperienced authors plus publication pressure plus easy tooling is a bad combinationby BrtByte
- This is really far too broad a brush.
Do most medical students publish useless case studies trying to jockey for residency spots and signal hustle/devotion? No doubt!
But there are a good handful of medical students who are still (surprisingly) in it for the medicine and not the money. And that handful is exceedingly capable; no reason they can’t publish valuable work with the right collaborators and resources.
by sebmellen - Well, that is a statement..! As an MD PhD with over 60 (co-)publications including multiple in top 1% journals I can say for sure that this is untrue. Of course this may be different per topic and country, but there is perfect research being published by non-PhD scientists. In fact, the PI from a top-tier US university I collaborate with for over 10 years doesn't even have a PhD.by niekmaas
- As a clinician-academic who published in The Lancet during medical school, I think this goes a bit far. Unfortunately student doctors are encouraged to publish whether or not they actually have an interest in research… but that shouldn’t discount the work of those who are genuinely engaged.
But certainly we should always approach the literature critically, including the author list, journal of publication and its peer-review practices, and the methods.
by thomasfedb - That aside I am a bit perplexed that almost absolute insistence of medical students to become researchers as well, it seems just become a pure practitioner is not a feasible option. To make it worse it looks more acceptable that a doctor really provide bad service or lacking communication and empathy to patients than not being a researcher.by eunos
- It's kinda like saying high school students insist on taking AP or SAT Subject testsby frollogaston
- Clinical research is in a weird spot, where you need both clinical experience and research experience. Getting the former already requires long hours and you are swamped with work. The latter is highly age gated and if you want to pursue research you often need to achieve specific milestones before a certain age, e.g. to be considered for tenure etc.by alcasa
- Guidelines and standards can change quickly. It's important for clinicians to have a foundational understanding of statistical methods and to be able to critique studies.
For certain specialties, the number of residency positions is so limited that medical students have to publish research to be competitive, even if they have no interest in doing so later in their career.
But altogether I sort of agree, the incentives are pretty maligned such that for many it's just easier to become a bad scientist with more publications than a good one with fewer.
by nickgros - Most doctors drop the research once they get a residency spot. It's just the selection mechanism for residency.by GerryAdamsSF
- Researcher/academics pay/promotoins should be contingent on reviewing,challenging and reproducing papers rather than publishing quantity, because publishing cartels and AI has already degraded most research fields.by whizzter
- No. You can't spend all your money on rehashing past results. Some, OK, all, not. In many fields, the money is needed for discovery.
- Reproducibility in many scientific as areas has been made almost impossible. We have got to the stage where IP matters more than scientific rigour so methodology is purposely left out.by ktallett
- Should be, but you've got to tell the funders that.by pjc50
- I’m currently facing a severe health condition and I can’t help but ask various LLMs about it. They all will eventually offer solutions or avenues that sound promising or ‘easy’ when I know better, that the path ahead of me is hard, however they will non-chalantly offer a path forward they insist will work. The best part is when it cites commercial websites promotional statements as facts, though it will also misinterpret medical journals if I say to restrict itself to that.
In closing, my Redfin escapism has shifted to LLM medical escapism, I know better but if you don’t or you are in even more dire straits, it provides such an illusion of hope and that’s dangerous.
- Try OpenEvidence insteadby engcoach
- I was having a conversation with someone about AI taking over medical jobs the other day, and one of the things that came out of my thoughts about it is related:
It obviously has flaws, and we should never stop trying to improve it, but I think AI can be a great way to help connect a bunch of information they have to a bunch of information they don't, or to help spot patterns and potential avenues that they happened to miss. And obviously you want to be careful about becoming over-reliant on it, or being too trusting when it can be wrong. But I think we've been at the point for a long time where a doctor using a search engine to find medical literature should be a very reasonable thing to do, and I think AI can at least be an incremental (but massive) improvement on that workflow.
But I hope the end result of that is that doctors can not only deliver better (and maybe better-informed and more open-minded) treatment, but can spend the time focusing on patient care, managing expectations / risk management around uncertainty, managing the emotions inherent in someone who may be losing their life or the lives of their loved ones. Those are things AI is definitely not well-equipped to do as you point out.
by TallGuyShort - residencies have decided to outsource part of their hiring decisions to journal peer-review processes. so now for some submissions, editors and reviewers are not actually doing scientific peer review, but rather screening job candidates for hospitals.
peer review is built to assume good faith work by people who are all part of a community of scholarship, it can partially hold up to people within the community gaming metrics. if people are just going to appear, game the system to publish some papers, and then disappear into their real careers, there's no hope of this working.
i don't understand why residencies want med students to publish papers anyway. it's very difficult to do good scientific research, it requires training, time, and almost always apprenticeship. none of this is part of the medical school curriculum, which is why we need special MD-PhD programs for people who want to do both. nobody expects that doing a PhD in biology or epidemiology would give you any clinical know-how, why is it reasonable to expect the reverse?
by currymj - Residency is a guild (aka, anticompetitive trust) designed to limit the medical workforce and drive up doctor salaries. It needs artificial metrics to ratonalize the increasing salaries as being for "increasing talent".by gowld
- As a former scientist it's embarrassing how easily the dismal state of science today could have been predicted decades before by applying Goodhart's Law, or any simple train of thought on incentives and moral hazard. Instead we chose to assume scientists collectively behave on a higher plane. No wonder the general public distrusts "the intelectual elite", we deserved it.