[Lex Computer & Tech Group/LCTG] ChatGPT version 3.5 seems to be ailing

[Harry Forsdick]

I'm somewhat surprised that a JAMA Pediatrics would publish such an article
knowing that ChatGPT 3.5, although free, is inferior to ChatGPT 4.0, Bard
and others.  The article was submitted on October 24, 2023 and so the
better LLM Bots were available.  This headline makes JAMA Pediatrics look
like they don't understand what is going on in the AI world these days
where improvements are happening quickly.

-- Harry



Harry Forsdick <http://www.forsdick.com/resume/>
Lexington Photo Scanning <http://lexingtonphotoscan.com/>
Town Meeting Member Precinct 7 <http://lexingtontmma.org/>
harry at forsdick.com
www.forsdick.com
  46 Burlington St.
Lexington, MA 02420 <https://goo.gl/xZXT2F>
(781) 799-6002 (mobile) <callto:17817996002>
meet.jit.si/HarryForsdick (video)
Click <http://forsdick.weebly.com/home/my-websites> to see my other websites



On Thu, Jan 4, 2024 at 10:44 AM Robert Primak via LCTG <lctg at lists.toku.us>
wrote:

> (1) This was ChatGPT 3.5. ChatGPT 4.0 might have done better. BARD, with
> its more up to date and broader training data might have done better still.
>
> (2) These LLMs are general purpose, and are not designed to deal with
> specific, technical material. So of course they are not (yet) very good at
> making medical diagnoses. To expect otherwise is like asking WebMD to be
> your Primary Care Physician. Results may vary, to say the least!
>
> What LLM models can do, is point you toward possible diagnoses, and maybe
> suggest a diagnosis which you might not have considered. Then a trained and
> experienced clinician can narrow down the choices. This way the LLM has
> reduced the possibilities and helped direct the inquiry. Time can be saved
> in this way. Sometimes this is a good practice, sometimes not so much.
>
> -- Bob Primak
>
>
>
> On Thursday, January 4, 2024 at 10:31:19 AM EST, Slater Nancy via LCTG <
> lctg at lists.toku.us> wrote:
>
>
> ChatGPT gave the wrong diagnosis for the majority of pediatric cases
>
> MedPage Today
> <https://mailview.bulletinhealthcare.com/mailview.aspx?m=2024010301ama&r=4756748-1a09&l=030-2c0&t=c> (1/2,
> Henderson ) reports, “A large language model-based chatbot gave the wrong
> diagnosis for the majority of pediatric cases, researchers found.”
> According to the study, “ChatGPT version 3.5 reached an incorrect diagnosis
> in 83 out of 100 pediatric case challenges. Among the incorrect diagnoses,
> 72 were actually incorrect and 11 were clinically related to the correct
> diagnosis but too broad to be considered correct.” The findings
> <https://mailview.bulletinhealthcare.com/mailview.aspx?m=2024010301ama&r=4756748-1a09&l=031-14e&t=c> were
> published in JAMA Pediatrics.
>
> ===============================================
> ::The Lexington Computer and Technology Group Mailing List::
> Reply goes to sender only; Reply All to send to list.
> Send to the list: LCTG at lists.toku.us      Message archives:
> http://lists.toku.us/pipermail/lctg-toku.us/
> To subscribe: email lctg-subscribe at toku.us  To unsubscribe: email
> lctg-unsubscribe at toku.us
> Future and Past meeting information: http://LCTG.toku.us
> List information: http://lists.toku.us/listinfo.cgi/lctg-toku.us
> This message was sent to bobprimak at yahoo.com.
> Set your list options:
> http://lists.toku.us/options.cgi/lctg-toku.us/bobprimak@yahoo.com
> ===============================================
> ::The Lexington Computer and Technology Group Mailing List::
> Reply goes to sender only; Reply All to send to list.
> Send to the list: LCTG at lists.toku.us      Message archives:
> http://lists.toku.us/pipermail/lctg-toku.us/
> To subscribe: email lctg-subscribe at toku.us  To unsubscribe: email
> lctg-unsubscribe at toku.us
> Future and Past meeting information: http://LCTG.toku.us
> List information: http://lists.toku.us/listinfo.cgi/lctg-toku.us
> This message was sent to forsdick at gmail.com.
> Set your list options:
> http://lists.toku.us/options.cgi/lctg-toku.us/forsdick@gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.toku.us/pipermail/lctg-toku.us/attachments/20240104/b0ed9a79/attachment.htm>