[Lex Computer & Tech Group/LCTG] ChatGPT version 3.5 seems to be ailing

[Robert Primak]

 Harry: "This headline makes JAMA Pediatrics look like they don't understand what is going on in the AI world these days where improvements are happening quickly."
Or maybe they were being very conservative. Also, the study may have been done some time earlier. 
And yes, Mike, the models could point you in the wrong direction. Expert models also need humans to make sure the results make sense. 
-- Bob Primak 

    On Thursday, January 4, 2024 at 11:08:18 AM EST, Harry Forsdick <forsdick at gmail.com> wrote:  
 
 I'm somewhat surprised that a JAMA Pediatrics would publish such an article knowing that ChatGPT 3.5, although free, is inferior to ChatGPT 4.0, Bard and others.  The article was submitted on October 24, 2023 and so the better LLM Bots were available.  This headline makes JAMA Pediatrics look like they don't understand what is going on in the AI world these days where improvements are happening quickly.
-- Harry



| Harry Forsdick
Lexington Photo Scanning
Town Meeting Member Precinct 7
harry at forsdick.com
www.forsdick.com
 |   | 46 Burlington St.
Lexington, MA 02420
(781) 799-6002 (mobile)
meet.jit.si/HarryForsdick (video)
Click to see my other websites
 |




On Thu, Jan 4, 2024 at 10:44 AM Robert Primak via LCTG <lctg at lists.toku.us> wrote:

 (1) This was ChatGPT 3.5. ChatGPT 4.0 might have done better. BARD, with its more up to date and broader training data might have done better still.
(2) These LLMs are general purpose, and are not designed to deal with specific, technical material. So of course they are not (yet) very good at making medical diagnoses. To expect otherwise is like asking WebMD to be your Primary Care Physician. Results may vary, to say the least! 
What LLM models can do, is point you toward possible diagnoses, and maybe suggest a diagnosis which you might not have considered. Then a trained and experienced clinician can narrow down the choices. This way the LLM has reduced the possibilities and helped direct the inquiry. Time can be saved in this way. Sometimes this is a good practice, sometimes not so much. 
-- Bob Primak 


    On Thursday, January 4, 2024 at 10:31:19 AM EST, Slater Nancy via LCTG <lctg at lists.toku.us> wrote:  
 
 
ChatGPT gave the wrong diagnosis for the majority of pediatric cases

MedPage Today (1/2,  Henderson ) reports, “A large language model-based chatbot gave the wrong diagnosis for the majority of pediatric cases, researchers found.” According to the study, “ChatGPT version 3.5 reached an incorrect diagnosis in 83 out of 100 pediatric case challenges. Among the incorrect diagnoses, 72 were actually incorrect and 11 were clinically related to the correct diagnosis but too broad to be considered correct.” The findings were published in JAMA Pediatrics.

===============================================
::The Lexington Computer and Technology Group Mailing List::
Reply goes to sender only; Reply All to send to list.
Send to the list: LCTG at lists.toku.us      Message archives: http://lists.toku.us/pipermail/lctg-toku.us/
To subscribe: email lctg-subscribe at toku.us  To unsubscribe: email lctg-unsubscribe at toku.us
Future and Past meeting information: http://LCTG.toku.us
List information: http://lists.toku.us/listinfo.cgi/lctg-toku.us
This message was sent to bobprimak at yahoo.com.
Set your list options: http://lists.toku.us/options.cgi/lctg-toku.us/bobprimak@yahoo.com
  ===============================================
::The Lexington Computer and Technology Group Mailing List::
Reply goes to sender only; Reply All to send to list.
Send to the list: LCTG at lists.toku.us      Message archives: http://lists.toku.us/pipermail/lctg-toku.us/
To subscribe: email lctg-subscribe at toku.us  To unsubscribe: email lctg-unsubscribe at toku.us
Future and Past meeting information: http://LCTG.toku.us
List information: http://lists.toku.us/listinfo.cgi/lctg-toku.us
This message was sent to forsdick at gmail.com.
Set your list options: http://lists.toku.us/options.cgi/lctg-toku.us/forsdick@gmail.com

  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.toku.us/pipermail/lctg-toku.us/attachments/20240104/db2763fc/attachment.htm>