ChatGPT is versatile, for example, it can write and debug computer programs, compose music, teleplays, fairy tales, and student essays. It can even answer test questions (sometimes, depending on the test, at a level above the average human test-taker).
It clearly announces a new class of knowledge tools, in the same vein as search engines, but with the potential to disrupt intellectual professions that rely on less than state of art knowledge.
Complex central central nervous system tumour cases require multidisciplinary expert recommendations that incorporate multimodal disease information. Thus, the potential of ChatGPT to integrate comprehensive treatment information may be of tremendous benefit for central nervous system tumour decision-making.
Scientists from Switzerland evaluated the ChatGPT recommendations for glioma management by a panel of central nervous system tumour experts composed of neurosurgeons, oncologists, neurologists, pathologists, radiation oncologists and neuroradiologists
They randomly selected 10 patients with primary central nervous system gliomas discussed at authors' institutions Tumour Board.
Patients clinical status, surgical, imaging, and immuno-pathology-related information was provided to ChatGPT and seven central nervous system tumour experts.
The description contained the main clinical information, context of admission, preoperative radiological and surgical information, postoperative clinical information, neuropathological findings, and results of the immunohistochemical and molecular examination. No diagnosis nor patient identification information was provided to ChatGPT.
Two questions were then asked to ChatGPT: 1) “What is the best adjuvant treatment?”, 2) “What would be the regimen of radiotherapy and chemotherapy for this patient?”.
The experts rated the AI-based recommendations from 0 (complete disagreement) to 10 (complete agreement). An intraclass correlation agreement (ICC) was used to measure the inter-rater agreement.
Eight patients (80%) met the criteria for glioblastoma and two (20%) were low-grade gliomas. The experts rated the quality of ChatGPT recommendations as poor for diagnosis, but good for treatment recommendation, for therapy regimen, yet moderate for functional status consideration, and moderate for overall agreement with the recommendations.
ChatGPT’s performance was mediocre regarding recommendations that are based on less extensive knowledge base. The consideration of patient functional status was rated as moderate, even though the clinical pre- and postoperative state of the included cases was presented to ChatGPT. This consideration is much less documented in the literature as only a few clinical trials studied adjuvant therapy for glioblastoma in impaired functional conditions or in older patients.
No difference were observed between the glioblastomas and low-grade glioma ratings.
In conclusion, ChatGPT performed poorly in classifying glioma types but was good for adjuvant treatment recommendations as evaluated by central nervous system Tumour Board experts.
Even though the ChatGPT lacks the precision to replace expert opinion, it may become a promising tool to supplement experts, especially in low-resource settings.