PhD Scientific Days 2024

Poster Session S - Health Sciences 2.

ScreenGPT – Evaluating Different Customization Techniques for Language Models in Helathcare Applications

Előadó neve

Angyal, Viola, MSc

Neptun code

RDU2PC

Előadó munkahelye

Semmelweis University Doctoral School, Health Sciences Section, Budapest

Előadó telefonszáma

0036304700105

Előadó e-mail címe

angyal.viola@phd.semmelweis.hu

Az előadás címe

ScreenGPT – Evaluating Different Customization Techniques for Language Models in Helathcare Applications

Szerző(k) neve és munkahelye

Angyal Viola¹

1: Semmelweis University, Doctoral School of Health Sciences

Bemutatás módja

Poszter

Szekció

Poster Session S - Health Sciences 2.

Language of the presentation

English

Preferred session

Health Sciences

Összefoglaló szövege

Introduction: In the field of language modeling, distinctions are commonly made based on model size, categorized as either Small Language Models (SLM) or Large Language Models (LLM). Both types are generative artificial intelligence systems that belong to Natural Language Processing (NLP) and can be used for similar purposes. When developing healthcare applications with language models, it's important to consider their size, efficiency, costs, and application areas to choose the most suitable one.
Aims: We aimed to develop a preliminary version of our healthcare-related web application, ScreenGPT, and make it testable for external testers. We customized it for primary and secondary healthcare prevention, especially in cervical cancer screenings, and gathered feedback. We examined whether a customized LLM or SLM is better suited for this purpose.
Methods: We examined SLMs through scientific publications and utilized the GPT-4 model to test various methods for LLM customization. We explored the rapid custom-GPT building offered by OpenAI's GPT store. Additionally, we prepared a training dataset and fine-tuned the GPT-4 model. Ultimately, we tried prompt engineering for customization. For cervical cancer-related information, we referenced WHO guidelines. We used Python programming language and Streamlit to make the application testable.
Results: In comparison, we found that customizing a LLM, such as the GPT-4 model, is significantly easier than developing a SLM from the begining. The Open-AI custom-GPT building is intuitive and does not require programming skills, although it provides limited customization options. Fine-tuning the GPT-4 model can be challenging, as the quality of the training dataset is crucial. The highest instances of hallucinations and errors occurred when using this technique. Prompt engineering proved to be the most effective method. This technique provided the greatest flexibility and maintained consistency.
Conclusion: The possibilities for developing customized language models are expanding rapidly, with increasing options available, sometimes even without requiring any programming skills. SLMs are often more cost-effective and can be operated offline. However, LLMs offer greater customization capabilities, can exhibit more empathetic responses, and possess more extensive informational capabilities.

University

Semmelweis University

Supervisor

Prof. Dr. Dinya Elek

Publication of my abstract

I give consent to the publication of my abstract on the website of the congress.

Kind

Szabad

Status

elfogadva

Accepted presentation method

poszter

Előadás fájl jóváhagyás

nem rendelkezett róla

Előadó

7462

Start

16:10

End

16:13