Amir Jafari
Associate Professor of Data Science
George Washington University
Area of Expertise: Trustworthy AI for Large Language Models
Amir Jafari is an assistant professor of data science at George Washington University. His research focuses on operationalizing principles of trustworthy AI in the development of customized applications of open-source large language models, with the goal of creating a practical planning guide and toolkit for student and faculty project teams.
-
Mudgal, A., Kush, U., Kumar, A. et al. Multimodal fusion: advancing medical visual question-answering. Neural Comput & Applic 36, 20949–20962 (2024).
This paper explores the application of Visual Question-Answering (VQA) technology, which combines computer vision and natural language processing (NLP), in the medical domain, specifically for analyzing radiology scans. VQA can facilitate medical decision-making and improve patient outcomes by accurately interpreting medical imaging, which requires specialized expertise and time. The paper proposes developing an advanced VQA system for medical datasets using the Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation (BLIP) architecture from Salesforce, leveraging deep learning and transfer learning techniques to handle the unique challenges of medical/radiology images. The paper discusses the underlying concepts, methodologies, and results of applying the BLIP architecture and fine-tuning approaches for VQA in the medical domain, highlighting their effectiveness in addressing the complexities of VQA tasks for radiology scans. Inspired by the BLIP architecture from Salesforce, we propose a novel multi-modal fusion approach for medical VQA and evaluating its promising potential.
Full Paper -
Tsai, HT., Wu, J., Gupta, P. et al. Predicting blood transfusions for coronary artery bypass graft patients using deep neural networks and synthetic data. Neural Comput & Applic 36, 21153–21162 (2024).
Coronary Artery Bypass Graft (CABG) is a common cardiac surgery, but it continues to have many associated risks, including the need for blood transfusions. Previous research has shown that blood transfusion during CABG surgery is associated with an increased risk for infection and mortality. The current study aims to use modern techniques, such as deep neural networks and data synthesis, to develop models that can best predict the need for blood transfusion among CABG patients. Results show that neural networks with synthetic data generated by DataSynthesizer have the best performance. Implications of results and future directions are discussed.
Full Paper -
Huang, G., Jafari, A.H. Enhanced balancing GAN: minority-class image generation. Neural Comput & Applic 35, 5145–5154 (2023).
Generative adversarial networks (GANs) are one of the most powerful generative models, but always require a large and balanced dataset to train. Traditional GANs are not applicable to generate minority-class images in a highly imbalanced dataset. Balancing GAN (BAGAN) is proposed to mitigate this problem, but it is unstable when images in different classes look similar, e.g., flowers and cells. In this work, we propose a supervised autoencoder with an intermediate embedding model to disperse the labeled latent vectors. With the enhanced autoencoder initialization, we also build an architecture of BAGAN with gradient penalty (BAGAN-GP). Our proposed model overcomes the unstable issue in original BAGAN and converges faster to high-quality generations. Our model achieves high performance on the imbalanced scale-down version of MNIST Fashion, CIFAR-10, and one small-scale medical image dataset.
Full paper