Skewness-Guided Pruning of Multimodal Swin Transformers for Federated Skin Lesion Classification on Edge Devices

arXiv ID: 2512.08751v1

Published: 2025-12-09

Authors: Kuniko Paxton, Koorosh Aslansefat, Dhavalkumar Thakker, Yiannis Papadopoulos

Categories: cs.CV, cs.DC

Relevance Score: 1.00 / 1.00

View on arXiv Download PDF

Summary

This paper introduces a novel skewness-guided pruning method for multimodal Swin Transformers to enable efficient and privacy-preserving skin lesion classification on edge devices using Federated Learning (FL). By selectively pruning Multi-Head Self-Attention and Multi-Layer Perceptron layers based on output distribution skewness, the method achieves significant model size reduction (approx. 36%) without sacrificing diagnostic accuracy, making high-performance medical AI deployable in resource-constrained, privacy-sensitive environments.

Medical Relevance

This research is crucial for making advanced medical AI accessible and practical in real-world clinical settings, enabling high-accuracy diagnostic tools for conditions like skin lesions to be utilized directly on edge devices while strictly adhering to patient data privacy regulations.

AI Health Application

The AI application is the development of robust, efficient, and privacy-preserving AI models (specifically multimodal Swin Transformers) for automated skin lesion classification and diagnosis. This aims to facilitate the deployment of advanced medical diagnostic AI on edge devices, such as those used in clinics or by patients, while adhering to data privacy regulations through Federated Learning.

Key Points

Addresses the challenge of deploying large, computationally intensive medical AI models (e.g., for skin lesion classification) on edge devices.
Proposes a skewness-guided pruning method specifically targeting multimodal Swin Transformers.
Selectively prunes Multi-Head Self-Attention (MHSA) and Multi-Layer Perceptron (MLP) layers.
Pruning decisions are based on the statistical skewness of the layers' output distributions.
Validated in a horizontal Federated Learning (FL) environment to ensure privacy-preserving distributed learning.
Achieved approximately 36% model size reduction for a compact Swin Transformer without any loss in classification accuracy.
Demonstrates the feasibility of efficient model compression combined with privacy-preserving distributed learning for medical AI on edge devices.

Methodology

The study proposes a novel skewness-guided pruning approach applied to multimodal Swin Transformers. This method selectively removes parameters from the Multi-Head Self-Attention (MHSA) and Multi-Layer Perceptron (MLP) layers. The pruning criterion is based on the statistical skewness of the output distributions of these layers, aiming to identify and remove less informative or redundant components. The effectiveness of this method was validated within a horizontal Federated Learning environment.

Key Findings

The proposed skewness-guided pruning method successfully achieved a substantial reduction in model complexity. Specifically, experiments on a compact Swin Transformer demonstrated an approximate 36% reduction in model size without any discernible loss in diagnostic accuracy. These findings confirm the method's ability to maintain high performance while significantly enhancing model efficiency, which is critical for edge device deployment.

Clinical Impact

This technology holds the potential to revolutionize the deployment of AI in dermatology by enabling highly accurate skin lesion classification models to run directly on local, resource-constrained edge devices (e.g., clinic workstations, specialized handheld devices). This facilitates immediate diagnostic support, improves data privacy by reducing reliance on centralized data, and makes advanced AI accessible in settings where internet connectivity or computational resources are limited, ultimately enhancing patient care and diagnostic efficiency.

Limitations

Not explicitly mentioned in the abstract.

Future Directions

Not explicitly mentioned in the abstract.

Medical Domains

Dermatology Medical Imaging Computational Pathology AI in Healthcare

Keywords

Federated Learning Model Pruning Swin Transformer Edge AI Skin Lesion Classification Multimodal AI Model Compression Skewness

Abstract

In recent years, high-performance computer vision models have achieved remarkable success in medical imaging, with some skin lesion classification systems even surpassing dermatology specialists in diagnostic accuracy. However, such models are computationally intensive and large in size, making them unsuitable for deployment on edge devices. In addition, strict privacy constraints hinder centralized data management, motivating the adoption of Federated Learning (FL). To address these challenges, this study proposes a skewness-guided pruning method that selectively prunes the Multi-Head Self-Attention and Multi-Layer Perceptron layers of a multimodal Swin Transformer based on the statistical skewness of their output distributions. The proposed method was validated in a horizontal FL environment and shown to maintain performance while substantially reducing model complexity. Experiments on the compact Swin Transformer demonstrate approximately 36\% model size reduction with no loss in accuracy. These findings highlight the feasibility of achieving efficient model compression and privacy-preserving distributed learning for multimodal medical AI on edge devices.