Unsupervised Video Summarization Using GAN and BiLSTM-based Self-Attention Network

Document Type : Original Article

Authors

1 Faculty of Electrical and Computer Engineering, Qom University of Technology, Qom, Iran

2 University of Genoa

Abstract
This paper presents an approach for automated unsupervised video summarization, that means, nothing more than video is needed to train the model. The goal is to extract a sequence of frames from an input video and assign each frame a score between 0 and 1. By doing so, we can select a subset of the most informative and diverse shots to make a summarized video. We build upon the foundation of SUM-GAN, particularly SUM-GAN-SLA, which utilize Generative Adversarial Networks to compare and distinguish between the original video and its regenerated counterpart. A key contribution of our work lies in the novel biLSTM-based self-attention network that we introduce to handle the crucial scoring layer of our model. We adjusted several aspects of the model, particularly in the loss functions and learning steps, to enhance the training process and achieve superior performance compared to state-of-the-art unsupervised and even supervised methods. To ensure a fair comparison, we evaluate our proposed model using two widely used datasets: SumMe and TVSum. The experimental results highlight the effectiveness of our proposed approach in automated unsupervised video summarization, achieving a 1.2% improvement over the best-performing methods' average F-score on SumMe and TVSum datasets. Additionally, our method ranks second among state-of-the-art unsupervised methods on each dataset. Notably, the top-performing methods exhibited inconsistent results across datasets, underscoring the broader applicability of our approach to diverse types of videos. Furthermore, our method demonstrates competitive performance compared to supervised approaches, with the best supervised method surpassing our results by only 0.75%.

Keywords

Subjects

Volume 1, Issue 4
Autumn 2025
Pages 205-216

  • Receive Date 16 November 2024
  • Revise Date 07 January 2025
  • Accept Date 15 February 2025
  • First Publish Date 12 July 2025