PaveCap: A Multimodal Framework for Comprehensive Pavement Condition Assessment with Dense Captioning and PCI Estimation

This research introduces a multimodal framework for automated pavement condition assessment that provides pavement condition index (PCI) predictions and qualitative descriptions using a single-shot PCI estimation network and a fine-grained dense captioning network. The PCI estimation network uses YOLOv8, the segment anything model, and a four-layer convolutional neural network for PCI prediction. The dense captioning network uses a YOLOv8 backbone, a transformer architecture, and a convolutional feed-forward module to generate textual descriptions. To train and evaluate these networks, we developed a pavement dataset containing bounding box annotations, textual descriptions, and PCI values. The PCI estimation…