Large Multimodal Models for Low-Resource Languages: A Survey

In this survey, we systematically analyze techniques used to adapt large multimodal models (LMMs) for low-resource (LR) languages, examining approaches ranging from visual enhancement and data creation to cross-modal transfer and fusion strategies. Through a comprehensive analysis of 117 studies across 96 LR languages, we identify key patterns in how researchers tackle the challenges of limited data and computational resources. We categorize works into resource-oriented and method-oriented contributions, further dividing contributions into relevant sub-categories. We compare method-oriented contributions in terms of performance and efficiency, discussing benefits and limitations of representative studies. We find that visual information often serves as a crucial bridge for improving model performance in LR settings, though significant challenges remain in areas such as hallucination mitigation and computational efficiency. In summary, we provide researchers with a clear understanding of current approaches and remaining challenges in making LMMs more accessible to speakers of LR (understudied) languages. We complement our survey with an open-source repository available at: https://github.com/marianlupascu/LMM4LRL-Survey.

💡 Research Summary

**
This survey systematically reviews the literature on adapting large multimodal models (LMMs) to low‑resource (LR) languages, covering the period from 2018 to 2025. A total of 117 papers addressing 96 distinct LR languages were collected from major digital libraries (ACM DL, IEEE Xplore, ACL Anthology, arXiv, SpringerLink, ScienceDirect, Google Scholar) using keyword combinations that capture multimodality, low‑resource aspects, and language‑specific tasks. After a two‑stage manual screening (title/abstract followed by full‑paper relevance check), only works that (i) involve at least two modalities, (ii) target at least one LR language, and (iii) propose, adapt, or evaluate a multimodal model were retained.

The analysis is organized around two high‑level categories: resource‑oriented contributions (datasets, benchmarks, evaluation protocols) and method‑oriented contributions (model architectures, training strategies, fusion mechanisms). Method‑oriented works are further divided into four principal technical families:

1. Visual Enhancement – leveraging image‑text pairs to enrich textual representations. Techniques include multimodal pre‑training à la CLIP, image‑captioning, and vision‑language adapters. Visual cues act as a “bridge” for languages with scarce textual corpora, providing semantic grounding that improves downstream tasks such as classification, retrieval, and translation.

2. Data Creation – generating synthetic multimodal data through image synthesis, speech synthesis, back‑translation, and automatic labeling pipelines. The “Multimodal Data Augmentation (MMDA)” paradigm combines language‑model‑generated text with image generation models (e.g., Stable Diffusion) to produce large‑scale paired corpora for languages that lack any native resources.

3. Cross‑modal Transfer – transferring knowledge from high‑resource (HR) languages to LR languages via shared visual embeddings, multilingual visual lexicons, and word‑image attention mechanisms. By aligning visual concepts across languages, researchers can reuse pretrained vision‑language encoders and fine‑tune only lightweight language adapters, dramatically reducing data requirements.

4. Fusion Strategies – designing how multiple modalities are combined inside the model. Early fusion (concatenating raw inputs), mid‑fusion (cross‑modal attention layers), and late fusion (ensemble of unimodal experts) are compared. For LR settings, lightweight fusion modules that share parameters and employ sparse attention have been shown to preserve performance while keeping GPU memory and FLOPs within modest budgets.

The survey also presents a quantitative landscape of research focus. Text‑image is the dominant modality pair (76 papers, 65 % of the corpus), while more complex combinations involving audio or video appear in fewer than 10 % of studies. Language coverage is highly uneven: Hindi (31 papers), Arabic (23), Bengali (21), Malayalam (19), and Tamil (14) dominate, whereas 42 languages appear in only a single study each. Table 1 in the paper identifies six interacting factors that explain this disparity: (1) institutional research capacity, (2) speaker population size, (3) existing digital resources, (4) script similarity to HR languages, (5) geographic proximity to major NLP venues, and (6) geopolitical interest. The authors argue that strategic state interests (e.g., post‑9/11 Arabic research, post‑2022 Ukrainian research, US‑China AI competition) drive disproportionate investment toward certain LR languages, leaving many truly marginal languages under‑studied.

Performance comparisons across method‑oriented works reveal that visual enhancement typically yields 3‑5 % absolute gains on text‑image tasks, while synthetic data creation can improve low‑data baselines by 10‑15 %. However, multimodal models that incorporate three or more modalities suffer from steep increases in computational cost (2‑3× FLOPs) and higher rates of hallucination—generation of content not grounded in any input modality. The survey highlights emerging solutions such as multimodal verification frameworks, knowledge‑constrained decoding, and the use of Mixture‑of‑Experts, sparse attention, and quantization to curb resource demands.

In the discussion, the authors stress three major open challenges: (a) hallucination mitigation, (b) computational efficiency for resource‑constrained environments, and (c) the lack of standardized multilingual multimodal benchmarks. They recommend a community‑driven approach to dataset creation, emphasizing the CARE principles for Indigenous data governance, and suggest framing future research around broader societal concerns (climate migration, pandemic communication, regional stability) to attract funding for under‑represented languages.

Overall, the survey provides a comprehensive taxonomy of techniques, a detailed statistical overview of language and modality coverage, and a forward‑looking agenda that calls for more equitable resource allocation, robust evaluation protocols, and ethically grounded data practices to make large multimodal models truly inclusive for speakers of low‑resource languages.