Other Works

Remote sensing foundation models (RSFMs) are commonly evaluated using aggregate metrics, which can hide systematic performance disparities across ecological regions. We introduce FairRSFM, a biome-aware benchmark for evaluating ecological group robustness in RSFMs. FairRSFM maps georeferenced samples from 14 terrestrial biome classes into six ecologically meaningful macro-groups and evaluates models under a unified frozen-backbone evaluation protocol. The benchmark covers four downstream datasets: m-EuroSAT, m-BigEarthNet, m-SA-Crop-Type, and MMEarth20K with Dynamic World label maps. Using Prithvi-EO-2.0, SatMAE, and DOFA across three random seeds, we show that aggregate performance consistently masks biome-dependent disparities across architectures and tasks. For example, Prithvi-EO-2.0 reaches 90.98% overall macro-F1 on m-EuroSAT but a mean worst-group score of only 83.72%, while m-SA-Crop-Type drops from 27.30% overall mIoU to 18.47% in the Xeric and Mineralogical group. We further evaluate Biome-Orthogonal Linear Probing (BOLP), Dynamic Biome Reweighting (DBR), and GroupDRO as complementary mitigation baselines. Their effectiveness is model- and task-dependent; for example, BOLP improves Prithvi-EO-2.0 worst-group F1@opt on m-BigEarthNet from 46.12% to 50.27% without updating the RSFM backbone. FairRSFM provides a reusable protocol for diagnosing and mitigating ecological robustness gaps in remote sensing foundation models. Code and datasets are available at: https://github.com/aminurhossain/FairRSFM.

@article{hossain2026fairrsfm,
  title   = {FairRSFM: A Biome-Aware Benchmark and Debiasing Framework for Remote Sensing Foundation Models},
  author  = {Md Aminur Hossain and Omkumar Vaghasiya and Rajeev Ranjan Dwivedi and Vinod Kurmi and Biplab Banerjee},
  journal = {arXiv preprint arXiv:2610.05790},
  year    = {2026}
}

Self-supervised speech foundation models have substantially improved speech recognition and downstream speech classification. However, their behavior can remain uneven across demographic groups even when the downstream fine-tuning data is balanced. We study this issue using Wav2Vec 2.0 and S2T models fine-tuned on balanced splits of Fair-speech and FLEURS. Across gender, ethnicity, and economic-class prediction tasks, we find that precision and recall can diverge sharply across groups, indicating that class balance alone does not remove latent bias inherited from pre-training or introduced during adaptation. To mitigate this problem, we propose a three-phase framework that discovers proxy bias groups without requiring pre-specified protected attributes, learns bias-correlated representations from those proxy labels, and reduces their influence through feature fusion and reweighted supervision. Experiments show strong improvements for gender classification, reducing precision–recall gaps to nearly zero across both datasets and both model families. Economic and ethnicity classification tasks show partial parity improvements while revealing important remaining challenges for complex multi-class demographic attributes.

@inproceedings{sharma2026balanced,
  title     = {Balanced Fine-Tuning Is Not Enough: Latent Bias Mitigation in Speech Foundation Models},
  author    = {Utkarsh Sharma and Rajeev Ranjan Dwivedi and Vinod K Kurmi},
  booktitle = {GlobalSouthML Affinity Workshop at ICML},
  year      = {2026}
}

A deployed model can look healthy on average while silently becoming unsafe for a small intersectional subgroup. We propose E-FairDrift, a sequential test for this failure mode. The method first discovers candidate subgroups from black-box signals - representations, confidence, residuals, or explanation embeddings - and then monitors each subgroup with an e-process, a non-negative supermartingale that remains valid under optional stopping induced by repeated interim analyses. At any common stopping or reporting time, e-BH converts the subgroup e-values into false-discovery-rate-controlled alarms under arbitrary dependence. For sparse labels, predictable audit sampling gives inverse-audited e-processes. In 30 no-shift seeds, no method raises an alarm; this is a sanity check rather than a standalone validation of 5% FDR. At medium and high shift severity, E-FairDrift detects harmful latent subgroup drift in 53% and 77% of streams, respectively, substantially outperforming global monitoring while approaching an oracle monitor given the hidden sensitive groups.

@inproceedings{dwivedi2026efairdrift,
  title     = {E-FairDrift: Anytime-Valid Tests for Intersectional Fairness Drift},
  author    = {Rajeev Ranjan Dwivedi and Vinod K Kurmi},
  booktitle = {ICML Workshop on Hypothesis Testing},
  year      = {2026}
}

Foundation models (FMs) and vision–language models (VLMs) are entering dermatology without clear evidence that their behavior is equitable across skin tones. We present a unified audit of nine recent models on three complementary datasets—ISIC, DDI, and Fitzpatrick17k—covering two clinically distinct tasks: multiclass diagnosis and binary malignancy triage. Beyond pooled accuracy/AUROC, we evaluate fairness at clinical operating points and introduce a threshold-transfer test that quantifies how thresholds tuned on one tone stratum degrade on another. Across models and datasets we observe (i) pronounced task asymmetry (malignancy looks easy in aggregate while diagnosis remains challenging), (ii) nontrivial tone-conditioned gaps in TPR/FNR at fixed specificity, and (iii) tone-wise miscalibration that undermines a single global threshold. Our results argue that claims of “safety” for clinical FMs must be conditioned on subgroup performance and threshold portability, not solely on pooled curves.

@inproceedings{dwivedi2026operating,
  title     = {Operating-Point Fairness in Dermatology Foundation Models: A Multi-Dataset Audit of Skin-Tone Disparities and Threshold Transfer},
  author    = {Rajeev Ranjan Dwivedi and Monika Sharma and Amit Sangroya},
  booktitle = {International Workshop on Health Intelligence (W3PHIAI) at AAAI},
  year      = {2026}
}

Remote sensing (RS) foundation models provide transferable Earth observation representations across sensors, resolutions, and geographies, yet most remain weakly aligned with natural language, limiting natural-language archive search, image-text retrieval, and question-conditioned analysis. We propose AlignJEPA, a JEPA-inspired predictive vision-language alignment framework for remote sensing foundation models. AlignJEPA uses a pretrained AnySat visual encoder and a RemoteCLIP text encoder while training only a lightweight predictive alignment network. Instead of relying on global image–text contrastive alignment alone, the framework predicts remote-sensing text embeddings from masked visual foundation-model tokens. Its mask-aware multi-scale predictive aligner aggregates visible tokens at fine, regional, and global scales, jointly models them with a cross-scale Transformer, and projects the resulting representation into the text space using learned query pooling. Training combines semantic prediction with bidirectional contrastive retrieval. We train and evaluate AlignJEPA on BigEarthNet.txt for natural-language Sentinel retrieval, evaluate cross-dataset adaptation on RSICD, and use RSVQA only as a closed-set representation probe. AlignJEPA provides a parameter-efficient route for aligning Earth observation foundation models with language.

@article{hossain2026alignjepa,
  title   = {AlignJEPA: Predictive Vision-Language Alignment for Remote Sensing Foundation Models},
  author  = {Md Aminur Hossain and Omkumar Vaghasiya and Rajeev Ranjan Dwivedi and Vinod Kurmi and Biplab Banerjee},
  journal = {arXiv preprint arXiv:2608.15456},
  year    = {2026}
}

We introduce a streamlined style‐transfer framework that represents “style” using only the channel‐wise first and second moments (means and variances) of CNN feature maps. Despite its simplicity, this moment‐based descriptor faithfully encodes texture, color, and illumination cues, allowing us to: (i) Perform style transfer by matching means and variances instead of full Gram matrices, reducing computation by over 30 times while producing visually comparable results. (ii) Leverage style for discrimination, showing that these compact statistics alone extracted from deeper layers of both convolutional and transformer models—achieve competitive classification accuracies across benchmarks such as Celeb‑A, CIFAR‑10, and DTD. (iii) Apply style features to bias mitigation, where clustering and aligning style statistics helps identify and correct spurious correlations (eg, background or demographic biases) in datasets like BFFHQ and Cats‑and‑Dogs. Through extensive experiments, we demonstrate that first‐and second‐order style moments form an efficient, interpretable complement to conventional deep features boosting domain adaptation and transfer learning, while offering a novel tool for coarse‐grained bias detection and mitigation.

@misc{dwivedi2025bridging,
  title        = {Bridging Style and Function: Evaluating the Role of Style Features in Vision Models},
  author       = {Rajeev Ranjan Dwivedi and Ankur Kumar and Anshuman Dangwal and Vinod Kumar Kurmi},
  howpublished = {SSRN preprint},
  year         = {2025},
  doi          = {10.2139/ssrn.5599503}
}