selected projects

Systems for crisis response have required several different models for the analysis of unstructured text, such as identifying needs, locations, topics, routing, and matching of needs with available responders. Large Language Models (LLMs) have replaced task-specific models across various language processing tasks. However, LLMs are known to be limited by their training data, collected before the crisis. In this demo, we explore the use of LLMs for crisis response scenarios with rapidly evolving information environments. We show how augmentation of these models with external reliable sources of crisis-specific information can help build adaptive systems for response.

We introduce AutoAC, an agent-computer interface (ACI) for agentic self-refinement in competitive programming that couples LLM code generation with grounded feedback from an online-judge–style execution loop. AutoAC extends a baseline “prompt-solution-judge” workflow with (i) multi-agent chain-of-thought specialization and (ii) automated stress testing. Experiments on 300 problems from the CSES Problem Set across multiple open LLM families show that domain-specific CoT prompting improves partial correctness and that stress-testing yields substantial gains for stronger models; for example, on Qwen2.5-14B-Instruct, adding stress-testing to the best prompt condition increases performance by +5.97% NTPR and +6.33% SR (absolute). Overall, AutoAC demonstrates that embedding competitive-programming debugging practices into an ACI enables measurable improvements in reliable code generation under hidden-test evaluation.

Vision transformer-based (ViT) models have achieved state-of-the-art performance in computer vision tasks by effectively capturing nuanced global relationships between image features. Despite their success, these models are computationally intensive and challenging to deploy on resource-constrained edge devices without dedicated GPUs, such as drones and satellites. This work addresses the optimization of ViT models for enhanced resource efficiency, enabling high-performance inference on low-powered edge devices in a disaster response setting. We focus on Meta’s state-of-the-art Segment Anything Model (SAM) for semantic image segmentation, implementing a series of architectural and hardware-specific modifications to improve its viability for edge deployment. Our optimizations include (1) a faster attention mechanism, (2) post-training weight and activation quantization, and (3) PyTorch tensor operation optimizations. These enhancements result in an over 40% speedup in inference time with minimal degradation in segmentation quality on the SA-1B image dataset.

Scene graphs offer a structured way to represent visual scenes by modelling objects, attributes, and their relationships, greatly benefiting downstream vision-language tasks. Extending this to videos, Panoptic Video Scene Graph Generation (PVSG) aims to capture fine-grained spatial and temporal interactions through pixel-level segmentation and coherent dynamic graphs. However, current methods rely heavily on costly annotations and struggle to generalise to novel concepts. We introduce PanViS, a modular and interpretable framework for zero-shot PVSG that leverages foundation models to recognise unseen objects and relationships without requiring scene graph supervision. PanViS builds temporally consistent scene graphs through four stages: object identification, panoptic segmentation, relationship identification, and relationship verification. By unifying pretrained models via lightweight prompting and refinement, PanViS achieves accurate, scalable, and generalizable scene graph generation. We further propose tailored evaluation metrics to assess scene graph completeness and their utility for downstream tasks like video question answering. Experiments on the STAR dataset would show its effectiveness in addressing challenges of entity disambiguation, segmentation consistency, and relational reasoning across time.

We propose a distributed approach for generating summarized causal explanations of fairness-relevant disparities in aggregate query answers, extending the CauSumX framework to settings where data are partitioned across sites. The method produces explanation patterns that relate outcome differences across subpopulations to high-impact treatments via conditional average treatment effects, enabling interpretable, group-level causal insights. Empirically, the distributed formulation preserves the ability to surface consistent fairness insights across partitions and highlights a key tradeoff between explanation specificity, overall explainability, and coverage of affected groups.