Building-Damage Segmentation

Context

After a hurricane, responders need to know which buildings are damaged, fast. Trained on RescueNet (UAV imagery of hurricane damage), the task is 11-class semantic segmentation — water, road, trees, vehicles, pools, and tiers of building damage.

Problem

Real damage maps are brutally imbalanced: a scene is mostly background and intact structures, with the rare-but-critical classes (damaged vehicles, pools, severe damage) covering a tiny fraction of pixels. A naive model just predicts the majority classes and scores well on paper while being useless.

Approach

• Custom attention U-Net family. I synthesized ideas from the CV literature — CBAM channel+spatial attention, DeepLab-style ASPP for multi-scale context, attention gates on the skip connections, and deep-supervision fusion across decoder scales.
• Imbalance handling. A compound loss (weighted cross-entropy + Focal + Dice) plus ENet-style class weighting, so the rare damage classes actually contribute to the gradient.
• Production-minded inference. Tiled, Gaussian-blended overlapping-patch inference so large aerial images can be segmented without edge artifacts, with mixed precision and gradient clipping in training.

Results & what I learned

• Across multiple 200-epoch runs: ~0.40–0.45 validation mIoU, ~0.42 test mIoU on the full 11-class set, with per-class IoU tracked every epoch.
• As expected, the rare classes (e.g. pools) lag the large ones — an honest, visible signature of the imbalance problem rather than a hidden one.
• Learned: for imbalanced segmentation, the loss function and class weighting matter as much as the architecture; and tracking per-class metrics (not just mean) is what keeps you honest about where the model actually fails.