David EtterPh.D.

Thousands of public livestream cameras run continuously around the world. Most of what they capture is static, or shows the typical activity for that place and time of day, and there is far too much of it for a person to watch. This work defines an anomaly as an event that is rare in a camera's own history, and asks a system to rank each camera's clips by how likely they are to be anomalous, with no query and no predefined list of anomaly types.

MultiVENT-Raw Anomaly is a benchmark for this open-set task, built from livestream cameras captured globally between June 2025 and April 2026. Evaluations with both vision-feature and vision-language model approaches show the task remains hard for current models.

livestream cameras

34

video clips

5,859

human-annotated anomalies

76

Most video retrieval benchmarks match short visual descriptions against small collections of edited, English-language clips. The MultiVENT collections are built around real-world events in many languages, where the evidence may sit in the picture, the audio, on-screen text, or the metadata.

• arXiv 2026
  MultiVENT-Raw: A Benchmark for Retrieval and Reasoning over Raw Videos

  Nearly 120,000 mostly raw videos (over 5,300 hours) from phones, hand-held cameras, and CCTV, paired with 130 events and 222 event-centric queries. Supports both retrieval and report generation over footage that has no script, captions, or graphics to lean on.

• CVPR 2025
  MultiVENT 2.0: A Massive Multilingual Benchmark for Event-Centric Video Retrieval

  More than 218,000 news videos and 3,906 queries targeting specific world events. State-of-the-art vision-language models struggle on it.

• SIGIR 2025
  MMMORRF: Multimodal Multilingual Modularized Reciprocal Rank Fusion

  A video search system that extracts text and features from both the visual and audio channels and combines them with modality-aware reciprocal rank fusion. Improves nDCG@20 by 81% over leading multimodal encoders and 37% over single-modality retrieval.

• EMNLP Findings 2024
  Grounding Partially-Defined Events in Multimodal Data

  Introduces MultiVENT-G: 14.5 hours of densely annotated current-event video and 1,168 text documents, with 22.8K labeled event-centric entities.

• NeurIPS 2023
  MultiVENT: Multilingual Videos of Events and Aligned Natural Text

  About 2,400 event-centric videos grounded in text documents across five languages (Arabic, Chinese, English, Korean, and Russian), mixing news broadcasts with first-hand footage.

Text recognition that works across scripts, and models that read text as pixels instead of tokens.

• ICDAR 2023 IAPR Best Paper
  A Hybrid Model for Multilingual OCR

  A transformer encoder-decoder that trains the encoder with a CTC objective and the decoder with cross-entropy. The fast encoder can run on its own, or with the full autoregressive decoder when accuracy matters most. Evaluated on the multilingual CAMIO dataset.

• EMNLP 2021
  Robust Open-Vocabulary Translation from Visual Text Representations

  Replaces a translation model's subword vocabulary with images of rendered text read through sliding windows. On character-permuted German to English it reaches 25.9 BLEU where subword models fall to 1.9.

• ICDAR 2019
  A Synthetic Recipe for OCR

  One of four ICDAR 2019 papers, alongside work on speech recognition methods for OCR, script identification, and Chinese and Korean character decomposition.

Deep learning models for fast, scalable, and accurate face recognition across a wide range of camera capture settings. The work covers training data and techniques that reduce model bias across gender and ethnicity, metric-learning loss functions, score calibration, and evaluation on the open-set 1:N protocol of the IARPA Janus IJB-C benchmark. More recent models use Vision Transformers and masked autoencoder pretraining.

• ICASSP 2018
  Audio-Visual Person Recognition in Multimedia Data from the IARPA Janus Program

  Combining face and speaker recognition to identify people in video, the subject of the 2017 SCALE workshop.