DORAEMONOpen Dataset Challenge

Open Dataset Challenge

Cross-Detector Transfer

Measure whether learned detector representations transfer across experiments, detector geometries, and event modalities.

Read protocolAll OpenDC
Muon particle tracks in a detector event
CategoryRepresentation learning
Detector typeCross-detector
Technical methodRepresentation learning
TimeframeProtocol owner needed
Dataset typeCross-detector benchmark records
Challenge typeTransfer benchmark
Technical problemCross-detector generalization

Protocol

Scope

This planned challenge will evaluate representation transfer across detector contexts. It is intended for methods that learn broadly useful detector embeddings, including contrastive and self-supervised approaches (Oord et al. 2018).

Protocol needs

The final protocol should specify source datasets, held-out target tasks, and the probe suite used for comparison. A minimal transfer score can average normalized probe performance across target tasks,

\[ S_{\mathrm{transfer}} = \frac{1}{T}\sum_{t=1}^{T} \frac{s_t - s_t^{\mathrm{baseline}}} {s_t^{\mathrm{oracle}} - s_t^{\mathrm{baseline}}}. \]

Leaderboard

The leaderboard is not open until the dataset mix and evaluation tasks are fixed. The protocol should explicitly state which detector family is the source, which target family is held out, and whether fine-tuning is allowed.

References

Oord, Aaron van den, Yazhe Li, and Oriol Vinyals. 2018. “Representation Learning with Contrastive Predictive Coding.” arXiv Preprint arXiv:1807.03748.

Baseline

Baseline solution

Repository location for the minimal reproducible script (MRS) used to reproduce the baseline score for this task.

View baseline source

Validation

Validation

Fetch the evaluation bundle for this dataset, then send an HDF5 submission shaped according to the dataset schema.

Submission schema
opendc eval get multi-source-cross-detector-records
opendc eval send <submission.h5>