Infinigen full-object masks

Infinigen full-object masks have one production entry point: pipeline/reconstruction/preprocess/segmentation/sam3_object_masks.py. The lower-level SAM3 session and propagation primitives remain in sam3_masks.py, which is also used independently by human and ground segmentation.

Object prompt policy belongs to the dataset case JSON. Python code must not contain dataset-specific object prompts.

Case configuration

An ordinary case only specifies its primary prompt and uses single_prompt:

"object_mask": {
  "prompt": "<object prompt>"
}

Available strategies are:

• single_prompt: run the primary prompt through the original single-instance SAM3 path.
• reference_subset: select the best candidate subset for the primary prompt.
• prompt_select: select among the primary prompt and alternative_prompts.
• prompt_group_union: independently select one result per prompt_groups entry, then union them.

A difficult case can explicitly configure every prompt sent to SAM3:

"object_mask": {
  "prompt": "<primary prompt>",
  "strategy": "prompt_group_union",
  "prompt_groups": {
    "first_component": ["<candidate 1>", "<candidate 2>"],
    "second_component": ["<candidate 3>", "<candidate 4>"]
  },
  "completion": {
    "enabled": true,
    "fill_convex_hull": true,
    "close_kernel_px": 9,
    "dilate_px": 0,
    "choose_better": true
  }
}

Group names are artifact labels only. The code never converts a group name into a prompt or adds a category fallback. Prompt lists are evaluated in JSON insertion order.

Complex strategies decode the original source video into a temporary lossless frame sequence when it is available. This preserves the image-folder preprocessing path used by SAM3 without a JPEG round trip. Candidate prompts are scored in sessions containing only the lossless prompt frame; only the selected prompt from each group opens the complete sequence for propagation. Temporary sequences are removed after mask generation. single_prompt keeps using the canonical preprocessed frame cache.

Method configuration

Algorithm defaults live under reconstruction.preprocess.segmentation.object_sam3 in configs/methods/recon/infinigen_default.yaml. They contain only implementation policy such as the prompt frame, reference-mask source, scoring weights, subset limits, and whether to save selection artifacts. Per-case prompts and completion overrides remain in the dataset JSON.

Successful runs write the mask sequence plus:

• selection.json
• <prompt_frame>_selection_overlay.png
• reference_mask.png, when the configured reference image exists

To rerun only object masks against an existing preprocess cache:

python scripts/experiments/run_sam3_object_masks.py \
  --runtime-config <runtime/pipeline_config.yaml> \
  --preprocess-dir <recon/preprocess> \
  --output-dir <fresh-output-directory>