nemo-mbridge-perf-tp-dp-comm-overlaplisted

# TP / DP / PP Communication Overlap Skill For stable background and recommendation level, see: - @docs/training/communication-overlap.md ## Enablement Minimal Bridge override: ```python from megatron.bridge.training.comm_overlap import CommOverlapConfig cfg.model.tensor_model_parallel_size = 4 cfg.model.sequence_parallel = True cfg.model.pipeline_model_parallel_size = 4 cfg.model.virtual_pipeline_model_parallel_size = 2 cfg.comm_overlap = CommOverlapConfig( tp_comm_overlap=True, ) cfg.ddp.use_distributed_optimizer = True cfg.ddp.overlap_grad_reduce = True cfg.ddp.overlap_param_gather = True ``` Optional TP preset: ```python from megatron.bridge.training.comm_overlap import userbuffers_bf16_h100_h12288_tp4_mbs1_seqlen2048 cfg.comm_overlap.tp_comm_overlap_cfg = userbuffers_bf16_h100_h12288_tp4_mbs1_seqlen2048 ``` Precision knobs belong to mixed precision: ```python cfg.mixed_precision.grad_reduce_in_fp32 = False cfg.mixed_precision.fp8_param_gather = False ``` ## Code Anchors Bridge overlap gating: ```439:449:src/megatron/bridge/training/comm_overlap.py if self.user_comm_overlap_cfg.tp_comm_overlap is True: if model_cfg.tensor_model_parallel_size < 2: ... elif not model_cfg.sequence_parallel: ... elif not HAVE_TE: ... ``` PP overlap selection: ```451:458:src/megatron/bridge/training/comm_overlap.py if model_cfg.pipeline_model_parallel_size > 1: if vp_size > 1: comm_overlap_cfg.overlap_p2p_comm = True