pytorch-fsdp

# Pytorch-Fsdp Skill Comprehensive assistance with pytorch-fsdp development, generated from official documentation. ## When to Use This Skill This skill should be triggered when: - Working with pytorch-fsdp - Asking about pytorch-fsdp features or APIs - Implementing pytorch-fsdp solutions - Debugging pytorch-fsdp code - Learning pytorch-fsdp best practices ## Quick Reference ### Common Patterns **Pattern 1:** Generic Join Context Manager# Created On: Jun 06, 2025 | Last Updated On: Jun 06, 2025 The generic join context manager facilitates distributed training on uneven inputs. This page outlines the API of the relevant classes: Join, Joinable, and JoinHook. For a tutorial, see Distributed Training with Uneven Inputs Using the Join Context Manager. class torch.distributed.algorithms.Join(joinables, enable=True, throw_on_early_termination=False, **kwargs)[source]# This class defines the generic join context manager, which allows custom hooks to be called after a process joins. These hooks should shadow the collective communications of non-joined processes to prevent hanging and erroring and to ensure algorithmic correctness. Refer to JoinHook for details about the hook definition. Warning The context manager requires each participating Joinable to call the method notify_join_context() before its own per- iteration collective communications to ensure correctness. Warning The context manager requires that all process_group attributes in the JoinHook objects are the same. If...