axolotl

# Axolotl Skill Comprehensive assistance with axolotl development, generated from official documentation. ## When to Use This Skill This skill should be triggered when: - Working with axolotl - Asking about axolotl features or APIs - Implementing axolotl solutions - Debugging axolotl code - Learning axolotl best practices ## Quick Reference ### Common Patterns **Pattern 1:** To validate that acceptable data transfer speeds exist for your training job, running NCCL Tests can help pinpoint bottlenecks, for example: ``` ./build/all_reduce_perf -b 8 -e 128M -f 2 -g 3 ``` **Pattern 2:** Configure your model to use FSDP in the Axolotl yaml. For example: ``` fsdp_version: 2 fsdp_config: offload_params: true state_dict_type: FULL_STATE_DICT auto_wrap_policy: TRANSFORMER_BASED_WRAP transformer_layer_cls_to_wrap: LlamaDecoderLayer reshard_after_forward: true ``` **Pattern 3:** The context_parallel_size should be a divisor of the total number of GPUs. For example: ``` context_parallel_size ``` **Pattern 4:** For example: - With 8 GPUs and no sequence parallelism: 8 different batches processed per step - With 8 GPUs and context_parallel_size=4: Only 2 different batches processed per step (each split across 4 GPUs) - If your per-GPU micro_batch_size is 2, the global batch size decreases from 16 to 4 ``` context_parallel_size=4 ``` **Pattern 5:** Setting save_compressed: true in your configuration enables saving models in a compressed format, which: - Reduces disk s...