Thunder Sessions | Composable transforms to mix FSDP & DDP & nail memory vs communication trade-off

17 views

Lightning AI

1 month ago

Thunder Sessions | Composable transforms to mix FSDP & DDP & nail memory vs communication trade-off

Thunder Sessions | Composable transforms to mix FSDP & DDP & nail memory vs communication trade-off