Enable a soft ordering for collectives which adds dependencies as node attrs.
Before this change, OrderCollectives would add control edges between collective ops to make their execution sequential. This change introduces a technique of encoding the ordering information as a node attribute. In future changes, the collective executor will parse these changes and enforce a fine-grained ordering between collective ops. Additionally, this change also * restricts ordering to CollectiveReduce nodes, * and fixes a bug so that this ordering is invoked from both DirectSession and MasterSession. PiperOrigin-RevId: 228801501
Loading
Please sign in to comment