Streamlined representation of PendingCounts data structure.
(1) We now allocate a contiguous array of memory to hold count information for each node in a frame, allocating either 8 bits to hold a PendingCounts::PackedCounts structure (if the dead and pending count fields can fit in 8 bits), or an aligned region of 64 bits to hold a PendingCounts::LargeCounts structure (if the counts are too large to fit in the 8-bit representation). The byte offset and representation of the data for a given node are now held in a 32-bit handle. This makes all the count data contiguous for a given frame (better cache locality), and avoids the need for the FlatMap that we were using to hold a mapping from id to LargeCounts for those nodes that needed the overflow space. (2) Sped up the fast path of adjustments needed when normal non-merge nodes complete by adding a specialized "adjust_for_activation" routine that does all the necessary bookkeeping in one routine, rather than having to manipulate the count data structure several times. (3) Several tensorflow/core/common_runtime/executor.cc changes related to this: . Stored the new PendingCounts::Handle for a given node in the ItemState data structure, rather than in a separate frame_local_ids_ array. This means that we touch one less cache line per edge in ActivateNodes. . Changed initialization code to use the new PendingCounts::AllocateHandle routine for allocating space to hold the counts for a node. This allowed simplifying the ControlFlowInfo that is computed at initialization (we just need the unique set of frame names now, rather than a mapping from frame name to number of nodes in that frame). (4) Updated tensorflow/core/common_runtime/pending_counts_test.cc to reflect the new interface and to test the new compound adjust_for_activation routine. Speeds up an image processing benchmark on my desktop from 3370 images/sec to 3540 images/sec (+5.0%). Change: 146684811
Loading
Please sign in to comment