Commit d8d0bab6 authored by Shanqing Cai's avatar Shanqing Cai Committed by gunan
Browse files

tfdbg: improve file paths & causality check under suboptimal timestamp resolution

Older versions of Windows (e.g., Windows 7) have lower temporal resolution in timestamps than other platforms. On such Windows machines, multiple dumps of the same or different tensors may end up with the same timestamp, causing
1) failures in the causality check in the constructor of DebugDumpDir,
2) occasional overwriting of previous tensor dumps and core metadata files,
3) loss of ordering information for debug watches of a tensor that is dumped multiple times during a Session::Run() call, e.g., due to While loops.

This CL addresses those two issues:
1) DebugDumpDir._validate_dump_with_graphs now examines tensor dumps of the same timestamp to take into account the possibility of timestamp collision during dependency validation.
2) During the dumping of tensors, file path collisions are checked and if they occur, an index suffix (e.g., "-1", "-2") is appended to the file path.
3) The core metadata Events files now include session_run_count in their names, to avoid collision due to same timestamps.
Change: 150670298
parent bfe56aaa
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment