From 4e69e02241067129379f73dd4fefe57f0a12bdc9 Mon Sep 17 00:00:00 2001 From: Martin Wicke Date: Tue, 7 Nov 2017 11:35:23 -0800 Subject: [PATCH] Branch 174861804 (#14326) * Add ImportGraphDefTest.testMultipleImport to importer_test.py This tests the name deduping behavior of import_graph_def. This behavior is actually defined by the op creation logic, not import_graph_def, but I added a test here since the C++ ImportGraphDef function must emulate it (and presumably we'd like to maintain the import_graph_def behavior moving forward). PiperOrigin-RevId: 174536014 * Apply lib_internal defines to both lib_internal and lib_internal_impl Should fix checkpoint reading with snappy compression. Will follow up with testing for this sort of checkpoint issue. PiperOrigin-RevId: 174538693 * n/a (internal change only) PiperOrigin-RevId: 174539513 * A few changes to ApiDef generation: - Create a separate api_def_*.pbtxt file for each op. - Add attribute and argument descriptions to ApiDef. - Apply overrides based on op_gen_overrides.pbtxt file. PiperOrigin-RevId: 174540421 * Add uniquify_names option to ImportGraphDef. This option allows ImportGraphDef to mimic the behavior of the Python import_graph_def function, which automatically creates unique node names instead of raising an exception (this is due to the Python op construction logic, not import_graph_def directly). This change is a steps towards switching import_graph_def to use the C API version. PiperOrigin-RevId: 174541334 * Fix bad_color param on tf.contrib.summary.image PiperOrigin-RevId: 174549117 * Hlo parser: support control-predecessors. Also, - Changed from printing control-sucessors to printing control-predecessors because predecessors are defined before use. - Surround the predecessors with {}. PiperOrigin-RevId: 174552224 * Support pad node. PiperOrigin-RevId: 174581035 * Add tf.contrib.framework.sort, wrapping tf.nn.top_k (#288). Comparable to np.sort, but their "kind" parameter is not implemented (only one sort algorithm) and "order" is not applicable (tensors do not have fields). PiperOrigin-RevId: 174588000 * [TF2XLA] Don't change output port for control dependency in CopySubgraph. If the output is being squashed then we want control output 0, except where the input is a control dependency. PiperOrigin-RevId: 174633829 * Use latest nsync: allows running bazel after having downloaded for "make" build The downloads directory for the make build is within the source tree seen by bazel, which means that BUILD files (by whatever name) without those downloaded trees must all be valid in their new location, or not recognized by bazel as being BUILD files. The new version of nsync handles that, and this change pulls in that new version. PiperOrigin-RevId: 174652898 * Add profiling support to Service::ExecuteParallel. PiperOrigin-RevId: 174682772 * Replicate `Estimator.model_fn` across available GPUs. def replicate_model_fn(model_fn, optimizer_fn, devices=None): """Replicate `Estimator.model_fn` over GPUs. ... I tested that it seems to give the right result on cnn_mnist.py on 1 CPU, 1 real GPU, 4 allow_soft_placement=True GPUs. Some measurements on CNN MNIST across steps 19300-20000: 1) no replicate_model_fn call: global_step/sec: 156.254 global_step/sec: 155.074 global_step/sec: 155.74 global_step/sec: 153.636 global_step/sec: 157.218 global_step/sec: 159.644 2) replicate across one hardware GPU: global_step/sec: 158.171 global_step/sec: 165.618 global_step/sec: 162.773 global_step/sec: 159.204 global_step/sec: 162.289 global_step/sec: 167.173 3) replicate across 4 software GPUs on one hardware GPU (soft placement): global_step/sec: 75.47 global_step/sec: 76.16 global_step/sec: 75.18 Loss numbers didn't change across the three configurations. PiperOrigin-RevId: 174704385 * Enables wrapping input pipeline into tf.while_loop for all users. PiperOrigin-RevId: 174708213 * SerializeIterator: do not unref the resource until we're finished using it. This change avoids a potential use-after-free error if the resource is concurrently serialized and destroyed (e.g. by a DestroyResourceOp or Session::Reset()). PiperOrigin-RevId: 174713115 * Improve error message when a function is already defined with the same name and different hash string. PiperOrigin-RevId: 174715563 * Fix generate_examples build - Add -march=native to host_copts and host_cxxopts in configure.py - Make string.h for abstracting string differences at core interpreter level - Use tensorflow special arg parse instead of flags - Switch to using tool instead of data for dependency - Fix python3 compatibility + Use six.StringIO instead of StringIO.StringIO + Use print_function + Properly set binary flags on TempFile's used in toco_convert - Misc other path fixes PiperOrigin-RevId: 174717673 * Add input format agnostic way to parse HLOs. PiperOrigin-RevId: 174719153 * Remove misleading comment from Eigen build file. PiperOrigin-RevId: 174719222 * Basic plumbing for calling C API from import_graph_def() PiperOrigin-RevId: 174724070 * Memory leak detected when running a heap checker in our tests. PiperOrigin-RevId: 174726228 * [tpu:profiler] Support the Input Pipeline Analyzer tool in TPU profiler (WIP) o. move input pipeline analyzer related proto for grpc between red and green VMs o. rename perftools.gputools.profiler.collector::TfStatsHelperResult to tensorflow::tpu::TfOpStats. PiperOrigin-RevId: 174730411 * Clean up some reference cycles in eager mode. ResourceVariables enter graph mode to get a handle. We should probably revisit that, but in the meantime we can break the resulting reference cycles. PiperOrigin-RevId: 174732964 * Improved encoding on shapes in grappler. PiperOrigin-RevId: 174733491 * [tf.data] Remove unused members from IteratorContext. PiperOrigin-RevId: 174734277 * Refactor helper functions a bit for virtual gpu changes later. PiperOrigin-RevId: 174735029 * Fix invalid flush_secs argument. PiperOrigin-RevId: 174745329 * Replace the implementation of tf.flags with absl.flags. Previous tf.flags implementation is based on argparse. It contains -h/--help flags, which displays all flags. absl.app's --help flag only displays flags defined in the main module. There is a --helpfull flag that displays all flags. So added --helpshort --helpfull flags. app.run now raises SystemError on unknown flags (fixes #11195). Accessing flags before flags are parsed will now raise an UnparsedFlagAccessError, instead of causing implicit flag parsing previously. PiperOrigin-RevId: 174747028 * Fold Transpose into Matmul and SparseMatmul. Fold ConjugateTranspose in BatchMatmul. PiperOrigin-RevId: 174750173 * BUGFIX: special_math.ndtri didn't work with dynamic shapes. This was due to use of constant_op.constant(..., shape=p.shape), where sometimes p was a Tensor of unknown shape. PiperOrigin-RevId: 174764744 * Create a routine that can collapse a subgraph into a fused op PiperOrigin-RevId: 174765540 * Force CUDA runtime initialization only when device count is larger than 0. PiperOrigin-RevId: 174767565 * Remove use of xrange which is not python3 compatible. PiperOrigin-RevId: 174768741 * More thoroughly disable the should_use_result decorator when executing eagerly. It was creating reference cycles. Adds a test that TensorArrays create no reference cycles in eager mode. PiperOrigin-RevId: 174768765 * Fix device querying in Keras backend. PiperOrigin-RevId: 174769308 * Fix race bug in AdaptiveSharedBatchScheduler. In ASBSQueue::Schedule, when a new batch is created, it was added to the scheduler outside of the queue's lock. This was done to prevent any unforeseen interactions between the queue lock and scheduler lock. However, this wasn't being done in a thread safe way. PiperOrigin-RevId: 174769383 * Supports multi-dimensional logits and labels in multi class head. PiperOrigin-RevId: 174770444 * Refactor eager benchmarks to subclass Benchmark. PiperOrigin-RevId: 174770787 * Add `parallel_interleave` to tf/contrib/data/__init__.py so that it is directly addressable from tf.contrib.data. PiperOrigin-RevId: 174771870 * Fix DepthToSpaceGrad and SpaceToDepthGrad on data_format NCHW. This fixes #14243. PiperOrigin-RevId: 174772870 * Allow for an old_row_vocab_size, in case a subset of the old_row_vocab_file was used during the checkpoint creation (as is allowed in FeatureColumn._VocabularyListCategoricalColumn). PiperOrigin-RevId: 174781749 * Go: Update generated wrapper functions for TensorFlow ops. PiperOrigin-RevId: 174781987 * [BufferAssignment] Sort allocation's "Assigned" objects before converting to a proto. This makes the buffer assignment's proto dump deterministic. RELNOTES: BufferAssignment's protocol buffer dump is now deterministic. PiperOrigin-RevId: 174783549 * [TF TensorArray] allow reading from an unwritten index if fully defined element_shape is given. This allows one to write to only some indices of a TensorArray before calling stack. Elements that were not written to are treated as all zero tensors. PiperOrigin-RevId: 174783569 * Remove binary dependency from optimize_for_inference_lib PiperOrigin-RevId: 174787363 * Update ops-related pbtxt files. PiperOrigin-RevId: 174787397 * Automated g4 rollback of changelist 174523638 PiperOrigin-RevId: 174788331 * Skip non-existent fetch nodes PiperOrigin-RevId: 174795864 * Automated g4 rollback of changelist 174735029 PiperOrigin-RevId: 174796480 * Add InceptionResNetV2 to tf.keras and update applications module to match Keras 2.0.9. PiperOrigin-RevId: 174796893 * Fix for LLVM API changes for fast math (https://reviews.llvm.org/rL317488). PiperOrigin-RevId: 174799735 * [TF:XLA] Add two disabled tests with while ops that permute tuple elements. These tests permute the tuple elements of a 3-tuple in each iteration in the following cyclic manner (132), i.e. a shift to the left. The first test just return the result tuple, the second returns the sum of all tuple elements (which is expected to be constant 6, no matter which permutation) Both tests are disabled for now because they fail on all back-ends. PiperOrigin-RevId: 174806092 * Refactor function Optimize. PiperOrigin-RevId: 174813300 * Add a unit test for gradient computation with layout optimizer. PiperOrigin-RevId: 174814136 * Previously if ComputeConstant seen a parameter it failed to proceed. After this change we can specify a list of parameters to it and if we specify enough then it will do the computation. The primary goal of this change is to make the HloEvaluator usable with ComputationBuilder from tests through ComputeConstant in cases where the input is a parameter (fed by a literal). PiperOrigin-RevId: 174845108 * Use nesting to reduce the number of modules listed in the API TOC. PiperOrigin-RevId: 174846842 * Added CPU matrix exponential op to TensorFlow. Uses Eigen's unsupported implementation. PiperOrigin-RevId: 174858966 * variables_to_restore: Differentiate python variables by string name rather than object. variables_to_restore ensured that duplicate variables weren't added to the return map by comparing python variable object. Normally there is only one Variable object for each underlying variable, so this wasn't a problem. But when one initializes a graph by importing a GraphDef, duplicate python Variable objects are created for each occurrence of a variable in a collection (say, global variables and moving average variables). This change fixes variables_to_restore to work with an imported graph def by not comparing Variable objects. PiperOrigin-RevId: 174861804 --- configure.py | 7 +- .../tf2xla/functionalize_control_flow.cc | 4 +- .../compiler/tf2xla/kernels/gather_op.cc | 12 - .../xla/client/computation_builder.cc | 9 +- .../compiler/xla/client/computation_builder.h | 23 +- .../compiler/xla/service/buffer_assignment.cc | 5 + .../xla/service/cpu/llvm_ir_runtime.cc | 2 +- tensorflow/compiler/xla/service/executable.h | 10 + .../compiler/xla/service/hlo_instruction.cc | 13 +- tensorflow/compiler/xla/service/hlo_runner.cc | 31 +- tensorflow/compiler/xla/service/hlo_runner.h | 16 +- .../compiler/xla/service/llvm_ir/llvm_util.cc | 5 +- tensorflow/compiler/xla/service/service.cc | 116 +- tensorflow/compiler/xla/service/service.h | 3 +- .../compiler/xla/service/user_computation.cc | 267 +- .../compiler/xla/service/user_computation.h | 8 +- .../xla/tests/compute_constant_test.cc | 45 +- tensorflow/compiler/xla/tests/while_test.cc | 105 + .../compiler/xla/tools/parser/hlo_parser.cc | 71 +- .../xla/tools/parser/hlo_parser_test.cc | 2 +- tensorflow/compiler/xla/xla.proto | 2 + .../adaptive_shared_batch_scheduler.h | 7 +- .../python/training/functions/gbdt_batch.py | 2 +- tensorflow/contrib/cmake/tf_python.cmake | 1 + tensorflow/contrib/data/__init__.py | 2 + .../eager/python/examples/mnist/mnist.py | 4 +- .../examples/rnn_colorbot/rnn_colorbot.py | 4 +- tensorflow/contrib/estimator/BUILD | 64 +- .../python/estimator/replicate_model_fn.py | 470 +++ .../estimator/replicate_model_fn_test.py | 901 ++++++ tensorflow/contrib/framework/BUILD | 27 + tensorflow/contrib/framework/__init__.py | 2 + .../framework/python/framework/__init__.py | 1 + .../framework/python/framework/graph_util.py | 128 + .../python/framework/graph_util_test.py | 61 + .../contrib/framework/python/ops/__init__.py | 1 + .../contrib/framework/python/ops/sort_ops.py | 113 + .../framework/python/ops/sort_ops_test.py | 95 + tensorflow/contrib/summary/summary_ops.py | 4 +- tensorflow/contrib/tpu/profiler/BUILD | 7 + .../contrib/tpu/profiler/tf_op_stats.proto | 127 + .../contrib/tpu/python/tpu/tpu_estimator.py | 2 +- tensorflow/core/BUILD | 14 +- tensorflow/core/api_def/api_test.cc | 288 +- .../core/api_def/base_api/api_def_A.pbtxt | 670 ----- .../core/api_def/base_api/api_def_Abort.pbtxt | 16 + .../core/api_def/base_api/api_def_Abs.pbtxt | 9 + .../base_api/api_def_AccumulateNV2.pbtxt | 26 + .../api_def_AccumulatorApplyGradient.pbtxt | 32 + .../api_def_AccumulatorNumAccumulated.pbtxt | 16 + .../api_def_AccumulatorSetGlobalStep.pbtxt | 20 + .../api_def_AccumulatorTakeGradient.pbtxt | 36 + .../core/api_def/base_api/api_def_Acos.pbtxt | 4 + .../core/api_def/base_api/api_def_Acosh.pbtxt | 4 + .../core/api_def/base_api/api_def_Add.pbtxt | 8 + .../api_def_AddManySparseToTensorsMap.pbtxt | 68 + .../core/api_def/base_api/api_def_AddN.pbtxt | 10 + .../api_def_AddSparseToTensorsMap.pbtxt | 58 + .../core/api_def/base_api/api_def_AddV2.pbtxt | 8 + .../base_api/api_def_AdjustContrast.pbtxt | 4 + .../base_api/api_def_AdjustContrastv2.pbtxt | 36 + .../api_def/base_api/api_def_AdjustHue.pbtxt | 30 + .../base_api/api_def_AdjustSaturation.pbtxt | 30 + .../core/api_def/base_api/api_def_All.pbtxt | 42 + .../api_def_AllCandidateSampler.pbtxt | 80 + .../core/api_def/base_api/api_def_Angle.pbtxt | 23 + .../core/api_def/base_api/api_def_Any.pbtxt | 42 + .../base_api/api_def_ApplyAdadelta.pbtxt | 65 + .../base_api/api_def_ApplyAdagrad.pbtxt | 46 + .../base_api/api_def_ApplyAdagradDA.pbtxt | 65 + .../api_def/base_api/api_def_ApplyAdam.pbtxt | 90 + .../api_def_ApplyCenteredRMSProp.pbtxt | 86 + .../api_def/base_api/api_def_ApplyFtrl.pbtxt | 73 + .../base_api/api_def_ApplyFtrlV2.pbtxt | 75 + .../api_def_ApplyGradientDescent.pbtxt | 35 + .../base_api/api_def_ApplyMomentum.pbtxt | 62 + .../api_def_ApplyProximalAdagrad.pbtxt | 58 + ...api_def_ApplyProximalGradientDescent.pbtxt | 51 + .../base_api/api_def_ApplyRMSProp.pbtxt | 72 + .../base_api/api_def_ApproximateEqual.pbtxt | 4 + .../api_def/base_api/api_def_ArgMax.pbtxt | 15 + .../api_def/base_api/api_def_ArgMin.pbtxt | 15 + .../api_def/base_api/api_def_AsString.pbtxt | 42 + .../core/api_def/base_api/api_def_Asin.pbtxt | 4 + .../core/api_def/base_api/api_def_Asinh.pbtxt | 4 + .../api_def/base_api/api_def_Assert.pbtxt | 26 + .../api_def/base_api/api_def_Assign.pbtxt | 42 + .../api_def/base_api/api_def_AssignAdd.pbtxt | 34 + .../api_def_AssignAddVariableOp.pbtxt | 29 + .../api_def/base_api/api_def_AssignSub.pbtxt | 34 + .../api_def_AssignSubVariableOp.pbtxt | 29 + .../base_api/api_def_AssignVariableOp.pbtxt | 26 + .../core/api_def/base_api/api_def_Atan.pbtxt | 4 + .../core/api_def/base_api/api_def_Atan2.pbtxt | 11 + .../core/api_def/base_api/api_def_Atanh.pbtxt | 4 + .../base_api/api_def_AudioSpectrogram.pbtxt | 63 + .../base_api/api_def_AudioSummary.pbtxt | 47 + .../base_api/api_def_AudioSummaryV2.pbtxt | 50 + .../api_def/base_api/api_def_AvgPool.pbtxt | 48 + .../api_def/base_api/api_def_AvgPool3D.pbtxt | 46 + .../base_api/api_def_AvgPool3DGrad.pbtxt | 52 + .../base_api/api_def_AvgPoolGrad.pbtxt | 52 + .../core/api_def/base_api/api_def_B.pbtxt | 448 --- .../api_def/base_api/api_def_Barrier.pbtxt | 55 + .../base_api/api_def_BarrierClose.pbtxt | 26 + .../api_def_BarrierIncompleteSize.pbtxt | 17 + .../base_api/api_def_BarrierInsertMany.pbtxt | 35 + .../base_api/api_def_BarrierReadySize.pbtxt | 17 + .../base_api/api_def_BarrierTakeMany.pbtxt | 68 + .../base_api/api_def_BatchCholesky.pbtxt | 3 + .../base_api/api_def_BatchCholeskyGrad.pbtxt | 3 + .../base_api/api_def_BatchDataset.pbtxt | 11 + .../api_def/base_api/api_def_BatchFFT.pbtxt | 3 + .../api_def/base_api/api_def_BatchFFT2D.pbtxt | 3 + .../api_def/base_api/api_def_BatchFFT3D.pbtxt | 3 + .../api_def/base_api/api_def_BatchIFFT.pbtxt | 3 + .../base_api/api_def_BatchIFFT2D.pbtxt | 3 + .../base_api/api_def_BatchIFFT3D.pbtxt | 3 + .../base_api/api_def_BatchMatMul.pbtxt | 54 + .../api_def_BatchMatrixBandPart.pbtxt | 3 + .../api_def_BatchMatrixDeterminant.pbtxt | 3 + .../base_api/api_def_BatchMatrixDiag.pbtxt | 3 + .../api_def_BatchMatrixDiagPart.pbtxt | 3 + .../base_api/api_def_BatchMatrixInverse.pbtxt | 3 + .../base_api/api_def_BatchMatrixSetDiag.pbtxt | 3 + .../base_api/api_def_BatchMatrixSolve.pbtxt | 3 + .../base_api/api_def_BatchMatrixSolveLs.pbtxt | 3 + .../api_def_BatchMatrixTriangularSolve.pbtxt | 3 + ...def_BatchNormWithGlobalNormalization.pbtxt | 57 + ...BatchNormWithGlobalNormalizationGrad.pbtxt | 86 + .../api_def_BatchSelfAdjointEig.pbtxt | 3 + .../api_def_BatchSelfAdjointEigV2.pbtxt | 3 + .../api_def/base_api/api_def_BatchSvd.pbtxt | 3 + .../base_api/api_def_BatchToSpace.pbtxt | 104 + .../base_api/api_def_BatchToSpaceND.pbtxt | 139 + .../api_def/base_api/api_def_Betainc.pbtxt | 19 + .../api_def/base_api/api_def_BiasAdd.pbtxt | 38 + .../base_api/api_def_BiasAddGrad.pbtxt | 33 + .../api_def/base_api/api_def_BiasAddV1.pbtxt | 29 + .../api_def/base_api/api_def_Bincount.pbtxt | 40 + .../api_def/base_api/api_def_Bitcast.pbtxt | 18 + .../api_def/base_api/api_def_BitwiseAnd.pbtxt | 8 + .../api_def/base_api/api_def_BitwiseOr.pbtxt | 8 + .../api_def/base_api/api_def_BitwiseXor.pbtxt | 8 + .../base_api/api_def_BroadcastArgs.pbtxt | 11 + .../api_def_BroadcastGradientArgs.pbtxt | 8 + .../api_def/base_api/api_def_Bucketize.pbtxt | 38 + .../core/api_def/base_api/api_def_C.pbtxt | 513 ---- .../api_def_CTCBeamSearchDecoder.pbtxt | 72 + .../base_api/api_def_CTCGreedyDecoder.pbtxt | 61 + .../api_def/base_api/api_def_CTCLoss.pbtxt | 70 + .../base_api/api_def_CacheDataset.pbtxt | 17 + .../core/api_def/base_api/api_def_Cast.pbtxt | 4 + .../core/api_def/base_api/api_def_Ceil.pbtxt | 4 + .../base_api/api_def_CheckNumerics.pbtxt | 14 + .../api_def/base_api/api_def_Cholesky.pbtxt | 31 + .../base_api/api_def_CholeskyGrad.pbtxt | 30 + .../base_api/api_def_CompareAndBitpack.pbtxt | 53 + .../api_def/base_api/api_def_Complex.pbtxt | 20 + .../api_def/base_api/api_def_ComplexAbs.pbtxt | 10 + .../api_def_ComputeAccidentalHits.pbtxt | 62 + .../api_def/base_api/api_def_Concat.pbtxt | 27 + .../base_api/api_def_ConcatOffset.pbtxt | 36 + .../api_def/base_api/api_def_ConcatV2.pbtxt | 29 + .../base_api/api_def_ConcatenateDataset.pbtxt | 4 + .../api_def_ConditionalAccumulator.pbtxt | 44 + .../core/api_def/base_api/api_def_Conj.pbtxt | 19 + .../base_api/api_def_ConjugateTranspose.pbtxt | 9 + .../core/api_def/base_api/api_def_Const.pbtxt | 10 + .../base_api/api_def_ControlTrigger.pbtxt | 7 + .../api_def/base_api/api_def_Conv2D.pbtxt | 72 + .../api_def_Conv2DBackpropFilter.pbtxt | 57 + .../api_def_Conv2DBackpropInput.pbtxt | 56 + .../api_def/base_api/api_def_Conv3D.pbtxt | 47 + .../api_def_Conv3DBackpropFilter.pbtxt | 37 + .../api_def_Conv3DBackpropFilterV2.pbtxt | 49 + .../api_def_Conv3DBackpropInput.pbtxt | 37 + .../api_def_Conv3DBackpropInputV2.pbtxt | 49 + .../core/api_def/base_api/api_def_Cos.pbtxt | 4 + .../core/api_def/base_api/api_def_Cosh.pbtxt | 4 + .../api_def/base_api/api_def_CountUpTo.pbtxt | 24 + .../base_api/api_def_CropAndResize.pbtxt | 74 + .../api_def_CropAndResizeGradBoxes.pbtxt | 52 + .../api_def_CropAndResizeGradImage.pbtxt | 53 + .../core/api_def/base_api/api_def_Cross.pbtxt | 27 + .../api_def/base_api/api_def_Cumprod.pbtxt | 61 + .../api_def/base_api/api_def_Cumsum.pbtxt | 61 + .../core/api_def/base_api/api_def_D.pbtxt | 790 ----- .../api_def_DatasetToSingleElement.pbtxt | 16 + .../api_def_DebugGradientIdentity.pbtxt | 8 + .../base_api/api_def_DecodeAndCropJpeg.pbtxt | 86 + .../base_api/api_def_DecodeBase64.pbtxt | 20 + .../api_def/base_api/api_def_DecodeBmp.pbtxt | 26 + .../api_def/base_api/api_def_DecodeCSV.pbtxt | 49 + .../api_def/base_api/api_def_DecodeGif.pbtxt | 25 + .../base_api/api_def_DecodeJSONExample.pbtxt | 26 + .../api_def/base_api/api_def_DecodeJpeg.pbtxt | 80 + .../api_def/base_api/api_def_DecodePng.pbtxt | 39 + .../api_def/base_api/api_def_DecodeRaw.pbtxt | 26 + .../api_def/base_api/api_def_DecodeWav.pbtxt | 50 + .../api_def_DeleteSessionTensor.pbtxt | 10 + .../api_def_DenseToDenseSetOperation.pbtxt | 47 + .../api_def_DenseToSparseBatchDataset.pbtxt | 25 + .../api_def_DenseToSparseSetOperation.pbtxt | 70 + .../base_api/api_def_DepthToSpace.pbtxt | 101 + .../api_def_DepthwiseConv2dNative.pbtxt | 47 + ..._DepthwiseConv2dNativeBackpropFilter.pbtxt | 60 + ...f_DepthwiseConv2dNativeBackpropInput.pbtxt | 60 + .../api_def/base_api/api_def_Dequantize.pbtxt | 91 + .../api_def_DeserializeIterator.pbtxt | 17 + .../api_def_DeserializeManySparse.pbtxt | 60 + .../base_api/api_def_DestroyResourceOp.pbtxt | 21 + .../api_def_DestroyTemporaryVariable.pbtxt | 26 + .../core/api_def/base_api/api_def_Diag.pbtxt | 29 + .../api_def/base_api/api_def_DiagPart.pbtxt | 36 + .../api_def/base_api/api_def_Digamma.pbtxt | 7 + .../api_def/base_api/api_def_Dilation2D.pbtxt | 67 + .../api_def_Dilation2DBackpropFilter.pbtxt | 48 + .../api_def_Dilation2DBackpropInput.pbtxt | 48 + .../core/api_def/base_api/api_def_Div.pbtxt | 8 + .../base_api/api_def_DrawBoundingBoxes.pbtxt | 37 + .../base_api/api_def_DynamicPartition.pbtxt | 55 + .../base_api/api_def_DynamicStitch.pbtxt | 68 + .../core/api_def/base_api/api_def_E.pbtxt | 261 -- .../base_api/api_def_EditDistance.pbtxt | 96 + .../core/api_def/base_api/api_def_Elu.pbtxt | 8 + .../api_def/base_api/api_def_EluGrad.pbtxt | 24 + .../base_api/api_def_EncodeBase64.pbtxt | 30 + .../api_def/base_api/api_def_EncodeJpeg.pbtxt | 89 + .../api_def/base_api/api_def_EncodePng.pbtxt | 35 + .../api_def/base_api/api_def_EncodeWav.pbtxt | 31 + .../core/api_def/base_api/api_def_Enter.pbtxt | 42 + .../core/api_def/base_api/api_def_Equal.pbtxt | 8 + .../core/api_def/base_api/api_def_Erf.pbtxt | 4 + .../core/api_def/base_api/api_def_Erfc.pbtxt | 4 + .../core/api_def/base_api/api_def_Exit.pbtxt | 20 + .../core/api_def/base_api/api_def_Exp.pbtxt | 4 + .../api_def/base_api/api_def_ExpandDims.pbtxt | 52 + .../core/api_def/base_api/api_def_Expm1.pbtxt | 7 + .../base_api/api_def_ExtractGlimpse.pbtxt | 77 + .../api_def_ExtractImagePatches.pbtxt | 57 + .../base_api/api_def_ExtractJpegShape.pbtxt | 26 + .../core/api_def/base_api/api_def_F.pbtxt | 411 --- .../core/api_def/base_api/api_def_FFT.pbtxt | 25 + .../core/api_def/base_api/api_def_FFT2D.pbtxt | 25 + .../core/api_def/base_api/api_def_FFT3D.pbtxt | 25 + .../api_def/base_api/api_def_FIFOQueue.pbtxt | 47 + .../base_api/api_def_FIFOQueueV2.pbtxt | 49 + .../core/api_def/base_api/api_def_Fact.pbtxt | 4 + .../api_def_FakeQuantWithMinMaxArgs.pbtxt | 13 + ..._def_FakeQuantWithMinMaxArgsGradient.pbtxt | 23 + .../api_def_FakeQuantWithMinMaxVars.pbtxt | 16 + ..._def_FakeQuantWithMinMaxVarsGradient.pbtxt | 50 + ...ef_FakeQuantWithMinMaxVarsPerChannel.pbtxt | 17 + ...uantWithMinMaxVarsPerChannelGradient.pbtxt | 53 + .../api_def/base_api/api_def_FakeQueue.pbtxt | 5 + .../core/api_def/base_api/api_def_Fill.pbtxt | 31 + .../base_api/api_def_FilterDataset.pbtxt | 24 + .../api_def_FixedLengthRecordDataset.pbtxt | 37 + .../api_def_FixedLengthRecordReader.pbtxt | 50 + .../api_def_FixedLengthRecordReaderV2.pbtxt | 59 + ...api_def_FixedUnigramCandidateSampler.pbtxt | 144 + .../base_api/api_def_FlatMapDataset.pbtxt | 17 + .../core/api_def/base_api/api_def_Floor.pbtxt | 4 + .../api_def/base_api/api_def_FloorDiv.pbtxt | 8 + .../api_def/base_api/api_def_FloorMod.pbtxt | 11 + .../base_api/api_def_FractionalAvgPool.pbtxt | 90 + .../api_def_FractionalAvgPoolGrad.pbtxt | 59 + .../base_api/api_def_FractionalMaxPool.pbtxt | 114 + .../api_def_FractionalMaxPoolGrad.pbtxt | 58 + .../base_api/api_def_FusedBatchNorm.pbtxt | 99 + .../base_api/api_def_FusedBatchNormGrad.pbtxt | 102 + .../api_def_FusedBatchNormGradV2.pbtxt | 108 + .../base_api/api_def_FusedBatchNormV2.pbtxt | 105 + .../base_api/api_def_FusedPadConv2D.pbtxt | 50 + .../api_def_FusedResizeAndPadConv2D.pbtxt | 64 + .../core/api_def/base_api/api_def_G.pbtxt | 257 -- .../api_def/base_api/api_def_Gather.pbtxt | 31 + .../api_def/base_api/api_def_GatherNd.pbtxt | 123 + .../api_def/base_api/api_def_GatherV2.pbtxt | 54 + .../api_def_GenerateVocabRemapping.pbtxt | 68 + .../base_api/api_def_GetSessionHandle.pbtxt | 17 + .../base_api/api_def_GetSessionHandleV2.pbtxt | 17 + .../base_api/api_def_GetSessionTensor.pbtxt | 22 + .../api_def/base_api/api_def_Greater.pbtxt | 8 + .../base_api/api_def_GreaterEqual.pbtxt | 8 + .../api_def_GroupByWindowDataset.pbtxt | 14 + .../core/api_def/base_api/api_def_H.pbtxt | 52 - .../api_def/base_api/api_def_HSVToRGB.pbtxt | 23 + .../api_def/base_api/api_def_HashTable.pbtxt | 49 + .../base_api/api_def_HashTableV2.pbtxt | 51 + .../api_def_HistogramFixedWidth.pbtxt | 47 + .../base_api/api_def_HistogramSummary.pbtxt | 29 + .../core/api_def/base_api/api_def_I.pbtxt | 518 ---- .../core/api_def/base_api/api_def_IFFT.pbtxt | 25 + .../api_def/base_api/api_def_IFFT2D.pbtxt | 25 + .../api_def/base_api/api_def_IFFT3D.pbtxt | 25 + .../core/api_def/base_api/api_def_IRFFT.pbtxt | 43 + .../api_def/base_api/api_def_IRFFT2D.pbtxt | 44 + .../api_def/base_api/api_def_IRFFT3D.pbtxt | 44 + .../api_def/base_api/api_def_Identity.pbtxt | 4 + .../api_def/base_api/api_def_IdentityN.pbtxt | 21 + .../base_api/api_def_IdentityReader.pbtxt | 29 + .../base_api/api_def_IdentityReaderV2.pbtxt | 31 + .../api_def/base_api/api_def_Igamma.pbtxt | 19 + .../api_def/base_api/api_def_Igammac.pbtxt | 18 + .../api_def_IgnoreErrorsDataset.pbtxt | 4 + .../core/api_def/base_api/api_def_Imag.pbtxt | 17 + .../base_api/api_def_ImageSummary.pbtxt | 70 + .../base_api/api_def_ImmutableConst.pbtxt | 26 + .../api_def/base_api/api_def_InTopK.pbtxt | 44 + .../api_def/base_api/api_def_InTopKV2.pbtxt | 44 + .../base_api/api_def_InitializeTable.pbtxt | 23 + .../api_def_InitializeTableFromTextFile.pbtxt | 54 + ...pi_def_InitializeTableFromTextFileV2.pbtxt | 56 + .../base_api/api_def_InitializeTableV2.pbtxt | 25 + .../base_api/api_def_InterleaveDataset.pbtxt | 19 + .../core/api_def/base_api/api_def_Inv.pbtxt | 7 + .../api_def/base_api/api_def_InvGrad.pbtxt | 9 + .../api_def/base_api/api_def_Invert.pbtxt | 8 + .../base_api/api_def_InvertPermutation.pbtxt | 33 + .../api_def/base_api/api_def_IsFinite.pbtxt | 9 + .../core/api_def/base_api/api_def_IsInf.pbtxt | 9 + .../core/api_def/base_api/api_def_IsNan.pbtxt | 9 + .../api_def_IsVariableInitialized.pbtxt | 19 + .../api_def/base_api/api_def_Iterator.pbtxt | 11 + .../api_def_IteratorFromStringHandle.pbtxt | 30 + .../base_api/api_def_IteratorGetNext.pbtxt | 4 + .../api_def_IteratorToStringHandle.pbtxt | 16 + .../core/api_def/base_api/api_def_L.pbtxt | 392 --- .../api_def/base_api/api_def_L2Loss.pbtxt | 21 + .../api_def/base_api/api_def_LMDBReader.pbtxt | 24 + .../core/api_def/base_api/api_def_LRN.pbtxt | 47 + .../api_def/base_api/api_def_LRNGrad.pbtxt | 53 + ...i_def_LearnedUnigramCandidateSampler.pbtxt | 86 + .../api_def/base_api/api_def_LeftShift.pbtxt | 8 + .../core/api_def/base_api/api_def_Less.pbtxt | 8 + .../api_def/base_api/api_def_LessEqual.pbtxt | 8 + .../api_def/base_api/api_def_Lgamma.pbtxt | 4 + .../api_def/base_api/api_def_LinSpace.pbtxt | 39 + .../api_def/base_api/api_def_ListDiff.pbtxt | 54 + .../base_api/api_def_LoadAndRemapMatrix.pbtxt | 105 + .../core/api_def/base_api/api_def_Log.pbtxt | 7 + .../core/api_def/base_api/api_def_Log1p.pbtxt | 7 + .../api_def_LogMatrixDeterminant.pbtxt | 34 + .../api_def/base_api/api_def_LogSoftmax.pbtxt | 21 + .../api_def_LogUniformCandidateSampler.pbtxt | 86 + .../api_def/base_api/api_def_LogicalAnd.pbtxt | 8 + .../api_def/base_api/api_def_LogicalNot.pbtxt | 4 + .../api_def/base_api/api_def_LogicalOr.pbtxt | 8 + .../base_api/api_def_LookupTableExport.pbtxt | 23 + .../api_def_LookupTableExportV2.pbtxt | 25 + .../base_api/api_def_LookupTableFind.pbtxt | 31 + .../base_api/api_def_LookupTableFindV2.pbtxt | 33 + .../base_api/api_def_LookupTableImport.pbtxt | 27 + .../api_def_LookupTableImportV2.pbtxt | 29 + .../base_api/api_def_LookupTableInsert.pbtxt | 27 + .../api_def_LookupTableInsertV2.pbtxt | 29 + .../base_api/api_def_LookupTableSize.pbtxt | 17 + .../base_api/api_def_LookupTableSizeV2.pbtxt | 19 + .../api_def/base_api/api_def_LoopCond.pbtxt | 20 + .../core/api_def/base_api/api_def_M.pbtxt | 749 ----- .../base_api/api_def_MakeIterator.pbtxt | 8 + .../base_api/api_def_MapAndBatchDataset.pbtxt | 26 + .../api_def/base_api/api_def_MapClear.pbtxt | 4 + .../api_def/base_api/api_def_MapDataset.pbtxt | 4 + .../base_api/api_def_MapIncompleteSize.pbtxt | 4 + .../api_def/base_api/api_def_MapPeek.pbtxt | 8 + .../api_def/base_api/api_def_MapSize.pbtxt | 4 + .../api_def/base_api/api_def_MapStage.pbtxt | 37 + .../api_def/base_api/api_def_MapUnstage.pbtxt | 8 + .../base_api/api_def_MapUnstageNoKey.pbtxt | 8 + .../api_def/base_api/api_def_MatMul.pbtxt | 25 + .../base_api/api_def_MatchingFiles.pbtxt | 20 + .../base_api/api_def_MatrixBandPart.pbtxt | 71 + .../base_api/api_def_MatrixDeterminant.pbtxt | 21 + .../api_def/base_api/api_def_MatrixDiag.pbtxt | 44 + .../base_api/api_def_MatrixDiagPart.pbtxt | 47 + .../base_api/api_def_MatrixInverse.pbtxt | 33 + .../base_api/api_def_MatrixSetDiag.pbtxt | 36 + .../base_api/api_def_MatrixSolve.pbtxt | 37 + .../base_api/api_def_MatrixSolveLs.pbtxt | 68 + .../api_def_MatrixTriangularSolve.pbtxt | 57 + .../core/api_def/base_api/api_def_Max.pbtxt | 42 + .../api_def/base_api/api_def_MaxPool.pbtxt | 45 + .../api_def/base_api/api_def_MaxPool3D.pbtxt | 46 + .../base_api/api_def_MaxPool3DGrad.pbtxt | 52 + .../base_api/api_def_MaxPool3DGradGrad.pbtxt | 58 + .../base_api/api_def_MaxPoolGrad.pbtxt | 58 + .../base_api/api_def_MaxPoolGradGrad.pbtxt | 57 + .../base_api/api_def_MaxPoolGradGradV2.pbtxt | 57 + .../api_def_MaxPoolGradGradWithArgmax.pbtxt | 48 + .../base_api/api_def_MaxPoolGradV2.pbtxt | 57 + .../api_def_MaxPoolGradWithArgmax.pbtxt | 49 + .../api_def/base_api/api_def_MaxPoolV2.pbtxt | 45 + .../base_api/api_def_MaxPoolWithArgmax.pbtxt | 51 + .../api_def/base_api/api_def_Maximum.pbtxt | 8 + .../core/api_def/base_api/api_def_Mean.pbtxt | 42 + .../core/api_def/base_api/api_def_Merge.pbtxt | 29 + .../base_api/api_def_MergeSummary.pbtxt | 26 + .../base_api/api_def_MergeV2Checkpoints.pbtxt | 33 + .../core/api_def/base_api/api_def_Mfcc.pbtxt | 51 + .../core/api_def/base_api/api_def_Min.pbtxt | 42 + .../api_def/base_api/api_def_Minimum.pbtxt | 8 + .../api_def/base_api/api_def_MirrorPad.pbtxt | 60 + .../base_api/api_def_MirrorPadGrad.pbtxt | 50 + .../core/api_def/base_api/api_def_Mod.pbtxt | 11 + .../core/api_def/base_api/api_def_Mul.pbtxt | 14 + .../base_api/api_def_Multinomial.pbtxt | 37 + .../api_def_MutableDenseHashTable.pbtxt | 72 + .../api_def_MutableDenseHashTableV2.pbtxt | 74 + .../base_api/api_def_MutableHashTable.pbtxt | 49 + .../api_def_MutableHashTableOfTensors.pbtxt | 42 + .../api_def_MutableHashTableOfTensorsV2.pbtxt | 44 + .../base_api/api_def_MutableHashTableV2.pbtxt | 51 + .../core/api_def/base_api/api_def_N.pbtxt | 94 - .../core/api_def/base_api/api_def_Neg.pbtxt | 13 + .../api_def/base_api/api_def_NegTrain.pbtxt | 40 + .../base_api/api_def_NextIteration.pbtxt | 16 + .../core/api_def/base_api/api_def_NoOp.pbtxt | 4 + .../base_api/api_def_NonMaxSuppression.pbtxt | 56 + .../api_def_NonMaxSuppressionV2.pbtxt | 58 + .../api_def/base_api/api_def_NotEqual.pbtxt | 8 + .../api_def/base_api/api_def_NthElement.pbtxt | 39 + .../core/api_def/base_api/api_def_O.pbtxt | 195 -- .../api_def/base_api/api_def_OneHot.pbtxt | 130 + .../base_api/api_def_OneShotIterator.pbtxt | 37 + .../api_def/base_api/api_def_OnesLike.pbtxt | 16 + .../base_api/api_def_OrderedMapClear.pbtxt | 4 + .../api_def_OrderedMapIncompleteSize.pbtxt | 4 + .../base_api/api_def_OrderedMapPeek.pbtxt | 9 + .../base_api/api_def_OrderedMapSize.pbtxt | 4 + .../base_api/api_def_OrderedMapStage.pbtxt | 40 + .../base_api/api_def_OrderedMapUnstage.pbtxt | 8 + .../api_def_OrderedMapUnstageNoKey.pbtxt | 8 + .../core/api_def/base_api/api_def_P.pbtxt | 431 --- .../core/api_def/base_api/api_def_Pack.pbtxt | 47 + .../core/api_def/base_api/api_def_Pad.pbtxt | 28 + .../core/api_def/base_api/api_def_PadV2.pbtxt | 30 + .../base_api/api_def_PaddedBatchDataset.pbtxt | 27 + .../base_api/api_def_PaddingFIFOQueue.pbtxt | 56 + .../base_api/api_def_PaddingFIFOQueueV2.pbtxt | 58 + .../base_api/api_def_ParallelConcat.pbtxt | 42 + .../api_def_ParallelDynamicStitch.pbtxt | 67 + .../api_def_ParallelInterleaveDataset.pbtxt | 21 + .../base_api/api_def_ParallelMapDataset.pbtxt | 15 + ...api_def_ParameterizedTruncatedNormal.pbtxt | 66 + .../base_api/api_def_ParseExample.pbtxt | 78 + .../api_def_ParseSingleSequenceExample.pbtxt | 112 + .../base_api/api_def_ParseTensor.pbtxt | 23 + .../base_api/api_def_Placeholder.pbtxt | 28 + .../base_api/api_def_PlaceholderV2.pbtxt | 28 + .../api_def_PlaceholderWithDefault.pbtxt | 28 + .../api_def/base_api/api_def_Polygamma.pbtxt | 12 + .../base_api/api_def_PopulationCount.pbtxt | 12 + .../core/api_def/base_api/api_def_Pow.pbtxt | 14 + .../base_api/api_def_PrefetchDataset.pbtxt | 11 + .../base_api/api_def_PreventGradient.pbtxt | 32 + .../core/api_def/base_api/api_def_Print.pbtxt | 43 + .../base_api/api_def_PriorityQueue.pbtxt | 54 + .../base_api/api_def_PriorityQueueV2.pbtxt | 56 + .../core/api_def/base_api/api_def_Prod.pbtxt | 42 + .../api_def/base_api/api_def_PyFunc.pbtxt | 40 + .../base_api/api_def_PyFuncStateless.pbtxt | 5 + .../core/api_def/base_api/api_def_Q.pbtxt | 609 ---- .../core/api_def/base_api/api_def_Qr.pbtxt | 45 + .../api_def_QuantizeAndDequantize.pbtxt | 4 + .../api_def_QuantizeAndDequantizeV2.pbtxt | 93 + .../api_def_QuantizeAndDequantizeV3.pbtxt | 8 + .../api_def_QuantizeDownAndShrinkRange.pbtxt | 64 + .../api_def/base_api/api_def_QuantizeV2.pbtxt | 128 + .../base_api/api_def_QuantizedAdd.pbtxt | 43 + .../base_api/api_def_QuantizedAvgPool.pbtxt | 54 + ...izedBatchNormWithGlobalNormalization.pbtxt | 118 + .../base_api/api_def_QuantizedBiasAdd.pbtxt | 49 + .../base_api/api_def_QuantizedConcat.pbtxt | 50 + .../base_api/api_def_QuantizedConv2D.pbtxt | 65 + .../api_def_QuantizedInstanceNorm.pbtxt | 72 + .../base_api/api_def_QuantizedMatMul.pbtxt | 77 + .../base_api/api_def_QuantizedMaxPool.pbtxt | 54 + .../base_api/api_def_QuantizedMul.pbtxt | 43 + .../base_api/api_def_QuantizedRelu.pbtxt | 34 + .../base_api/api_def_QuantizedRelu6.pbtxt | 34 + .../base_api/api_def_QuantizedReluX.pbtxt | 34 + .../base_api/api_def_QuantizedReshape.pbtxt | 37 + .../api_def_QuantizedResizeBilinear.pbtxt | 35 + .../api_def/base_api/api_def_QueueClose.pbtxt | 25 + .../base_api/api_def_QueueCloseV2.pbtxt | 27 + .../base_api/api_def_QueueDequeue.pbtxt | 39 + .../base_api/api_def_QueueDequeueMany.pbtxt | 52 + .../base_api/api_def_QueueDequeueManyV2.pbtxt | 54 + .../base_api/api_def_QueueDequeueUpTo.pbtxt | 56 + .../base_api/api_def_QueueDequeueUpToV2.pbtxt | 58 + .../base_api/api_def_QueueDequeueV2.pbtxt | 41 + .../base_api/api_def_QueueEnqueue.pbtxt | 32 + .../base_api/api_def_QueueEnqueueMany.pbtxt | 37 + .../base_api/api_def_QueueEnqueueManyV2.pbtxt | 39 + .../base_api/api_def_QueueEnqueueV2.pbtxt | 34 + .../base_api/api_def_QueueIsClosed.pbtxt | 14 + .../base_api/api_def_QueueIsClosedV2.pbtxt | 14 + .../api_def/base_api/api_def_QueueSize.pbtxt | 17 + .../base_api/api_def_QueueSizeV2.pbtxt | 19 + .../core/api_def/base_api/api_def_R.pbtxt | 1392 --------- .../core/api_def/base_api/api_def_RFFT.pbtxt | 40 + .../api_def/base_api/api_def_RFFT2D.pbtxt | 42 + .../api_def/base_api/api_def_RFFT3D.pbtxt | 42 + .../api_def/base_api/api_def_RGBToHSV.pbtxt | 25 + .../api_def/base_api/api_def_RandomCrop.pbtxt | 44 + .../base_api/api_def_RandomGamma.pbtxt | 45 + .../base_api/api_def_RandomPoisson.pbtxt | 4 + .../base_api/api_def_RandomPoissonV2.pbtxt | 51 + .../base_api/api_def_RandomShuffle.pbtxt | 42 + .../base_api/api_def_RandomShuffleQueue.pbtxt | 68 + .../api_def_RandomShuffleQueueV2.pbtxt | 70 + .../api_def_RandomStandardNormal.pbtxt | 42 + .../base_api/api_def_RandomUniform.pbtxt | 40 + .../base_api/api_def_RandomUniformInt.pbtxt | 51 + .../core/api_def/base_api/api_def_Range.pbtxt | 41 + .../base_api/api_def_RangeDataset.pbtxt | 22 + .../core/api_def/base_api/api_def_Rank.pbtxt | 19 + .../api_def/base_api/api_def_ReadFile.pbtxt | 4 + .../base_api/api_def_ReadVariableOp.pbtxt | 24 + .../api_def_ReaderNumRecordsProduced.pbtxt | 15 + .../api_def_ReaderNumRecordsProducedV2.pbtxt | 17 + .../api_def_ReaderNumWorkUnitsCompleted.pbtxt | 11 + ...pi_def_ReaderNumWorkUnitsCompletedV2.pbtxt | 13 + .../api_def/base_api/api_def_ReaderRead.pbtxt | 34 + .../base_api/api_def_ReaderReadUpTo.pbtxt | 41 + .../base_api/api_def_ReaderReadUpToV2.pbtxt | 43 + .../base_api/api_def_ReaderReadV2.pbtxt | 36 + .../base_api/api_def_ReaderReset.pbtxt | 11 + .../base_api/api_def_ReaderResetV2.pbtxt | 13 + .../base_api/api_def_ReaderRestoreState.pbtxt | 22 + .../api_def_ReaderRestoreStateV2.pbtxt | 24 + .../api_def_ReaderSerializeState.pbtxt | 15 + .../api_def_ReaderSerializeStateV2.pbtxt | 17 + .../core/api_def/base_api/api_def_Real.pbtxt | 17 + .../api_def/base_api/api_def_RealDiv.pbtxt | 10 + .../api_def/base_api/api_def_Reciprocal.pbtxt | 7 + .../base_api/api_def_ReciprocalGrad.pbtxt | 9 + .../base_api/api_def_RecordInput.pbtxt | 47 + .../api_def/base_api/api_def_ReduceJoin.pbtxt | 59 + .../api_def/base_api/api_def_RefEnter.pbtxt | 41 + .../api_def/base_api/api_def_RefExit.pbtxt | 20 + .../base_api/api_def_RefIdentity.pbtxt | 5 + .../api_def/base_api/api_def_RefMerge.pbtxt | 30 + .../base_api/api_def_RefNextIteration.pbtxt | 16 + .../api_def/base_api/api_def_RefSelect.pbtxt | 22 + .../api_def/base_api/api_def_RefSwitch.pbtxt | 34 + .../core/api_def/base_api/api_def_Relu.pbtxt | 4 + .../core/api_def/base_api/api_def_Relu6.pbtxt | 4 + .../api_def/base_api/api_def_Relu6Grad.pbtxt | 25 + .../api_def/base_api/api_def_ReluGrad.pbtxt | 24 + .../api_def/base_api/api_def_RemoteCall.pbtxt | 40 + .../api_def_RemoteFusedGraphExecute.pbtxt | 32 + .../base_api/api_def_RepeatDataset.pbtxt | 11 + .../api_def_RequantizationRange.pbtxt | 39 + .../api_def/base_api/api_def_Requantize.pbtxt | 60 + .../api_def/base_api/api_def_Reshape.pbtxt | 68 + .../api_def/base_api/api_def_ResizeArea.pbtxt | 40 + .../base_api/api_def_ResizeBicubic.pbtxt | 35 + .../base_api/api_def_ResizeBicubicGrad.pbtxt | 34 + .../base_api/api_def_ResizeBilinear.pbtxt | 35 + .../base_api/api_def_ResizeBilinearGrad.pbtxt | 34 + .../api_def_ResizeNearestNeighbor.pbtxt | 32 + .../api_def_ResizeNearestNeighborGrad.pbtxt | 33 + .../api_def_ResourceApplyAdadelta.pbtxt | 59 + .../api_def_ResourceApplyAdagrad.pbtxt | 40 + .../api_def_ResourceApplyAdagradDA.pbtxt | 59 + .../base_api/api_def_ResourceApplyAdam.pbtxt | 84 + ...api_def_ResourceApplyCenteredRMSProp.pbtxt | 80 + .../base_api/api_def_ResourceApplyFtrl.pbtxt | 67 + .../api_def_ResourceApplyFtrlV2.pbtxt | 69 + ...api_def_ResourceApplyGradientDescent.pbtxt | 29 + .../api_def_ResourceApplyMomentum.pbtxt | 56 + ...api_def_ResourceApplyProximalAdagrad.pbtxt | 52 + ...ResourceApplyProximalGradientDescent.pbtxt | 45 + .../api_def_ResourceApplyRMSProp.pbtxt | 66 + .../base_api/api_def_ResourceCountUpTo.pbtxt | 24 + .../base_api/api_def_ResourceGather.pbtxt | 19 + .../base_api/api_def_ResourceScatterAdd.pbtxt | 43 + .../api_def_ResourceScatterUpdate.pbtxt | 34 + .../api_def_ResourceSparseApplyAdadelta.pbtxt | 53 + .../api_def_ResourceSparseApplyAdagrad.pbtxt | 47 + ...api_def_ResourceSparseApplyAdagradDA.pbtxt | 65 + ...f_ResourceSparseApplyCenteredRMSProp.pbtxt | 84 + .../api_def_ResourceSparseApplyFtrl.pbtxt | 74 + .../api_def_ResourceSparseApplyFtrlV2.pbtxt | 76 + .../api_def_ResourceSparseApplyMomentum.pbtxt | 64 + ...f_ResourceSparseApplyProximalAdagrad.pbtxt | 60 + ...ceSparseApplyProximalGradientDescent.pbtxt | 52 + .../api_def_ResourceSparseApplyRMSProp.pbtxt | 72 + .../api_def_ResourceStridedSliceAssign.pbtxt | 12 + .../api_def/base_api/api_def_Restore.pbtxt | 55 + .../base_api/api_def_RestoreSlice.pbtxt | 52 + .../api_def/base_api/api_def_RestoreV2.pbtxt | 52 + .../api_def/base_api/api_def_Reverse.pbtxt | 69 + .../base_api/api_def_ReverseSequence.pbtxt | 91 + .../api_def/base_api/api_def_ReverseV2.pbtxt | 74 + .../api_def/base_api/api_def_RightShift.pbtxt | 11 + .../core/api_def/base_api/api_def_Rint.pbtxt | 15 + .../core/api_def/base_api/api_def_Round.pbtxt | 8 + .../core/api_def/base_api/api_def_Rsqrt.pbtxt | 7 + .../api_def/base_api/api_def_RsqrtGrad.pbtxt | 9 + .../core/api_def/base_api/api_def_S.pbtxt | 2678 ----------------- .../api_def_SampleDistortedBoundingBox.pbtxt | 131 + ...api_def_SampleDistortedBoundingBoxV2.pbtxt | 131 + .../core/api_def/base_api/api_def_Save.pbtxt | 29 + .../api_def/base_api/api_def_SaveSlices.pbtxt | 53 + .../api_def/base_api/api_def_SaveV2.pbtxt | 35 + .../base_api/api_def_ScalarSummary.pbtxt | 26 + .../base_api/api_def_ScanDataset.pbtxt | 4 + .../api_def/base_api/api_def_ScatterAdd.pbtxt | 60 + .../api_def/base_api/api_def_ScatterDiv.pbtxt | 58 + .../api_def/base_api/api_def_ScatterMul.pbtxt | 58 + .../api_def/base_api/api_def_ScatterNd.pbtxt | 102 + .../base_api/api_def_ScatterNdAdd.pbtxt | 74 + .../api_def_ScatterNdNonAliasingAdd.pbtxt | 68 + .../base_api/api_def_ScatterNdSub.pbtxt | 74 + .../base_api/api_def_ScatterNdUpdate.pbtxt | 76 + .../api_def/base_api/api_def_ScatterSub.pbtxt | 60 + .../base_api/api_def_ScatterUpdate.pbtxt | 63 + .../api_def/base_api/api_def_SdcaFprint.pbtxt | 17 + .../base_api/api_def_SdcaOptimizer.pbtxt | 167 + .../base_api/api_def_SdcaShrinkL1.pbtxt | 29 + .../api_def/base_api/api_def_SegmentMax.pbtxt | 32 + .../base_api/api_def_SegmentMean.pbtxt | 33 + .../api_def/base_api/api_def_SegmentMin.pbtxt | 32 + .../base_api/api_def_SegmentProd.pbtxt | 32 + .../api_def/base_api/api_def_SegmentSum.pbtxt | 32 + .../api_def/base_api/api_def_Select.pbtxt | 69 + .../base_api/api_def_SelfAdjointEig.pbtxt | 24 + .../base_api/api_def_SelfAdjointEigV2.pbtxt | 44 + .../core/api_def/base_api/api_def_Selu.pbtxt | 9 + .../api_def/base_api/api_def_SeluGrad.pbtxt | 24 + .../base_api/api_def_SerializeIterator.pbtxt | 17 + .../api_def_SerializeManySparse.pbtxt | 31 + .../base_api/api_def_SerializeSparse.pbtxt | 22 + .../base_api/api_def_SerializeTensor.pbtxt | 22 + .../api_def/base_api/api_def_SetSize.pbtxt | 38 + .../core/api_def/base_api/api_def_Shape.pbtxt | 14 + .../api_def/base_api/api_def_ShapeN.pbtxt | 7 + .../base_api/api_def_ShardedFilename.pbtxt | 7 + .../base_api/api_def_ShardedFilespec.pbtxt | 4 + .../base_api/api_def_ShuffleDataset.pbtxt | 36 + .../api_def/base_api/api_def_Sigmoid.pbtxt | 7 + .../base_api/api_def_SigmoidGrad.pbtxt | 9 + .../core/api_def/base_api/api_def_Sign.pbtxt | 9 + .../core/api_def/base_api/api_def_Sin.pbtxt | 4 + .../core/api_def/base_api/api_def_Sinh.pbtxt | 4 + .../core/api_def/base_api/api_def_Size.pbtxt | 15 + .../base_api/api_def_SkipDataset.pbtxt | 11 + .../api_def/base_api/api_def_Skipgram.pbtxt | 78 + .../core/api_def/base_api/api_def_Slice.pbtxt | 28 + .../api_def/base_api/api_def_Softmax.pbtxt | 21 + ...pi_def_SoftmaxCrossEntropyWithLogits.pbtxt | 33 + .../api_def/base_api/api_def_Softplus.pbtxt | 4 + .../base_api/api_def_SoftplusGrad.pbtxt | 23 + .../api_def/base_api/api_def_Softsign.pbtxt | 4 + .../base_api/api_def_SoftsignGrad.pbtxt | 23 + .../base_api/api_def_SpaceToBatch.pbtxt | 109 + .../base_api/api_def_SpaceToBatchND.pbtxt | 140 + .../base_api/api_def_SpaceToDepth.pbtxt | 95 + ...i_def_SparseAccumulatorApplyGradient.pbtxt | 55 + ...pi_def_SparseAccumulatorTakeGradient.pbtxt | 49 + .../api_def/base_api/api_def_SparseAdd.pbtxt | 62 + .../base_api/api_def_SparseAddGrad.pbtxt | 50 + .../api_def_SparseApplyAdadelta.pbtxt | 59 + .../base_api/api_def_SparseApplyAdagrad.pbtxt | 53 + .../api_def_SparseApplyAdagradDA.pbtxt | 71 + .../api_def_SparseApplyCenteredRMSProp.pbtxt | 90 + .../base_api/api_def_SparseApplyFtrl.pbtxt | 80 + .../base_api/api_def_SparseApplyFtrlV2.pbtxt | 82 + .../api_def_SparseApplyMomentum.pbtxt | 70 + .../api_def_SparseApplyProximalAdagrad.pbtxt | 66 + ...f_SparseApplyProximalGradientDescent.pbtxt | 58 + .../base_api/api_def_SparseApplyRMSProp.pbtxt | 78 + .../base_api/api_def_SparseConcat.pbtxt | 90 + ...api_def_SparseConditionalAccumulator.pbtxt | 44 + .../base_api/api_def_SparseCross.pbtxt | 106 + .../api_def_SparseDenseCwiseAdd.pbtxt | 45 + .../api_def_SparseDenseCwiseDiv.pbtxt | 39 + .../api_def_SparseDenseCwiseMul.pbtxt | 43 + .../api_def_SparseFillEmptyRows.pbtxt | 87 + .../api_def_SparseFillEmptyRowsGrad.pbtxt | 38 + .../base_api/api_def_SparseMatMul.pbtxt | 13 + .../base_api/api_def_SparseReduceMax.pbtxt | 55 + .../api_def_SparseReduceMaxSparse.pbtxt | 49 + .../base_api/api_def_SparseReduceSum.pbtxt | 55 + .../api_def_SparseReduceSumSparse.pbtxt | 49 + .../base_api/api_def_SparseReorder.pbtxt | 46 + .../base_api/api_def_SparseReshape.pbtxt | 55 + .../base_api/api_def_SparseSegmentMean.pbtxt | 30 + .../api_def_SparseSegmentMeanGrad.pbtxt | 32 + .../base_api/api_def_SparseSegmentSqrtN.pbtxt | 29 + .../api_def_SparseSegmentSqrtNGrad.pbtxt | 32 + .../base_api/api_def_SparseSegmentSum.pbtxt | 53 + .../base_api/api_def_SparseSlice.pbtxt | 67 + .../base_api/api_def_SparseSoftmax.pbtxt | 46 + ..._SparseSoftmaxCrossEntropyWithLogits.pbtxt | 37 + .../api_def_SparseSparseMaximum.pbtxt | 56 + .../api_def_SparseSparseMinimum.pbtxt | 56 + .../base_api/api_def_SparseSplit.pbtxt | 70 + .../api_def_SparseTensorDenseAdd.pbtxt | 31 + .../api_def_SparseTensorDenseMatMul.pbtxt | 53 + .../api_def_SparseTensorSliceDataset.pbtxt | 4 + .../base_api/api_def_SparseToDense.pbtxt | 65 + .../api_def_SparseToSparseSetOperation.pbtxt | 93 + .../core/api_def/base_api/api_def_Split.pbtxt | 33 + .../api_def/base_api/api_def_SplitV.pbtxt | 34 + .../api_def/base_api/api_def_SqlDataset.pbtxt | 22 + .../core/api_def/base_api/api_def_Sqrt.pbtxt | 7 + .../api_def/base_api/api_def_SqrtGrad.pbtxt | 9 + .../api_def/base_api/api_def_Square.pbtxt | 7 + .../base_api/api_def_SquaredDifference.pbtxt | 8 + .../api_def/base_api/api_def_Squeeze.pbtxt | 46 + .../core/api_def/base_api/api_def_Stack.pbtxt | 5 + .../api_def/base_api/api_def_StackClose.pbtxt | 5 + .../base_api/api_def_StackCloseV2.pbtxt | 11 + .../api_def/base_api/api_def_StackPop.pbtxt | 5 + .../api_def/base_api/api_def_StackPopV2.pbtxt | 23 + .../api_def/base_api/api_def_StackPush.pbtxt | 5 + .../base_api/api_def_StackPushV2.pbtxt | 29 + .../api_def/base_api/api_def_StackV2.pbtxt | 31 + .../core/api_def/base_api/api_def_Stage.pbtxt | 42 + .../api_def/base_api/api_def_StageClear.pbtxt | 4 + .../api_def/base_api/api_def_StagePeek.pbtxt | 9 + .../api_def/base_api/api_def_StageSize.pbtxt | 4 + .../api_def_StatelessRandomNormal.pbtxt | 33 + .../api_def_StatelessRandomUniform.pbtxt | 34 + .../api_def_StatelessTruncatedNormal.pbtxt | 35 + .../base_api/api_def_StopGradient.pbtxt | 25 + .../base_api/api_def_StridedSlice.pbtxt | 167 + .../base_api/api_def_StridedSliceAssign.pbtxt | 12 + .../base_api/api_def_StridedSliceGrad.pbtxt | 14 + .../api_def/base_api/api_def_StringJoin.pbtxt | 21 + .../base_api/api_def_StringSplit.pbtxt | 64 + .../base_api/api_def_StringToHashBucket.pbtxt | 24 + .../api_def_StringToHashBucketFast.pbtxt | 30 + .../api_def_StringToHashBucketStrong.pbtxt | 41 + .../base_api/api_def_StringToNumber.pbtxt | 20 + .../core/api_def/base_api/api_def_Sub.pbtxt | 14 + .../api_def/base_api/api_def_Substr.pbtxt | 103 + .../core/api_def/base_api/api_def_Sum.pbtxt | 42 + .../core/api_def/base_api/api_def_Svd.pbtxt | 62 + .../api_def/base_api/api_def_Switch.pbtxt | 34 + .../base_api/api_def_SymbolicGradient.pbtxt | 51 + .../core/api_def/base_api/api_def_T.pbtxt | 619 ---- .../base_api/api_def_TFRecordDataset.pbtxt | 25 + .../base_api/api_def_TFRecordReader.pbtxt | 25 + .../base_api/api_def_TFRecordReaderV2.pbtxt | 27 + .../base_api/api_def_TakeDataset.pbtxt | 12 + ...api_def_TakeManySparseFromTensorsMap.pbtxt | 100 + .../core/api_def/base_api/api_def_Tan.pbtxt | 4 + .../core/api_def/base_api/api_def_Tanh.pbtxt | 4 + .../api_def/base_api/api_def_TanhGrad.pbtxt | 9 + .../base_api/api_def_TemporaryVariable.pbtxt | 45 + .../base_api/api_def_TensorArray.pbtxt | 3 + .../base_api/api_def_TensorArrayClose.pbtxt | 3 + .../base_api/api_def_TensorArrayCloseV2.pbtxt | 5 + .../base_api/api_def_TensorArrayCloseV3.pbtxt | 17 + .../base_api/api_def_TensorArrayConcat.pbtxt | 3 + .../api_def_TensorArrayConcatV2.pbtxt | 5 + .../api_def_TensorArrayConcatV3.pbtxt | 62 + .../base_api/api_def_TensorArrayGather.pbtxt | 3 + .../api_def_TensorArrayGatherV2.pbtxt | 5 + .../api_def_TensorArrayGatherV3.pbtxt | 49 + .../base_api/api_def_TensorArrayGrad.pbtxt | 3 + .../base_api/api_def_TensorArrayGradV2.pbtxt | 5 + .../base_api/api_def_TensorArrayGradV3.pbtxt | 64 + .../base_api/api_def_TensorArrayPack.pbtxt | 3 + .../base_api/api_def_TensorArrayRead.pbtxt | 3 + .../base_api/api_def_TensorArrayReadV2.pbtxt | 5 + .../base_api/api_def_TensorArrayReadV3.pbtxt | 31 + .../base_api/api_def_TensorArrayScatter.pbtxt | 3 + .../api_def_TensorArrayScatterV2.pbtxt | 5 + .../api_def_TensorArrayScatterV3.pbtxt | 40 + .../base_api/api_def_TensorArraySize.pbtxt | 3 + .../base_api/api_def_TensorArraySizeV2.pbtxt | 5 + .../base_api/api_def_TensorArraySizeV3.pbtxt | 25 + .../base_api/api_def_TensorArraySplit.pbtxt | 3 + .../base_api/api_def_TensorArraySplitV2.pbtxt | 5 + .../base_api/api_def_TensorArraySplitV3.pbtxt | 57 + .../base_api/api_def_TensorArrayUnpack.pbtxt | 3 + .../base_api/api_def_TensorArrayV2.pbtxt | 5 + .../base_api/api_def_TensorArrayV3.pbtxt | 65 + .../base_api/api_def_TensorArrayWrite.pbtxt | 3 + .../base_api/api_def_TensorArrayWriteV2.pbtxt | 5 + .../base_api/api_def_TensorArrayWriteV3.pbtxt | 37 + .../base_api/api_def_TensorDataset.pbtxt | 4 + .../base_api/api_def_TensorSliceDataset.pbtxt | 4 + .../base_api/api_def_TensorSummary.pbtxt | 33 + .../base_api/api_def_TensorSummaryV2.pbtxt | 23 + .../base_api/api_def_TextLineDataset.pbtxt | 24 + .../base_api/api_def_TextLineReader.pbtxt | 31 + .../base_api/api_def_TextLineReaderV2.pbtxt | 33 + ..._ThreadUnsafeUnigramCandidateSampler.pbtxt | 87 + .../core/api_def/base_api/api_def_Tile.pbtxt | 23 + .../api_def/base_api/api_def_TileGrad.pbtxt | 9 + .../core/api_def/base_api/api_def_TopK.pbtxt | 50 + .../api_def/base_api/api_def_TopKV2.pbtxt | 51 + .../api_def/base_api/api_def_Transpose.pbtxt | 8 + .../base_api/api_def_TruncateDiv.pbtxt | 13 + .../base_api/api_def_TruncateMod.pbtxt | 11 + .../base_api/api_def_TruncatedNormal.pbtxt | 42 + .../core/api_def/base_api/api_def_U.pbtxt | 150 - .../api_def_UniformCandidateSampler.pbtxt | 86 + .../api_def/base_api/api_def_Unique.pbtxt | 39 + .../base_api/api_def_UniqueWithCounts.pbtxt | 47 + .../api_def/base_api/api_def_Unpack.pbtxt | 40 + .../base_api/api_def_UnsortedSegmentMax.pbtxt | 36 + .../base_api/api_def_UnsortedSegmentSum.pbtxt | 36 + .../api_def/base_api/api_def_Unstage.pbtxt | 8 + .../core/api_def/base_api/api_def_V.pbtxt | 19 - .../base_api/api_def_VarHandleOp.pbtxt | 29 + .../base_api/api_def_VarIsInitializedOp.pbtxt | 17 + .../api_def/base_api/api_def_Variable.pbtxt | 5 + .../base_api/api_def_VariableShape.pbtxt | 14 + .../api_def/base_api/api_def_VariableV2.pbtxt | 44 + .../{api_def_W.pbtxt => api_def_Where.pbtxt} | 67 +- .../base_api/api_def_WholeFileReader.pbtxt | 29 + .../base_api/api_def_WholeFileReaderV2.pbtxt | 31 + .../api_def/base_api/api_def_WriteFile.pbtxt | 19 + .../core/api_def/base_api/api_def_Z.pbtxt | 27 - .../api_def/base_api/api_def_ZerosLike.pbtxt | 16 + .../core/api_def/base_api/api_def_Zeta.pbtxt | 10 + .../api_def/base_api/api_def_ZipDataset.pbtxt | 4 + .../core/api_def/python_api/api_def_A.pbtxt | 56 - .../core/api_def/python_api/api_def_Abs.pbtxt | 4 + .../python_api/api_def_AccumulateNV2.pbtxt | 4 + .../api_def_AddManySparseToTensorsMap.pbtxt | 4 + .../api_def/python_api/api_def_AddN.pbtxt | 4 + .../api_def_AddSparseToTensorsMap.pbtxt | 4 + .../api_def/python_api/api_def_AddV2.pbtxt | 4 + .../python_api/api_def_AdjustContrastv2.pbtxt | 4 + .../core/api_def/python_api/api_def_All.pbtxt | 4 + .../api_def_AllCandidateSampler.pbtxt | 4 + .../core/api_def/python_api/api_def_Any.pbtxt | 4 + .../api_def/python_api/api_def_Assert.pbtxt | 4 + .../python_api/api_def_AudioSummary.pbtxt | 4 + .../python_api/api_def_AudioSummaryV2.pbtxt | 4 + .../api_def/python_api/api_def_AvgPool.pbtxt | 4 + .../python_api/api_def_AvgPool3DGrad.pbtxt | 4 + .../core/api_def/python_api/api_def_B.pbtxt | 142 - .../api_def/python_api/api_def_Barrier.pbtxt | 4 + .../python_api/api_def_BarrierClose.pbtxt | 4 + .../api_def_BarrierIncompleteSize.pbtxt | 4 + .../api_def_BarrierInsertMany.pbtxt | 4 + .../python_api/api_def_BarrierReadySize.pbtxt | 4 + .../python_api/api_def_BarrierTakeMany.pbtxt | 4 + .../python_api/api_def_BatchCholesky.pbtxt | 4 + .../api_def_BatchCholeskyGrad.pbtxt | 4 + .../api_def/python_api/api_def_BatchFFT.pbtxt | 4 + .../python_api/api_def_BatchFFT2D.pbtxt | 4 + .../python_api/api_def_BatchFFT3D.pbtxt | 4 + .../python_api/api_def_BatchIFFT.pbtxt | 4 + .../python_api/api_def_BatchIFFT2D.pbtxt | 4 + .../python_api/api_def_BatchIFFT3D.pbtxt | 4 + .../python_api/api_def_BatchMatMul.pbtxt | 4 + .../api_def_BatchMatrixDeterminant.pbtxt | 4 + .../api_def_BatchMatrixInverse.pbtxt | 4 + .../python_api/api_def_BatchMatrixSolve.pbtxt | 4 + .../api_def_BatchMatrixSolveLs.pbtxt | 4 + .../api_def_BatchMatrixTriangularSolve.pbtxt | 4 + ...def_BatchNormWithGlobalNormalization.pbtxt | 4 + ...BatchNormWithGlobalNormalizationGrad.pbtxt | 4 + .../api_def_BatchSelfAdjointEig.pbtxt | 4 + .../api_def_BatchSelfAdjointEigV2.pbtxt | 4 + .../api_def/python_api/api_def_BatchSvd.pbtxt | 4 + .../python_api/api_def_BatchToSpace.pbtxt | 4 + .../api_def/python_api/api_def_BiasAdd.pbtxt | 4 + .../python_api/api_def_BiasAddV1.pbtxt | 4 + .../python_api/api_def_BitwiseAnd.pbtxt | 6 + .../python_api/api_def_BitwiseOr.pbtxt | 6 + .../python_api/api_def_BitwiseXor.pbtxt | 6 + .../python_api/api_def_BroadcastArgs.pbtxt | 4 + .../python_api/api_def_Bucketize.pbtxt | 4 + .../core/api_def/python_api/api_def_C.pbtxt | 59 - .../api_def_CTCBeamSearchDecoder.pbtxt | 4 + .../python_api/api_def_CTCGreedyDecoder.pbtxt | 4 + .../api_def/python_api/api_def_CTCLoss.pbtxt | 4 + .../api_def/python_api/api_def_Cholesky.pbtxt | 9 + .../api_def/python_api/api_def_Complex.pbtxt | 4 + .../python_api/api_def_ComplexAbs.pbtxt | 4 + .../api_def_ComputeAccidentalHits.pbtxt | 4 + .../api_def/python_api/api_def_Concat.pbtxt | 4 + .../python_api/api_def_ConcatOffset.pbtxt | 4 + .../api_def/python_api/api_def_ConcatV2.pbtxt | 4 + .../api_def/python_api/api_def_Conj.pbtxt | 4 + .../api_def_ConjugateTranspose.pbtxt | 4 + .../api_def/python_api/api_def_Const.pbtxt | 4 + .../python_api/api_def_CropAndResize.pbtxt | 6 + .../core/api_def/python_api/api_def_D.pbtxt | 74 - .../api_def_DebugGradientIdentity.pbtxt | 4 + .../api_def_DecodeAndCropJpeg.pbtxt | 6 + .../python_api/api_def_DecodeBmp.pbtxt | 6 + .../python_api/api_def_DecodeCSV.pbtxt | 4 + .../python_api/api_def_DecodeGif.pbtxt | 6 + .../python_api/api_def_DecodeJpeg.pbtxt | 6 + .../python_api/api_def_DecodePng.pbtxt | 6 + .../api_def_DeleteSessionTensor.pbtxt | 4 + .../api_def_DepthwiseConv2dNative.pbtxt | 6 + ..._DepthwiseConv2dNativeBackpropFilter.pbtxt | 6 + ...f_DepthwiseConv2dNativeBackpropInput.pbtxt | 6 + .../api_def_DeserializeManySparse.pbtxt | 4 + .../api_def_DestroyTemporaryVariable.pbtxt | 4 + .../api_def_DrawBoundingBoxes.pbtxt | 6 + .../core/api_def/python_api/api_def_E.pbtxt | 46 - .../python_api/api_def_EditDistance.pbtxt | 4 + .../core/api_def/python_api/api_def_Elu.pbtxt | 6 + .../python_api/api_def_EncodeJpeg.pbtxt | 6 + .../python_api/api_def_EncodePng.pbtxt | 6 + .../python_api/api_def_ExpandDims.pbtxt | 4 + .../python_api/api_def_ExtractGlimpse.pbtxt | 6 + .../python_api/api_def_ExtractJpegShape.pbtxt | 6 + .../core/api_def/python_api/api_def_F.pbtxt | 73 - .../core/api_def/python_api/api_def_FFT.pbtxt | 9 + .../python_api/api_def_FIFOQueue.pbtxt | 4 + .../python_api/api_def_FIFOQueueV2.pbtxt | 4 + .../api_def/python_api/api_def_Fact.pbtxt | 4 + .../python_api/api_def_FakeQueue.pbtxt | 4 + .../api_def_FixedLengthRecordReader.pbtxt | 4 + .../api_def_FixedLengthRecordReaderV2.pbtxt | 4 + ...api_def_FixedUnigramCandidateSampler.pbtxt | 4 + .../api_def/python_api/api_def_FloorDiv.pbtxt | 4 + .../api_def/python_api/api_def_FloorMod.pbtxt | 4 + .../api_def_FractionalAvgPool.pbtxt | 6 + .../api_def_FractionalMaxPool.pbtxt | 6 + .../python_api/api_def_FusedBatchNorm.pbtxt | 4 + .../python_api/api_def_FusedBatchNormV2.pbtxt | 4 + .../core/api_def/python_api/api_def_G.pbtxt | 16 - .../api_def_GenerateVocabRemapping.pbtxt | 4 + .../python_api/api_def_GetSessionHandle.pbtxt | 4 + .../api_def_GetSessionHandleV2.pbtxt | 4 + .../python_api/api_def_GetSessionTensor.pbtxt | 4 + .../core/api_def/python_api/api_def_H.pbtxt | 18 - .../api_def/python_api/api_def_HSVToRGB.pbtxt | 6 + .../python_api/api_def_HashTable.pbtxt | 4 + .../python_api/api_def_HashTableV2.pbtxt | 4 + .../api_def_HistogramFixedWidth.pbtxt | 4 + .../python_api/api_def_HistogramSummary.pbtxt | 4 + .../core/api_def/python_api/api_def_I.pbtxt | 55 - .../api_def/python_api/api_def_IFFT.pbtxt | 9 + .../python_api/api_def_IdentityReader.pbtxt | 4 + .../python_api/api_def_IdentityReaderV2.pbtxt | 4 + .../python_api/api_def_ImageSummary.pbtxt | 4 + .../api_def/python_api/api_def_InTopK.pbtxt | 4 + .../api_def/python_api/api_def_InTopKV2.pbtxt | 4 + .../python_api/api_def_InitializeTable.pbtxt | 4 + .../api_def_InitializeTableFromTextFile.pbtxt | 4 + ...pi_def_InitializeTableFromTextFileV2.pbtxt | 4 + .../api_def_InitializeTableV2.pbtxt | 4 + .../api_def/python_api/api_def_Invert.pbtxt | 6 + .../core/api_def/python_api/api_def_L.pbtxt | 96 - .../api_def/python_api/api_def_L2Loss.pbtxt | 6 + .../python_api/api_def_LMDBReader.pbtxt | 4 + .../core/api_def/python_api/api_def_LRN.pbtxt | 9 + ...i_def_LearnedUnigramCandidateSampler.pbtxt | 4 + .../python_api/api_def_LeftShift.pbtxt | 6 + .../api_def/python_api/api_def_LinSpace.pbtxt | 9 + .../api_def/python_api/api_def_ListDiff.pbtxt | 4 + .../api_def_LoadAndRemapMatrix.pbtxt | 4 + .../api_def_LogMatrixDeterminant.pbtxt | 4 + .../python_api/api_def_LogSoftmax.pbtxt | 4 + .../api_def_LogUniformCandidateSampler.pbtxt | 4 + .../api_def_LookupTableExport.pbtxt | 4 + .../api_def_LookupTableExportV2.pbtxt | 4 + .../python_api/api_def_LookupTableFind.pbtxt | 4 + .../api_def_LookupTableFindV2.pbtxt | 4 + .../api_def_LookupTableImport.pbtxt | 4 + .../api_def_LookupTableImportV2.pbtxt | 4 + .../api_def_LookupTableInsert.pbtxt | 4 + .../api_def_LookupTableInsertV2.pbtxt | 4 + .../python_api/api_def_LookupTableSize.pbtxt | 4 + .../api_def_LookupTableSizeV2.pbtxt | 4 + .../core/api_def/python_api/api_def_M.pbtxt | 174 -- .../api_def/python_api/api_def_MatMul.pbtxt | 4 + .../python_api/api_def_MatrixBandPart.pbtxt | 9 + .../api_def_MatrixDeterminant.pbtxt | 9 + .../python_api/api_def_MatrixDiag.pbtxt | 9 + .../python_api/api_def_MatrixDiagPart.pbtxt | 9 + .../python_api/api_def_MatrixInverse.pbtxt | 9 + .../python_api/api_def_MatrixSetDiag.pbtxt | 9 + .../python_api/api_def_MatrixSolve.pbtxt | 9 + .../python_api/api_def_MatrixSolveLs.pbtxt | 4 + .../api_def_MatrixTriangularSolve.pbtxt | 9 + .../core/api_def/python_api/api_def_Max.pbtxt | 4 + .../api_def/python_api/api_def_MaxPool.pbtxt | 4 + .../python_api/api_def_MaxPool3DGrad.pbtxt | 4 + .../api_def_MaxPool3DGradGrad.pbtxt | 4 + .../python_api/api_def_MaxPoolGradGrad.pbtxt | 4 + .../api_def_MaxPoolGradGradWithArgmax.pbtxt | 4 + .../python_api/api_def_MaxPoolV2.pbtxt | 4 + .../api_def_MaxPoolWithArgmax.pbtxt | 6 + .../api_def/python_api/api_def_Mean.pbtxt | 4 + .../api_def/python_api/api_def_Merge.pbtxt | 4 + .../python_api/api_def_MergeSummary.pbtxt | 4 + .../core/api_def/python_api/api_def_Min.pbtxt | 4 + .../python_api/api_def_MirrorPad.pbtxt | 4 + .../core/api_def/python_api/api_def_Mul.pbtxt | 4 + .../api_def_MutableDenseHashTable.pbtxt | 4 + .../api_def_MutableDenseHashTableV2.pbtxt | 4 + .../python_api/api_def_MutableHashTable.pbtxt | 4 + .../api_def_MutableHashTableOfTensors.pbtxt | 4 + .../api_def_MutableHashTableOfTensorsV2.pbtxt | 4 + .../api_def_MutableHashTableV2.pbtxt | 4 + .../core/api_def/python_api/api_def_N.pbtxt | 16 - .../core/api_def/python_api/api_def_Neg.pbtxt | 4 + .../api_def/python_api/api_def_NegTrain.pbtxt | 4 + .../api_def_NonMaxSuppression.pbtxt | 4 + .../api_def_NonMaxSuppressionV2.pbtxt | 4 + .../{api_def_O.pbtxt => api_def_OneHot.pbtxt} | 0 .../core/api_def/python_api/api_def_P.pbtxt | 68 - .../api_def/python_api/api_def_Pack.pbtxt | 4 + .../core/api_def/python_api/api_def_Pad.pbtxt | 4 + .../api_def/python_api/api_def_PadV2.pbtxt | 4 + .../python_api/api_def_PaddingFIFOQueue.pbtxt | 4 + .../api_def_PaddingFIFOQueueV2.pbtxt | 4 + .../python_api/api_def_ParallelConcat.pbtxt | 4 + ...api_def_ParameterizedTruncatedNormal.pbtxt | 4 + .../python_api/api_def_ParseExample.pbtxt | 4 + .../api_def_ParseSingleSequenceExample.pbtxt | 4 + .../python_api/api_def_Placeholder.pbtxt | 4 + .../core/api_def/python_api/api_def_Pow.pbtxt | 4 + .../api_def/python_api/api_def_Print.pbtxt | 4 + .../python_api/api_def_PriorityQueue.pbtxt | 4 + .../python_api/api_def_PriorityQueueV2.pbtxt | 4 + .../api_def/python_api/api_def_Prod.pbtxt | 4 + .../api_def/python_api/api_def_PyFunc.pbtxt | 4 + .../python_api/api_def_PyFuncStateless.pbtxt | 4 + .../core/api_def/python_api/api_def_Q.pbtxt | 83 - .../core/api_def/python_api/api_def_Qr.pbtxt | 9 + .../python_api/api_def_QuantizedAvgPool.pbtxt | 6 + .../python_api/api_def_QuantizedMaxPool.pbtxt | 6 + .../python_api/api_def_QuantizedReluX.pbtxt | 6 + .../python_api/api_def_QueueClose.pbtxt | 4 + .../python_api/api_def_QueueCloseV2.pbtxt | 4 + .../python_api/api_def_QueueDequeue.pbtxt | 4 + .../python_api/api_def_QueueDequeueMany.pbtxt | 4 + .../api_def_QueueDequeueManyV2.pbtxt | 4 + .../python_api/api_def_QueueDequeueUpTo.pbtxt | 4 + .../api_def_QueueDequeueUpToV2.pbtxt | 4 + .../python_api/api_def_QueueDequeueV2.pbtxt | 4 + .../python_api/api_def_QueueEnqueue.pbtxt | 4 + .../python_api/api_def_QueueEnqueueMany.pbtxt | 4 + .../api_def_QueueEnqueueManyV2.pbtxt | 4 + .../python_api/api_def_QueueEnqueueV2.pbtxt | 4 + .../python_api/api_def_QueueSize.pbtxt | 4 + .../python_api/api_def_QueueSizeV2.pbtxt | 4 + .../core/api_def/python_api/api_def_R.pbtxt | 192 -- .../api_def/python_api/api_def_RGBToHSV.pbtxt | 6 + .../python_api/api_def_RandomCrop.pbtxt | 4 + .../python_api/api_def_RandomGamma.pbtxt | 4 + .../python_api/api_def_RandomPoisson.pbtxt | 4 + .../python_api/api_def_RandomShuffle.pbtxt | 4 + .../api_def_RandomShuffleQueue.pbtxt | 4 + .../api_def_RandomShuffleQueueV2.pbtxt | 4 + .../api_def_RandomStandardNormal.pbtxt | 4 + .../python_api/api_def_RandomUniform.pbtxt | 4 + .../python_api/api_def_RandomUniformInt.pbtxt | 4 + .../api_def/python_api/api_def_Range.pbtxt | 4 + .../api_def_ReaderNumRecordsProduced.pbtxt | 4 + .../api_def_ReaderNumRecordsProducedV2.pbtxt | 4 + .../api_def_ReaderNumWorkUnitsCompleted.pbtxt | 4 + ...pi_def_ReaderNumWorkUnitsCompletedV2.pbtxt | 4 + .../python_api/api_def_ReaderRead.pbtxt | 4 + .../python_api/api_def_ReaderReadUpTo.pbtxt | 4 + .../python_api/api_def_ReaderReadUpToV2.pbtxt | 4 + .../python_api/api_def_ReaderReadV2.pbtxt | 4 + .../python_api/api_def_ReaderReset.pbtxt | 4 + .../python_api/api_def_ReaderResetV2.pbtxt | 4 + .../api_def_ReaderRestoreState.pbtxt | 4 + .../api_def_ReaderRestoreStateV2.pbtxt | 4 + .../api_def_ReaderSerializeState.pbtxt | 4 + .../api_def_ReaderSerializeStateV2.pbtxt | 4 + .../api_def/python_api/api_def_RealDiv.pbtxt | 4 + .../api_def/python_api/api_def_Relu.pbtxt | 6 + .../api_def/python_api/api_def_Relu6.pbtxt | 4 + .../python_api/api_def_ResizeArea.pbtxt | 6 + .../python_api/api_def_ResizeBicubic.pbtxt | 6 + .../python_api/api_def_ResizeBilinear.pbtxt | 6 + .../api_def_ResizeNearestNeighbor.pbtxt | 6 + .../api_def/python_api/api_def_Restore.pbtxt | 4 + .../python_api/api_def_RestoreSlice.pbtxt | 4 + .../api_def/python_api/api_def_Reverse.pbtxt | 4 + .../python_api/api_def_ReverseV2.pbtxt | 6 + .../python_api/api_def_RightShift.pbtxt | 6 + .../core/api_def/python_api/api_def_S.pbtxt | 252 -- .../api_def_SampleDistortedBoundingBox.pbtxt | 4 + ...api_def_SampleDistortedBoundingBoxV2.pbtxt | 4 + .../api_def/python_api/api_def_Save.pbtxt | 4 + .../python_api/api_def_SaveSlices.pbtxt | 4 + .../python_api/api_def_ScalarSummary.pbtxt | 4 + .../python_api/api_def_SdcaFprint.pbtxt | 6 + .../python_api/api_def_SdcaOptimizer.pbtxt | 6 + .../python_api/api_def_SdcaShrinkL1.pbtxt | 6 + .../api_def/python_api/api_def_Select.pbtxt | 4 + .../python_api/api_def_SelfAdjointEig.pbtxt | 4 + .../python_api/api_def_SelfAdjointEigV2.pbtxt | 4 + .../api_def/python_api/api_def_Selu.pbtxt | 6 + .../api_def_SerializeManySparse.pbtxt | 4 + .../python_api/api_def_SerializeSparse.pbtxt | 4 + .../python_api/api_def_ShardedFilename.pbtxt | 4 + .../python_api/api_def_ShardedFilespec.pbtxt | 4 + .../api_def/python_api/api_def_Sigmoid.pbtxt | 4 + .../api_def/python_api/api_def_Skipgram.pbtxt | 4 + .../api_def/python_api/api_def_Slice.pbtxt | 4 + .../api_def/python_api/api_def_Softmax.pbtxt | 4 + ...pi_def_SoftmaxCrossEntropyWithLogits.pbtxt | 4 + .../api_def/python_api/api_def_Softplus.pbtxt | 6 + .../api_def/python_api/api_def_Softsign.pbtxt | 6 + .../python_api/api_def_SpaceToBatch.pbtxt | 4 + .../python_api/api_def_SparseAdd.pbtxt | 4 + .../python_api/api_def_SparseAddGrad.pbtxt | 4 + .../python_api/api_def_SparseConcat.pbtxt | 4 + .../python_api/api_def_SparseCross.pbtxt | 4 + .../api_def_SparseFillEmptyRows.pbtxt | 4 + .../api_def_SparseFillEmptyRowsGrad.pbtxt | 4 + .../python_api/api_def_SparseMatMul.pbtxt | 4 + .../python_api/api_def_SparseReorder.pbtxt | 4 + .../python_api/api_def_SparseReshape.pbtxt | 4 + ..._SparseSoftmaxCrossEntropyWithLogits.pbtxt | 4 + .../python_api/api_def_SparseSplit.pbtxt | 4 + .../api_def_SparseTensorDenseAdd.pbtxt | 4 + .../api_def_SparseTensorDenseMatMul.pbtxt | 4 + .../python_api/api_def_SparseToDense.pbtxt | 4 + .../api_def/python_api/api_def_Split.pbtxt | 4 + .../api_def/python_api/api_def_SplitV.pbtxt | 4 + .../api_def/python_api/api_def_Squeeze.pbtxt | 4 + .../api_def/python_api/api_def_Stack.pbtxt | 4 + .../python_api/api_def_StackClose.pbtxt | 4 + .../python_api/api_def_StackCloseV2.pbtxt | 4 + .../api_def/python_api/api_def_StackPop.pbtxt | 4 + .../python_api/api_def_StackPopV2.pbtxt | 4 + .../python_api/api_def_StackPush.pbtxt | 4 + .../python_api/api_def_StackPushV2.pbtxt | 4 + .../api_def/python_api/api_def_StackV2.pbtxt | 4 + .../python_api/api_def_StringSplit.pbtxt | 4 + .../core/api_def/python_api/api_def_Sub.pbtxt | 4 + .../core/api_def/python_api/api_def_Sum.pbtxt | 4 + .../core/api_def/python_api/api_def_Svd.pbtxt | 4 + .../api_def/python_api/api_def_Switch.pbtxt | 4 + .../python_api/api_def_SymbolicGradient.pbtxt | 4 + .../core/api_def/python_api/api_def_T.pbtxt | 196 -- .../python_api/api_def_TFRecordReader.pbtxt | 4 + .../python_api/api_def_TFRecordReaderV2.pbtxt | 4 + ...api_def_TakeManySparseFromTensorsMap.pbtxt | 4 + .../api_def/python_api/api_def_Tanh.pbtxt | 4 + .../api_def_TemporaryVariable.pbtxt | 4 + .../python_api/api_def_TensorArray.pbtxt | 4 + .../python_api/api_def_TensorArrayClose.pbtxt | 4 + .../api_def_TensorArrayCloseV2.pbtxt | 4 + .../api_def_TensorArrayCloseV3.pbtxt | 4 + .../api_def_TensorArrayConcat.pbtxt | 4 + .../api_def_TensorArrayConcatV2.pbtxt | 4 + .../api_def_TensorArrayConcatV3.pbtxt | 4 + .../api_def_TensorArrayGather.pbtxt | 4 + .../api_def_TensorArrayGatherV2.pbtxt | 4 + .../api_def_TensorArrayGatherV3.pbtxt | 4 + .../python_api/api_def_TensorArrayGrad.pbtxt | 4 + .../api_def_TensorArrayGradV2.pbtxt | 4 + .../api_def_TensorArrayGradV3.pbtxt | 4 + .../python_api/api_def_TensorArrayPack.pbtxt | 4 + .../python_api/api_def_TensorArrayRead.pbtxt | 4 + .../api_def_TensorArrayReadV2.pbtxt | 4 + .../api_def_TensorArrayReadV3.pbtxt | 4 + .../api_def_TensorArrayScatter.pbtxt | 4 + .../api_def_TensorArrayScatterV2.pbtxt | 4 + .../api_def_TensorArrayScatterV3.pbtxt | 4 + .../python_api/api_def_TensorArraySize.pbtxt | 4 + .../api_def_TensorArraySizeV2.pbtxt | 4 + .../api_def_TensorArraySizeV3.pbtxt | 4 + .../python_api/api_def_TensorArraySplit.pbtxt | 4 + .../api_def_TensorArraySplitV2.pbtxt | 4 + .../api_def_TensorArraySplitV3.pbtxt | 4 + .../api_def_TensorArrayUnpack.pbtxt | 4 + .../python_api/api_def_TensorArrayV2.pbtxt | 4 + .../python_api/api_def_TensorArrayV3.pbtxt | 4 + .../python_api/api_def_TensorArrayWrite.pbtxt | 4 + .../api_def_TensorArrayWriteV2.pbtxt | 4 + .../api_def_TensorArrayWriteV3.pbtxt | 4 + .../python_api/api_def_TensorSummary.pbtxt | 4 + .../python_api/api_def_TensorSummaryV2.pbtxt | 4 + .../python_api/api_def_TextLineReader.pbtxt | 4 + .../python_api/api_def_TextLineReaderV2.pbtxt | 4 + ..._ThreadUnsafeUnigramCandidateSampler.pbtxt | 4 + .../api_def/python_api/api_def_TileGrad.pbtxt | 4 + .../api_def/python_api/api_def_TopK.pbtxt | 4 + .../api_def/python_api/api_def_TopKV2.pbtxt | 4 + .../python_api/api_def_TruncateDiv.pbtxt | 4 + .../python_api/api_def_TruncateMod.pbtxt | 4 + .../python_api/api_def_TruncatedNormal.pbtxt | 4 + ... => api_def_UniformCandidateSampler.pbtxt} | 4 - .../api_def/python_api/api_def_Unpack.pbtxt | 4 + .../api_def/python_api/api_def_Variable.pbtxt | 4 + ...i_def_V.pbtxt => api_def_VariableV2.pbtxt} | 4 - .../python_api/api_def_WholeFileReader.pbtxt | 4 + ....pbtxt => api_def_WholeFileReaderV2.pbtxt} | 4 - ...pi_def_Z.pbtxt => api_def_ZerosLike.pbtxt} | 0 .../core/common_runtime/gpu/gpu_device.cc | 32 +- tensorflow/core/framework/shape_inference.h | 1 + tensorflow/core/graph/graph_constructor.cc | 121 +- tensorflow/core/graph/graph_constructor.h | 11 +- .../core/graph/graph_constructor_test.cc | 134 +- .../core/grappler/costs/graph_properties.cc | 19 +- .../core/grappler/costs/graph_properties.h | 14 +- .../grappler/costs/graph_properties_test.cc | 3 +- .../grappler/costs/op_level_cost_estimator.cc | 2 +- .../optimizers/arithmetic_optimizer.cc | 141 +- .../optimizers/arithmetic_optimizer_test.cc | 76 +- .../grappler/optimizers/constant_folding.cc | 4 +- .../grappler/optimizers/layout_optimizer.cc | 130 +- .../grappler/optimizers/layout_optimizer.h | 16 +- .../optimizers/layout_optimizer_test.cc | 28 + .../grappler/optimizers/meta_optimizer.cc | 62 +- .../core/grappler/optimizers/meta_optimizer.h | 3 + tensorflow/core/grappler/utils.cc | 5 +- tensorflow/core/kernels/BUILD | 7 + tensorflow/core/kernels/dataset.h | 25 +- .../kernels/generate_vocab_remapping_op.cc | 21 +- tensorflow/core/kernels/iterator_ops.cc | 6 +- .../core/kernels/matrix_exponential_op.cc | 59 + .../core/kernels/prefetch_dataset_op.cc | 23 +- tensorflow/core/kernels/tensor_array.h | 34 +- tensorflow/core/ops/checkpoint_ops.cc | 9 +- .../core/ops/compat/ops_history.v1.pbtxt | 38 + tensorflow/core/ops/linalg_ops.cc | 27 + tensorflow/core/ops/ops.pbtxt | 12 +- tensorflow/go/op/wrappers.go | 27 +- tensorflow/python/BUILD | 28 +- tensorflow/python/eager/BUILD | 16 +- tensorflow/python/eager/benchmarks_test.py | 411 ++- tensorflow/python/eager/pywrap_tensor.cc | 2 - tensorflow/python/estimator/canned/head.py | 55 +- .../python/estimator/canned/head_test.py | 160 +- .../python/estimator/warm_starting_util.py | 43 +- .../estimator/warm_starting_util_test.py | 71 + tensorflow/python/framework/c_api_util.py | 31 +- .../python/framework/graph_util_impl.py | 89 +- tensorflow/python/framework/importer.py | 33 +- tensorflow/python/framework/importer_test.py | 111 + tensorflow/python/framework/ops.py | 6 +- .../python/grappler/layout_optimizer_test.py | 32 + tensorflow/python/grappler/tf_optimizer.i | 5 +- tensorflow/python/keras/BUILD | 14 + tensorflow/python/keras/__init__.py | 2 + .../python/keras/_impl/keras/__init__.py | 2 + .../_impl/keras/applications/__init__.py | 1 + .../keras/applications/imagenet_utils.py | 21 +- .../keras/applications/inception_resnet_v2.py | 369 +++ .../applications/inception_resnet_v2_test.py | 59 + .../_impl/keras/applications/inception_v3.py | 18 +- .../_impl/keras/applications/mobilenet.py | 14 +- .../_impl/keras/applications/resnet50.py | 4 +- .../keras/_impl/keras/applications/vgg16.py | 6 +- .../keras/_impl/keras/applications/vgg19.py | 6 +- .../_impl/keras/applications/xception.py | 20 +- .../python/keras/_impl/keras/backend.py | 11 +- .../python/keras/applications/__init__.py | 2 + .../inception_resnet_v2/__init__.py | 27 + tensorflow/python/kernel_tests/BUILD | 26 + .../kernel_tests/checkpoint_ops_test.py | 15 + .../kernel_tests/depthtospace_op_test.py | 31 +- .../distributions/special_math_test.py | 16 + .../kernel_tests/garbage_collection_test.py | 88 + .../matrix_exponential_op_test.py | 196 ++ .../kernel_tests/spacetodepth_op_test.py | 27 +- .../kernel_tests/tensor_array_ops_test.py | 26 +- tensorflow/python/ops/array_grad.py | 12 +- .../python/ops/distributions/special_math.py | 6 +- tensorflow/python/ops/hidden_ops.txt | 1 + tensorflow/python/ops/linalg/linalg_impl.py | 1 + tensorflow/python/ops/math_ops.py | 1 + .../python/ops/resource_variable_ops.py | 22 +- tensorflow/python/ops/tensor_array_ops.py | 53 +- tensorflow/python/platform/app.py | 103 +- tensorflow/python/platform/flags.py | 195 +- tensorflow/python/platform/flags_test.py | 97 +- tensorflow/python/tools/BUILD | 1 - tensorflow/python/training/checkpoint_ops.py | 25 +- .../python/training/checkpoint_ops_test.py | 81 +- tensorflow/python/training/moving_averages.py | 7 +- .../python/training/moving_averages_test.py | 27 + tensorflow/python/util/tf_should_use.py | 14 +- tensorflow/tensorflow.bzl | 22 + .../api/golden/tensorflow.keras.-model.pbtxt | 269 ++ .../golden/tensorflow.keras.-sequential.pbtxt | 294 ++ ...ras.applications.inception_resnet_v2.pbtxt | 15 + .../tensorflow.keras.applications.pbtxt | 8 + ...nsorflow.keras.applications.resnet50.pbtxt | 2 +- .../tensorflow.keras.applications.vgg16.pbtxt | 2 +- .../tensorflow.keras.applications.vgg19.pbtxt | 2 +- .../tools/api/golden/tensorflow.keras.pbtxt | 8 + .../tools/api/golden/tensorflow.linalg.pbtxt | 4 + .../tools/api/tests/api_compatibility_test.py | 76 +- .../ci_build/install/install_pip_packages.sh | 4 + .../install/install_python3.5_pip_packages.sh | 1 + .../ci_build/windows/cpu/cmake/run_py.bat | 3 + .../ci_build/windows/gpu/cmake/run_py.bat | 3 + tensorflow/tools/docs/generate_lib.py | 31 +- tensorflow/tools/pip_package/setup.py | 1 + tensorflow/workspace.bzl | 17 +- third_party/eigen.BUILD | 2 +- third_party/eigen3/BUILD | 1 + .../eigen3/unsupported/Eigen/MatrixFunctions | 1 + 1305 files changed, 35474 insertions(+), 14134 deletions(-) create mode 100644 tensorflow/contrib/estimator/python/estimator/replicate_model_fn.py create mode 100644 tensorflow/contrib/estimator/python/estimator/replicate_model_fn_test.py create mode 100644 tensorflow/contrib/framework/python/framework/graph_util.py create mode 100644 tensorflow/contrib/framework/python/framework/graph_util_test.py create mode 100644 tensorflow/contrib/framework/python/ops/sort_ops.py create mode 100644 tensorflow/contrib/framework/python/ops/sort_ops_test.py create mode 100644 tensorflow/contrib/tpu/profiler/tf_op_stats.proto delete mode 100644 tensorflow/core/api_def/base_api/api_def_A.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Abort.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Abs.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AccumulateNV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AccumulatorApplyGradient.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AccumulatorNumAccumulated.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AccumulatorSetGlobalStep.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AccumulatorTakeGradient.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Acos.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Acosh.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Add.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AddManySparseToTensorsMap.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AddN.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AddSparseToTensorsMap.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AddV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AdjustContrast.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AdjustContrastv2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AdjustHue.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AdjustSaturation.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_All.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AllCandidateSampler.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Angle.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Any.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApplyAdadelta.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApplyAdagrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApplyAdagradDA.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApplyAdam.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApplyCenteredRMSProp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApplyFtrl.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApplyFtrlV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApplyGradientDescent.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApplyMomentum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApplyProximalAdagrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApplyProximalGradientDescent.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApplyRMSProp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ApproximateEqual.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ArgMax.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ArgMin.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AsString.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Asin.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Asinh.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Assert.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Assign.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AssignAdd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AssignAddVariableOp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AssignSub.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AssignSubVariableOp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AssignVariableOp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Atan.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Atan2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Atanh.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AudioSpectrogram.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AudioSummary.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AudioSummaryV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AvgPool.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AvgPool3D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AvgPool3DGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_AvgPoolGrad.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_B.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Barrier.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BarrierClose.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BarrierIncompleteSize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BarrierInsertMany.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BarrierReadySize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BarrierTakeMany.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchCholesky.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchCholeskyGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchFFT.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchFFT2D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchFFT3D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchIFFT.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchIFFT2D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchIFFT3D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchMatMul.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchMatrixBandPart.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchMatrixDeterminant.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchMatrixDiag.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchMatrixDiagPart.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchMatrixInverse.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchMatrixSetDiag.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchMatrixSolve.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchMatrixSolveLs.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchMatrixTriangularSolve.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchNormWithGlobalNormalization.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchNormWithGlobalNormalizationGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchSelfAdjointEig.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchSelfAdjointEigV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchSvd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchToSpace.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BatchToSpaceND.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Betainc.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BiasAdd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BiasAddGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BiasAddV1.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Bincount.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Bitcast.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BitwiseAnd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BitwiseOr.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BitwiseXor.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BroadcastArgs.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_BroadcastGradientArgs.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Bucketize.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_C.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_CTCBeamSearchDecoder.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_CTCGreedyDecoder.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_CTCLoss.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_CacheDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Cast.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Ceil.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_CheckNumerics.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Cholesky.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_CholeskyGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_CompareAndBitpack.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Complex.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ComplexAbs.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ComputeAccidentalHits.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Concat.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ConcatOffset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ConcatV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ConcatenateDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ConditionalAccumulator.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Conj.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ConjugateTranspose.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Const.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ControlTrigger.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Conv2D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Conv2DBackpropFilter.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Conv2DBackpropInput.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Conv3D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Conv3DBackpropFilter.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Conv3DBackpropFilterV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Conv3DBackpropInput.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Conv3DBackpropInputV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Cos.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Cosh.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_CountUpTo.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_CropAndResize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_CropAndResizeGradBoxes.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_CropAndResizeGradImage.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Cross.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Cumprod.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Cumsum.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DatasetToSingleElement.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DebugGradientIdentity.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DecodeAndCropJpeg.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DecodeBase64.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DecodeBmp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DecodeCSV.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DecodeGif.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DecodeJSONExample.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DecodeJpeg.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DecodePng.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DecodeRaw.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DecodeWav.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DeleteSessionTensor.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DenseToDenseSetOperation.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DenseToSparseBatchDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DenseToSparseSetOperation.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DepthToSpace.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DepthwiseConv2dNative.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DepthwiseConv2dNativeBackpropFilter.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DepthwiseConv2dNativeBackpropInput.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Dequantize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DeserializeIterator.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DeserializeManySparse.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DestroyResourceOp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DestroyTemporaryVariable.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Diag.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DiagPart.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Digamma.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Dilation2D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Dilation2DBackpropFilter.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Dilation2DBackpropInput.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Div.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DrawBoundingBoxes.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DynamicPartition.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_DynamicStitch.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_E.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_EditDistance.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Elu.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_EluGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_EncodeBase64.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_EncodeJpeg.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_EncodePng.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_EncodeWav.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Enter.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Equal.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Erf.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Erfc.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Exit.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Exp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ExpandDims.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Expm1.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ExtractGlimpse.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ExtractImagePatches.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ExtractJpegShape.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_F.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FFT.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FFT2D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FFT3D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FIFOQueue.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FIFOQueueV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Fact.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FakeQuantWithMinMaxArgs.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FakeQuantWithMinMaxArgsGradient.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FakeQuantWithMinMaxVars.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FakeQuantWithMinMaxVarsGradient.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FakeQuantWithMinMaxVarsPerChannel.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FakeQuantWithMinMaxVarsPerChannelGradient.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FakeQueue.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Fill.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FilterDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FixedLengthRecordDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FixedLengthRecordReader.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FixedLengthRecordReaderV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FixedUnigramCandidateSampler.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FlatMapDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Floor.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FloorDiv.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FloorMod.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FractionalAvgPool.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FractionalAvgPoolGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FractionalMaxPool.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FractionalMaxPoolGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FusedBatchNorm.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FusedBatchNormGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FusedBatchNormGradV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FusedBatchNormV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FusedPadConv2D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_FusedResizeAndPadConv2D.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_G.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Gather.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_GatherNd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_GatherV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_GenerateVocabRemapping.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_GetSessionHandle.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_GetSessionHandleV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_GetSessionTensor.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Greater.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_GreaterEqual.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_GroupByWindowDataset.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_H.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_HSVToRGB.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_HashTable.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_HashTableV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_HistogramFixedWidth.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_HistogramSummary.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_I.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IFFT.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IFFT2D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IFFT3D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IRFFT.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IRFFT2D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IRFFT3D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Identity.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IdentityN.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IdentityReader.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IdentityReaderV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Igamma.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Igammac.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IgnoreErrorsDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Imag.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ImageSummary.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ImmutableConst.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_InTopK.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_InTopKV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_InitializeTable.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_InitializeTableFromTextFile.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_InitializeTableFromTextFileV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_InitializeTableV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_InterleaveDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Inv.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_InvGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Invert.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_InvertPermutation.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IsFinite.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IsInf.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IsNan.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IsVariableInitialized.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Iterator.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IteratorFromStringHandle.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IteratorGetNext.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_IteratorToStringHandle.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_L.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_L2Loss.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LMDBReader.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LRN.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LRNGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LearnedUnigramCandidateSampler.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LeftShift.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Less.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LessEqual.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Lgamma.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LinSpace.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ListDiff.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LoadAndRemapMatrix.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Log.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Log1p.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LogMatrixDeterminant.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LogSoftmax.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LogUniformCandidateSampler.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LogicalAnd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LogicalNot.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LogicalOr.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LookupTableExport.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LookupTableExportV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LookupTableFind.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LookupTableFindV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LookupTableImport.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LookupTableImportV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LookupTableInsert.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LookupTableInsertV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LookupTableSize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LookupTableSizeV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_LoopCond.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_M.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MakeIterator.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MapAndBatchDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MapClear.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MapDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MapIncompleteSize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MapPeek.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MapSize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MapStage.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MapUnstage.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MapUnstageNoKey.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MatMul.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MatchingFiles.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MatrixBandPart.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MatrixDeterminant.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MatrixDiag.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MatrixDiagPart.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MatrixInverse.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MatrixSetDiag.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MatrixSolve.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MatrixSolveLs.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MatrixTriangularSolve.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Max.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MaxPool.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MaxPool3D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MaxPool3DGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MaxPool3DGradGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MaxPoolGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MaxPoolGradGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MaxPoolGradGradV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MaxPoolGradGradWithArgmax.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MaxPoolGradV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MaxPoolGradWithArgmax.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MaxPoolV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MaxPoolWithArgmax.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Maximum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Mean.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Merge.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MergeSummary.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MergeV2Checkpoints.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Mfcc.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Min.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Minimum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MirrorPad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MirrorPadGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Mod.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Mul.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Multinomial.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MutableDenseHashTable.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MutableDenseHashTableV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MutableHashTable.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MutableHashTableOfTensors.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MutableHashTableOfTensorsV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_MutableHashTableV2.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_N.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Neg.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_NegTrain.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_NextIteration.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_NoOp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_NonMaxSuppression.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_NonMaxSuppressionV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_NotEqual.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_NthElement.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_O.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_OneHot.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_OneShotIterator.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_OnesLike.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_OrderedMapClear.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_OrderedMapIncompleteSize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_OrderedMapPeek.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_OrderedMapSize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_OrderedMapStage.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_OrderedMapUnstage.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_OrderedMapUnstageNoKey.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_P.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Pack.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Pad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PadV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PaddedBatchDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PaddingFIFOQueue.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PaddingFIFOQueueV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ParallelConcat.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ParallelDynamicStitch.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ParallelInterleaveDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ParallelMapDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ParameterizedTruncatedNormal.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ParseExample.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ParseSingleSequenceExample.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ParseTensor.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Placeholder.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PlaceholderV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PlaceholderWithDefault.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Polygamma.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PopulationCount.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Pow.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PrefetchDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PreventGradient.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Print.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PriorityQueue.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PriorityQueueV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Prod.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PyFunc.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_PyFuncStateless.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_Q.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Qr.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizeAndDequantize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizeAndDequantizeV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizeAndDequantizeV3.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizeDownAndShrinkRange.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizeV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedAdd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedAvgPool.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedBatchNormWithGlobalNormalization.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedBiasAdd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedConcat.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedConv2D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedInstanceNorm.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedMatMul.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedMaxPool.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedMul.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedRelu.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedRelu6.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedReluX.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedReshape.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QuantizedResizeBilinear.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueClose.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueCloseV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueDequeue.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueDequeueMany.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueDequeueManyV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueDequeueUpTo.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueDequeueUpToV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueDequeueV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueEnqueue.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueEnqueueMany.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueEnqueueManyV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueEnqueueV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueIsClosed.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueIsClosedV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueSize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_QueueSizeV2.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_R.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RFFT.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RFFT2D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RFFT3D.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RGBToHSV.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RandomCrop.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RandomGamma.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RandomPoisson.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RandomPoissonV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RandomShuffle.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RandomShuffleQueue.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RandomShuffleQueueV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RandomStandardNormal.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RandomUniform.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RandomUniformInt.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Range.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RangeDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Rank.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReadFile.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReadVariableOp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderNumRecordsProduced.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderNumRecordsProducedV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderNumWorkUnitsCompleted.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderNumWorkUnitsCompletedV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderRead.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderReadUpTo.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderReadUpToV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderReadV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderReset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderResetV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderRestoreState.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderRestoreStateV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderSerializeState.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReaderSerializeStateV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Real.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RealDiv.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Reciprocal.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReciprocalGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RecordInput.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReduceJoin.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RefEnter.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RefExit.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RefIdentity.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RefMerge.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RefNextIteration.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RefSelect.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RefSwitch.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Relu.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Relu6.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Relu6Grad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReluGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RemoteCall.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RemoteFusedGraphExecute.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RepeatDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RequantizationRange.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Requantize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Reshape.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResizeArea.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResizeBicubic.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResizeBicubicGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResizeBilinear.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResizeBilinearGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResizeNearestNeighbor.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResizeNearestNeighborGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceApplyAdadelta.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceApplyAdagrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceApplyAdagradDA.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceApplyAdam.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceApplyCenteredRMSProp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceApplyFtrl.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceApplyFtrlV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceApplyGradientDescent.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceApplyMomentum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceApplyProximalAdagrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceApplyProximalGradientDescent.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceApplyRMSProp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceCountUpTo.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceGather.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceScatterAdd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceScatterUpdate.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceSparseApplyAdadelta.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceSparseApplyAdagrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceSparseApplyAdagradDA.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceSparseApplyCenteredRMSProp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceSparseApplyFtrl.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceSparseApplyFtrlV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceSparseApplyMomentum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceSparseApplyProximalAdagrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceSparseApplyProximalGradientDescent.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceSparseApplyRMSProp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ResourceStridedSliceAssign.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Restore.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RestoreSlice.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RestoreV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Reverse.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReverseSequence.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ReverseV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RightShift.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Rint.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Round.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Rsqrt.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_RsqrtGrad.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_S.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SampleDistortedBoundingBox.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SampleDistortedBoundingBoxV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Save.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SaveSlices.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SaveV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ScalarSummary.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ScanDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ScatterAdd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ScatterDiv.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ScatterMul.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ScatterNd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ScatterNdAdd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ScatterNdNonAliasingAdd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ScatterNdSub.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ScatterNdUpdate.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ScatterSub.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ScatterUpdate.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SdcaFprint.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SdcaOptimizer.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SdcaShrinkL1.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SegmentMax.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SegmentMean.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SegmentMin.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SegmentProd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SegmentSum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Select.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SelfAdjointEig.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SelfAdjointEigV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Selu.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SeluGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SerializeIterator.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SerializeManySparse.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SerializeSparse.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SerializeTensor.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SetSize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Shape.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ShapeN.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ShardedFilename.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ShardedFilespec.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ShuffleDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Sigmoid.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SigmoidGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Sign.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Sin.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Sinh.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Size.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SkipDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Skipgram.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Slice.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Softmax.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SoftmaxCrossEntropyWithLogits.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Softplus.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SoftplusGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Softsign.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SoftsignGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SpaceToBatch.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SpaceToBatchND.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SpaceToDepth.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseAccumulatorApplyGradient.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseAccumulatorTakeGradient.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseAdd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseAddGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseApplyAdadelta.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseApplyAdagrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseApplyAdagradDA.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseApplyCenteredRMSProp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseApplyFtrl.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseApplyFtrlV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseApplyMomentum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseApplyProximalAdagrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseApplyProximalGradientDescent.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseApplyRMSProp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseConcat.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseConditionalAccumulator.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseCross.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseDenseCwiseAdd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseDenseCwiseDiv.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseDenseCwiseMul.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseFillEmptyRows.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseFillEmptyRowsGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseMatMul.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseReduceMax.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseReduceMaxSparse.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseReduceSum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseReduceSumSparse.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseReorder.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseReshape.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseSegmentMean.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseSegmentMeanGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseSegmentSqrtN.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseSegmentSqrtNGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseSegmentSum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseSlice.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseSoftmax.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseSoftmaxCrossEntropyWithLogits.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseSparseMaximum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseSparseMinimum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseSplit.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseTensorDenseAdd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseTensorDenseMatMul.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseTensorSliceDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseToDense.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SparseToSparseSetOperation.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Split.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SplitV.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SqlDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Sqrt.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SqrtGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Square.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SquaredDifference.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Squeeze.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Stack.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StackClose.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StackCloseV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StackPop.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StackPopV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StackPush.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StackPushV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StackV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Stage.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StageClear.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StagePeek.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StageSize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StatelessRandomNormal.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StatelessRandomUniform.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StatelessTruncatedNormal.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StopGradient.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StridedSlice.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StridedSliceAssign.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StridedSliceGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StringJoin.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StringSplit.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StringToHashBucket.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StringToHashBucketFast.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StringToHashBucketStrong.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_StringToNumber.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Sub.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Substr.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Sum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Svd.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Switch.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_SymbolicGradient.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_T.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TFRecordDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TFRecordReader.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TFRecordReaderV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TakeDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TakeManySparseFromTensorsMap.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Tan.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Tanh.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TanhGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TemporaryVariable.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArray.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayClose.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayCloseV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayCloseV3.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayConcat.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayConcatV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayConcatV3.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayGather.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayGatherV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayGatherV3.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayGradV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayGradV3.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayPack.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayRead.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayReadV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayReadV3.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayScatter.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayScatterV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayScatterV3.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArraySize.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArraySizeV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArraySizeV3.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArraySplit.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArraySplitV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArraySplitV3.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayUnpack.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayV3.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayWrite.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayWriteV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorArrayWriteV3.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorSliceDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorSummary.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TensorSummaryV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TextLineDataset.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TextLineReader.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TextLineReaderV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ThreadUnsafeUnigramCandidateSampler.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Tile.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TileGrad.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TopK.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TopKV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Transpose.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TruncateDiv.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TruncateMod.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_TruncatedNormal.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_U.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_UniformCandidateSampler.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Unique.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_UniqueWithCounts.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Unpack.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_UnsortedSegmentMax.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_UnsortedSegmentSum.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Unstage.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_V.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_VarHandleOp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_VarIsInitializedOp.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Variable.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_VariableShape.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_VariableV2.pbtxt rename tensorflow/core/api_def/base_api/{api_def_W.pbtxt => api_def_Where.pbtxt} (51%) create mode 100644 tensorflow/core/api_def/base_api/api_def_WholeFileReader.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_WholeFileReaderV2.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_WriteFile.pbtxt delete mode 100644 tensorflow/core/api_def/base_api/api_def_Z.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ZerosLike.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_Zeta.pbtxt create mode 100644 tensorflow/core/api_def/base_api/api_def_ZipDataset.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_A.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Abs.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_AccumulateNV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_AddManySparseToTensorsMap.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_AddN.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_AddSparseToTensorsMap.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_AddV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_AdjustContrastv2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_All.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_AllCandidateSampler.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Any.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Assert.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_AudioSummary.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_AudioSummaryV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_AvgPool.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_AvgPool3DGrad.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_B.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Barrier.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BarrierClose.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BarrierIncompleteSize.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BarrierInsertMany.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BarrierReadySize.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BarrierTakeMany.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchCholesky.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchCholeskyGrad.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchFFT.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchFFT2D.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchFFT3D.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchIFFT.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchIFFT2D.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchIFFT3D.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchMatMul.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchMatrixDeterminant.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchMatrixInverse.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchMatrixSolve.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchMatrixSolveLs.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchMatrixTriangularSolve.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchNormWithGlobalNormalization.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchNormWithGlobalNormalizationGrad.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchSelfAdjointEig.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchSelfAdjointEigV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchSvd.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BatchToSpace.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BiasAdd.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BiasAddV1.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BitwiseAnd.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BitwiseOr.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BitwiseXor.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_BroadcastArgs.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Bucketize.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_C.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_CTCBeamSearchDecoder.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_CTCGreedyDecoder.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_CTCLoss.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Cholesky.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Complex.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ComplexAbs.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ComputeAccidentalHits.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Concat.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ConcatOffset.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ConcatV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Conj.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ConjugateTranspose.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Const.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_CropAndResize.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_D.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DebugGradientIdentity.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DecodeAndCropJpeg.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DecodeBmp.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DecodeCSV.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DecodeGif.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DecodeJpeg.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DecodePng.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DeleteSessionTensor.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DepthwiseConv2dNative.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DepthwiseConv2dNativeBackpropFilter.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DepthwiseConv2dNativeBackpropInput.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DeserializeManySparse.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DestroyTemporaryVariable.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_DrawBoundingBoxes.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_E.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_EditDistance.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Elu.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_EncodeJpeg.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_EncodePng.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ExpandDims.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ExtractGlimpse.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ExtractJpegShape.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_F.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FFT.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FIFOQueue.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FIFOQueueV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Fact.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FakeQueue.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FixedLengthRecordReader.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FixedLengthRecordReaderV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FixedUnigramCandidateSampler.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FloorDiv.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FloorMod.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FractionalAvgPool.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FractionalMaxPool.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FusedBatchNorm.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_FusedBatchNormV2.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_G.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_GenerateVocabRemapping.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_GetSessionHandle.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_GetSessionHandleV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_GetSessionTensor.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_H.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_HSVToRGB.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_HashTable.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_HashTableV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_HistogramFixedWidth.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_HistogramSummary.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_I.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_IFFT.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_IdentityReader.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_IdentityReaderV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ImageSummary.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_InTopK.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_InTopKV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_InitializeTable.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_InitializeTableFromTextFile.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_InitializeTableFromTextFileV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_InitializeTableV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Invert.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_L.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_L2Loss.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LMDBReader.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LRN.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LearnedUnigramCandidateSampler.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LeftShift.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LinSpace.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ListDiff.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LoadAndRemapMatrix.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LogMatrixDeterminant.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LogSoftmax.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LogUniformCandidateSampler.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LookupTableExport.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LookupTableExportV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LookupTableFind.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LookupTableFindV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LookupTableImport.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LookupTableImportV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LookupTableInsert.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LookupTableInsertV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LookupTableSize.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_LookupTableSizeV2.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_M.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MatMul.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MatrixBandPart.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MatrixDeterminant.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MatrixDiag.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MatrixDiagPart.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MatrixInverse.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MatrixSetDiag.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MatrixSolve.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MatrixSolveLs.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MatrixTriangularSolve.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Max.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MaxPool.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MaxPool3DGrad.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MaxPool3DGradGrad.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MaxPoolGradGrad.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MaxPoolGradGradWithArgmax.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MaxPoolV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MaxPoolWithArgmax.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Mean.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Merge.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MergeSummary.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Min.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MirrorPad.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Mul.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MutableDenseHashTable.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MutableDenseHashTableV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MutableHashTable.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MutableHashTableOfTensors.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MutableHashTableOfTensorsV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_MutableHashTableV2.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_N.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Neg.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_NegTrain.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_NonMaxSuppression.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_NonMaxSuppressionV2.pbtxt rename tensorflow/core/api_def/python_api/{api_def_O.pbtxt => api_def_OneHot.pbtxt} (100%) delete mode 100644 tensorflow/core/api_def/python_api/api_def_P.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Pack.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Pad.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_PadV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_PaddingFIFOQueue.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_PaddingFIFOQueueV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ParallelConcat.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ParameterizedTruncatedNormal.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ParseExample.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ParseSingleSequenceExample.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Placeholder.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Pow.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Print.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_PriorityQueue.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_PriorityQueueV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Prod.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_PyFunc.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_PyFuncStateless.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_Q.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Qr.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QuantizedAvgPool.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QuantizedMaxPool.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QuantizedReluX.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueClose.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueCloseV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueDequeue.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueDequeueMany.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueDequeueManyV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueDequeueUpTo.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueDequeueUpToV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueDequeueV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueEnqueue.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueEnqueueMany.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueEnqueueManyV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueEnqueueV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueSize.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_QueueSizeV2.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_R.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RGBToHSV.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RandomCrop.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RandomGamma.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RandomPoisson.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RandomShuffle.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RandomShuffleQueue.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RandomShuffleQueueV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RandomStandardNormal.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RandomUniform.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RandomUniformInt.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Range.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderNumRecordsProduced.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderNumRecordsProducedV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderNumWorkUnitsCompleted.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderNumWorkUnitsCompletedV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderRead.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderReadUpTo.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderReadUpToV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderReadV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderReset.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderResetV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderRestoreState.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderRestoreStateV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderSerializeState.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReaderSerializeStateV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RealDiv.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Relu.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Relu6.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ResizeArea.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ResizeBicubic.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ResizeBilinear.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ResizeNearestNeighbor.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Restore.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RestoreSlice.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Reverse.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ReverseV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_RightShift.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_S.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SampleDistortedBoundingBox.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SampleDistortedBoundingBoxV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Save.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SaveSlices.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ScalarSummary.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SdcaFprint.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SdcaOptimizer.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SdcaShrinkL1.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Select.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SelfAdjointEig.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SelfAdjointEigV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Selu.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SerializeManySparse.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SerializeSparse.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ShardedFilename.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ShardedFilespec.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Sigmoid.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Skipgram.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Slice.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Softmax.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SoftmaxCrossEntropyWithLogits.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Softplus.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Softsign.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SpaceToBatch.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseAdd.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseAddGrad.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseConcat.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseCross.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseFillEmptyRows.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseFillEmptyRowsGrad.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseMatMul.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseReorder.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseReshape.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseSoftmaxCrossEntropyWithLogits.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseSplit.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseTensorDenseAdd.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseTensorDenseMatMul.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SparseToDense.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Split.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SplitV.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Squeeze.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Stack.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_StackClose.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_StackCloseV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_StackPop.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_StackPopV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_StackPush.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_StackPushV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_StackV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_StringSplit.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Sub.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Sum.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Svd.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Switch.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_SymbolicGradient.pbtxt delete mode 100644 tensorflow/core/api_def/python_api/api_def_T.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TFRecordReader.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TFRecordReaderV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TakeManySparseFromTensorsMap.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Tanh.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TemporaryVariable.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArray.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayClose.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayCloseV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayCloseV3.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayConcat.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayConcatV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayConcatV3.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayGather.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayGatherV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayGatherV3.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayGrad.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayGradV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayGradV3.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayPack.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayRead.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayReadV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayReadV3.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayScatter.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayScatterV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayScatterV3.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArraySize.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArraySizeV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArraySizeV3.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArraySplit.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArraySplitV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArraySplitV3.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayUnpack.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayV3.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayWrite.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayWriteV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorArrayWriteV3.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorSummary.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TensorSummaryV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TextLineReader.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TextLineReaderV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_ThreadUnsafeUnigramCandidateSampler.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TileGrad.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TopK.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TopKV2.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TruncateDiv.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TruncateMod.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_TruncatedNormal.pbtxt rename tensorflow/core/api_def/python_api/{api_def_U.pbtxt => api_def_UniformCandidateSampler.pbtxt} (56%) create mode 100644 tensorflow/core/api_def/python_api/api_def_Unpack.pbtxt create mode 100644 tensorflow/core/api_def/python_api/api_def_Variable.pbtxt rename tensorflow/core/api_def/python_api/{api_def_V.pbtxt => api_def_VariableV2.pbtxt} (50%) create mode 100644 tensorflow/core/api_def/python_api/api_def_WholeFileReader.pbtxt rename tensorflow/core/api_def/python_api/{api_def_W.pbtxt => api_def_WholeFileReaderV2.pbtxt} (50%) rename tensorflow/core/api_def/python_api/{api_def_Z.pbtxt => api_def_ZerosLike.pbtxt} (100%) create mode 100644 tensorflow/core/kernels/matrix_exponential_op.cc create mode 100644 tensorflow/python/keras/_impl/keras/applications/inception_resnet_v2.py create mode 100644 tensorflow/python/keras/_impl/keras/applications/inception_resnet_v2_test.py create mode 100644 tensorflow/python/keras/applications/inception_resnet_v2/__init__.py create mode 100644 tensorflow/python/kernel_tests/garbage_collection_test.py create mode 100644 tensorflow/python/kernel_tests/matrix_exponential_op_test.py create mode 100644 tensorflow/tools/api/golden/tensorflow.keras.-model.pbtxt create mode 100644 tensorflow/tools/api/golden/tensorflow.keras.-sequential.pbtxt create mode 100644 tensorflow/tools/api/golden/tensorflow.keras.applications.inception_resnet_v2.pbtxt create mode 100644 third_party/eigen3/unsupported/Eigen/MatrixFunctions diff --git a/configure.py b/configure.py index 8572fa7fdb..6279c42610 100644 --- a/configure.py +++ b/configure.py @@ -25,10 +25,12 @@ import re import subprocess import sys +# pylint: disable=g-import-not-at-top try: from shutil import which except ImportError: from distutils.spawn import find_executable as which +# pylint: enable=g-import-not-at-top _TF_BAZELRC = os.path.join(os.path.dirname(os.path.abspath(__file__)), '.tf_configure.bazelrc') @@ -485,7 +487,10 @@ def set_cc_opt_flags(environ_cp): cc_opt_flags = get_from_env_or_user_or_default(environ_cp, 'CC_OPT_FLAGS', question, default_cc_opt_flags) for opt in cc_opt_flags.split(): - write_to_bazelrc('build:opt --cxxopt=%s --copt=%s' % (opt, opt)) + host_opt = '-march=native' # It should be safe on the same build host. + write_to_bazelrc( + 'build:opt --cxxopt=%s --copt=%s' % (opt, opt) + + ' --host_cxxopt=%s --host_copt=%s' % (host_opt, host_opt)) def set_tf_cuda_clang(environ_cp): diff --git a/tensorflow/compiler/tf2xla/functionalize_control_flow.cc b/tensorflow/compiler/tf2xla/functionalize_control_flow.cc index 893175373f..6ef4860f35 100644 --- a/tensorflow/compiler/tf2xla/functionalize_control_flow.cc +++ b/tensorflow/compiler/tf2xla/functionalize_control_flow.cc @@ -130,7 +130,9 @@ Status CopySubgraph(const Graph& graph, const Frame* frame, stack.push_back(src); } Node* src_copy = (*node_map)[e->src()->id()]; - int src_output = squash_src_outputs[e->src()->id()] ? 0 : e->src_output(); + int src_output = squash_src_outputs[e->src()->id()] && !e->IsControlEdge() + ? 0 + : e->src_output(); Node* dst_copy = (*node_map)[e->dst()->id()]; output->AddEdge(src_copy, src_output, dst_copy, e->dst_input()); } diff --git a/tensorflow/compiler/tf2xla/kernels/gather_op.cc b/tensorflow/compiler/tf2xla/kernels/gather_op.cc index 2c5d910d58..e420f21ca3 100644 --- a/tensorflow/compiler/tf2xla/kernels/gather_op.cc +++ b/tensorflow/compiler/tf2xla/kernels/gather_op.cc @@ -77,18 +77,6 @@ xla::ComputationDataHandle XlaComputeGatherDynamicSlice( out_shape.dim_sizes()); } - // Degenerate case: single slice. - if (num_indices == 1) { - auto index = builder->Reshape(indices, {1}); - auto start_index = builder->Pad( - index, XlaHelpers::Zero(builder, index_type), - xla::MakeEdgePaddingConfig( - {{input_shape_pre_axis.dims(), input_shape_post_axis.dims()}})); - auto slice = - builder->DynamicSlice(input, start_index, slice_shape.dim_sizes()); - return builder->Reshape(slice, out_shape.dim_sizes()); - } - // Specify the shape of the loop-carried Tensor tuple. xla::PrimitiveType ptype; TF_CHECK_OK(DataTypeToPrimitiveType(dtype, &ptype)); diff --git a/tensorflow/compiler/xla/client/computation_builder.cc b/tensorflow/compiler/xla/client/computation_builder.cc index 24774c4c2a..763d94e94c 100644 --- a/tensorflow/compiler/xla/client/computation_builder.cc +++ b/tensorflow/compiler/xla/client/computation_builder.cc @@ -1309,7 +1309,7 @@ Status ComputationBuilder::SetReturnValue( } StatusOr ComputationBuilder::IsConstant( - const ComputationDataHandle& operand) { + const ComputationDataHandle& operand, int64 num_parameters) { if (!first_error_.ok()) { return first_error_; } @@ -1317,6 +1317,7 @@ StatusOr ComputationBuilder::IsConstant( IsConstantRequest request; *request.mutable_computation() = computation_.handle(); *request.mutable_operand() = operand; + request.set_num_parameters(num_parameters); IsConstantResponse response; VLOG(2) << "making IsConstant request"; @@ -1330,7 +1331,8 @@ StatusOr ComputationBuilder::IsConstant( } StatusOr> ComputationBuilder::ComputeConstant( - const ComputationDataHandle& operand, const Layout* output_layout) { + const ComputationDataHandle& operand, const Layout* output_layout, + tensorflow::gtl::ArraySlice parameters) { if (!first_error_.ok()) { return first_error_; } @@ -1341,6 +1343,9 @@ StatusOr> ComputationBuilder::ComputeConstant( if (output_layout != nullptr) { *request.mutable_output_layout() = *output_layout; } + for (const auto& param : parameters) { + *request.add_parameters() = param.ToProto(); + } ComputeConstantResponse response; diff --git a/tensorflow/compiler/xla/client/computation_builder.h b/tensorflow/compiler/xla/client/computation_builder.h index bc7ad06a3f..8e1b4be1f3 100644 --- a/tensorflow/compiler/xla/client/computation_builder.h +++ b/tensorflow/compiler/xla/client/computation_builder.h @@ -746,11 +746,12 @@ class ComputationBuilder { ComputationDataHandle Recv(const Shape& shape, const ChannelHandle& handle); // Returns true if 'operand' is a compile-time constant. A compile-time - // constant does not depend on parameters, or on stateful operators such - // as `RngNormal` or `Infeed`. Unlike `ComputeConstant`, `IsConstant` tests - // whether a computation is a compile-time constant without evaluating the - // computation. - StatusOr IsConstant(const ComputationDataHandle& operand); + // constant does not depend on parameters with higher index then + // `num_parameters`, or on stateful operators such as `RngNormal` or `Infeed`. + // Unlike `ComputeConstant`, `IsConstant` tests whether a computation is a + // compile-time constant without evaluating the computation. + StatusOr IsConstant(const ComputationDataHandle& operand, + int64 num_parameters = 0); // Normalizes operand across spatial and batch dimensions for each feature. // @@ -795,7 +796,7 @@ class ComputationBuilder { float epsilon, int64 feature_index); // Computes the value of a constant indicated by a - // ComputationDataHandle. + // ComputationDataHandle using a non-optimized interpreter on the host. // // The operand must be from the computation currently being built - // i.e., returned from this builder with no intervening call to @@ -803,8 +804,11 @@ class ComputationBuilder { // that may stop working at any time. // // The operand must represent a constant value, which in this case - // means that it must not statically depend on a parameter to the - // computation that is being built. + // means that it must not statically depend on any parameter of the + // computation that is being built other then the ones specified on the + // paramtere list. The parameters in the list will be indexed by their + // parameter id property so the number of parameters specified should be at + // least as many as the largest used parameter index. // // `IsConstant` can be used to test whether a computation is a compile-time // constant without evaluation it. `ComputeConstant` only succeeds for @@ -822,7 +826,8 @@ class ComputationBuilder { // will be stored using that layout. StatusOr> ComputeConstant( const ComputationDataHandle& operand, - const Layout* output_layout = nullptr); + const Layout* output_layout = nullptr, + tensorflow::gtl::ArraySlice parameters = {}); // Returns a new ComputationBuilder whose resultant Computation is used only // by this ComputationBuilder. The sub-ComputationBuilder has the same diff --git a/tensorflow/compiler/xla/service/buffer_assignment.cc b/tensorflow/compiler/xla/service/buffer_assignment.cc index 8536429846..b422b22df9 100644 --- a/tensorflow/compiler/xla/service/buffer_assignment.cc +++ b/tensorflow/compiler/xla/service/buffer_assignment.cc @@ -101,6 +101,11 @@ BufferAllocationProto BufferAllocation::ToProto() const { proto_assigned->set_offset(buffer_offset_size.second.offset); proto_assigned->set_size(buffer_offset_size.second.size); } + std::sort(proto.mutable_assigned()->begin(), proto.mutable_assigned()->end(), + [](const BufferAllocationProto::Assigned& assign1, + const BufferAllocationProto::Assigned& assign2) { + return assign1.logical_buffer_id() < assign2.logical_buffer_id(); + }); return proto; } diff --git a/tensorflow/compiler/xla/service/cpu/llvm_ir_runtime.cc b/tensorflow/compiler/xla/service/cpu/llvm_ir_runtime.cc index b490472831..81c29e4726 100644 --- a/tensorflow/compiler/xla/service/cpu/llvm_ir_runtime.cc +++ b/tensorflow/compiler/xla/service/cpu/llvm_ir_runtime.cc @@ -52,7 +52,7 @@ llvm::Function* EmitVectorF32TanhIfNeeded(llvm::Module* module, llvm::IRBuilder<> ir_builder(vector_tanh_body); llvm::FastMathFlags fast_math_flags; - fast_math_flags.setUnsafeAlgebra(); + fast_math_flags.setFast(); ir_builder.setFastMathFlags(fast_math_flags); llvm::Value* input = &*vector_tanh_function->arg_begin(); diff --git a/tensorflow/compiler/xla/service/executable.h b/tensorflow/compiler/xla/service/executable.h index 2d32e59d36..7e0d182b36 100644 --- a/tensorflow/compiler/xla/service/executable.h +++ b/tensorflow/compiler/xla/service/executable.h @@ -88,6 +88,16 @@ class Executable { tensorflow::gtl::ArraySlice> arguments); + // Populates `hlo_execution_profile` from `executor`. This is implicit in any + // Execute* API call that takes a hlo_execution_profile argument, but must be + // called explicitly for other (async, for example) variants after the stream + // has completed. + virtual Status PopulateExecutionProfile( + HloExecutionProfile* hlo_execution_profile, + perftools::gputools::StreamExecutor* executor) { + return Status::OK(); + } + // Convenience wrapper for calling Executable::ExecuteOnStream. Sets up a // timer for the execution, sets up HLO profiling if enabled, and fills in the // given ExecutionProfile if non-null. The ExecuteOnStream overloads have diff --git a/tensorflow/compiler/xla/service/hlo_instruction.cc b/tensorflow/compiler/xla/service/hlo_instruction.cc index e09899e48d..5107ac782d 100644 --- a/tensorflow/compiler/xla/service/hlo_instruction.cc +++ b/tensorflow/compiler/xla/service/hlo_instruction.cc @@ -1901,12 +1901,13 @@ std::vector HloInstruction::ExtraAttributesToString() const { if (has_sharding()) { extra.push_back(StrCat("sharding=", sharding().ToString())); } - if (!control_successors_.empty()) { - extra.push_back(StrCat( - "control-successors=", - Join(control_successors_, ", ", [](string* out, HloInstruction* succ) { - StrAppend(out, succ->name()); - }))); + if (!control_predecessors_.empty()) { + extra.push_back(StrCat("control-predecessors={", + Join(control_predecessors_, ", ", + [](string* out, HloInstruction* pre) { + StrAppend(out, pre->name()); + }), + "}")); } return extra; } diff --git a/tensorflow/compiler/xla/service/hlo_runner.cc b/tensorflow/compiler/xla/service/hlo_runner.cc index aaa4e3a2e3..f463e57d99 100644 --- a/tensorflow/compiler/xla/service/hlo_runner.cc +++ b/tensorflow/compiler/xla/service/hlo_runner.cc @@ -41,11 +41,21 @@ namespace se = ::perftools::gputools; namespace xla { /*static*/ StatusOr> -HloRunner::ReadModuleFromHloProtoFile(const char* filename, +HloRunner::ReadModuleFromHloProtoFile(const std::string& filename, const DebugOptions& debug_options) { HloProto proto; - TF_RETURN_IF_ERROR(tensorflow::ReadBinaryProto(tensorflow::Env::Default(), - filename, &proto)); + + const Status s = + tensorflow::ReadBinaryProto(tensorflow::Env::Default(), filename, &proto); + + if (!s.ok()) { + const Status s2 = + tensorflow::ReadTextProto(tensorflow::Env::Default(), filename, &proto); + if (!s2.ok()) { + return Status(s2.code(), s.error_message() + "\n" + s2.error_message()); + } + } + TF_ASSIGN_OR_RETURN( HloModuleConfig config, HloModule::CreateModuleConfigFromProto(proto.hlo_module())); @@ -56,7 +66,7 @@ HloRunner::ReadModuleFromHloProtoFile(const char* filename, } /*static*/ StatusOr> -HloRunner::ReadModuleFromHloTextDumpFile(const char* filename, +HloRunner::ReadModuleFromHloTextDumpFile(const std::string& filename, const DebugOptions& debug_options) { string hlo_string; TF_RETURN_IF_ERROR(tensorflow::ReadFileToString(tensorflow::Env::Default(), @@ -66,6 +76,19 @@ HloRunner::ReadModuleFromHloTextDumpFile(const char* filename, return tools::Parse(hlo_string, config); } +/*static*/ StatusOr> HloRunner::ReadModule( + const std::string& filename, const DebugOptions& debug_options) { + auto module = HloRunner::ReadModuleFromHloProtoFile(filename, debug_options); + if (module.ok()) { + return module; + } + const std::string e = module.status().error_message(); + module = HloRunner::ReadModuleFromHloTextDumpFile(filename, debug_options); + return module.ok() ? std::move(module) + : Status(module.status().code(), + e + "\n" + module.status().error_message()); +} + // Define this in .cc file to avoid having to include eigen or forward declare // these types in the header. struct HloRunner::EigenThreadPoolWrapper { diff --git a/tensorflow/compiler/xla/service/hlo_runner.h b/tensorflow/compiler/xla/service/hlo_runner.h index b0e2b980e2..a5732848c6 100644 --- a/tensorflow/compiler/xla/service/hlo_runner.h +++ b/tensorflow/compiler/xla/service/hlo_runner.h @@ -44,15 +44,23 @@ class HloRunner { ~HloRunner(); - // Reads the binary proto file in xla.HloProto format, creates and returns the - // HloModule. + // Reads the proto file in xla.HloProto format, creates and returns the + // HloModule. Will try to parse the filename as binary proto, then try as + // text proto if that fails. static StatusOr> ReadModuleFromHloProtoFile( - const char* filename, const DebugOptions& debug_options); + const std::string& filename, const DebugOptions& debug_options); // Reads the hlo text dump file in HloModule::ToString format, creates and // returns the HloModule. static StatusOr> ReadModuleFromHloTextDumpFile( - const char* filename, const DebugOptions& debug_options); + const std::string& filename, const DebugOptions& debug_options); + + // Tries to parse the filename specified first as binary proto format, then + // as a textual proto format, then textual IR, then gives up if both fail. + // ReadModuleFromHloProtoFile or ReadModuleFromHloTextDumpFile should be used + // explicitly when you know the format, this if you don't. + static StatusOr> ReadModule( + const std::string& filename, const DebugOptions& debug_options); // Executes the given module with given literals as input and returns the // result as a Literal. The LiteralPtr type accepts Literal* or diff --git a/tensorflow/compiler/xla/service/llvm_ir/llvm_util.cc b/tensorflow/compiler/xla/service/llvm_ir/llvm_util.cc index 5dff4b5778..956c0d5f05 100644 --- a/tensorflow/compiler/xla/service/llvm_ir/llvm_util.cc +++ b/tensorflow/compiler/xla/service/llvm_ir/llvm_util.cc @@ -555,8 +555,9 @@ int64 ByteSizeOf(const Shape& shape, const llvm::DataLayout& data_layout) { llvm::FastMathFlags GetFastMathFlags(bool fast_math_enabled) { llvm::FastMathFlags flags; if (fast_math_enabled) { - // UnsafeAlgebra implies NoInfs, NoNaNs, NoSignedZeros, and AllowReciprocal. - flags.setUnsafeAlgebra(); + // Fast implies AllowReassoc, NoInfs, NoNaNs, NoSignedZeros, + // AllowReciprocal, AllowContract, and ApproxFunc. + flags.setFast(); } return flags; } diff --git a/tensorflow/compiler/xla/service/service.cc b/tensorflow/compiler/xla/service/service.cc index bac33d8102..71afbee456 100644 --- a/tensorflow/compiler/xla/service/service.cc +++ b/tensorflow/compiler/xla/service/service.cc @@ -490,14 +490,20 @@ Service::ExecuteParallelAndRegisterResult( std::vector> arguments, Backend* backend, tensorflow::gtl::ArraySlice device_handles, - tensorflow::gtl::ArraySlice result_tags) { + tensorflow::gtl::ArraySlice result_tags, + ExecutionProfile* profile) { // Streams where the computation are launched, so we can wait on the streams // to complete. std::vector::SmartPtr> streams; + std::vector> timers; // Global data handles for the computation results, one for each computation. std::vector result_handles; + // Device ID to stream executor, populated only with devices that are being + // profiled. + std::map index_to_profiled_streams; + TF_ASSIGN_OR_RETURN(DeviceAssignment device_assignment, backend->computation_placer()->AssignDevices( options_.number_of_replicas(), executables.size())); @@ -510,6 +516,21 @@ Service::ExecuteParallelAndRegisterResult( backend->BorrowStream(replicas[replica])); streams.push_back(std::move(stream)); + if (replica == 0 && profile != nullptr) { + timers.emplace_back( + new perftools::gputools::Timer(streams.back()->parent())); + streams.back() + ->InitTimer(timers.back().get()) + .ThenStartTimer(timers.back().get()); + CHECK(timers.front() != nullptr); + } + + if (replica == 0 && + executables[i]->module_config().debug_options().xla_hlo_profile() && + executables[i]->hlo_profiling_enabled()) { + index_to_profiled_streams[i] = streams.back().get(); + } + // Set up run options. ExecutableRunOptions options; options.set_stream(streams.back().get()); @@ -526,6 +547,10 @@ Service::ExecuteParallelAndRegisterResult( perftools::gputools::DeviceMemoryBase result, executables[i]->ExecuteAsyncOnStream(&run_options, arguments[i])); + if (replica == 0 && profile != nullptr) { + streams.back()->ThenStopTimer(timers.back().get()); + } + // All replicas share the same device address for the result allocation, // so only one of the replicas need to register the result handle. if (replica == 0) { @@ -543,6 +568,69 @@ Service::ExecuteParallelAndRegisterResult( } } + // For every stream that had profiling enabled, obtain and debug-dump the HLO + // profile. + for (auto& index_to_profiled_stream : index_to_profiled_streams) { + int64 device = index_to_profiled_stream.first; + se::Stream* stream = index_to_profiled_stream.second; + HloExecutionProfile hlo_profile; + TF_RETURN_IF_ERROR(executables[device]->PopulateExecutionProfile( + &hlo_profile, stream->parent())); + + std::unordered_set profiled_computations = + hlo_profile.profiled_computations(); + // To ensure we have print the profiles in a stable order, iterate over the + // computations in post order. + auto& module = executables[device]->module(); + std::list all_computations = + module.MakeComputationPostOrder(); + for (xla::HloComputation* computation : all_computations) { + if (profiled_computations.count(computation) > 0) { + string profile_string = hlo_profile.ToString( + *computation, streams[0]->parent()->GetDeviceDescription(), + executables[device]->CreateCostAnalysis().get()); + if (!profile_string.empty()) { + LOG(INFO) << "HLO profile for execution on device " << device + << ":\n"; + XLA_LOG_LINES(tensorflow::INFO, profile_string); + } + } + } + hlo_graph_dumper::MaybeDumpHloModule(module, "Service::Execute", + &hlo_profile); + } + + if (profile != nullptr) { + CHECK(!timers.empty()); + std::vector timer_nanoseconds; + timer_nanoseconds.reserve(timers.size()); + for (auto& timer : timers) { + timer_nanoseconds.push_back(timer->Nanoseconds()); + } + uint64 nanoseconds = + *std::max_element(timer_nanoseconds.begin(), timer_nanoseconds.end()); + + // Merge in run-time profile information from execution_profile on the + // zeroth device. + profile->MergeFrom(executables[0]->execution_profile()); + + // Overall execution time (in nanoseconds) from the executor timer. + profile->set_compute_and_transfer_time_ns(nanoseconds); + + // TODO(b/28123297): On GPU we end up including transfer time in + // the compute time this way. Instead, we should get the correct + // value by measuring it. Setting the field here at least lets + // benchmarks provide *some* value for GPU computations. + // + // TODO(b/28447609): The value in compute_and_transfer_time_ns is actually + // the compute time without the transfer time, so this way we get the + // correct compute time. We should instead have the correct value for + // compute_and_transfer_time and set compute_time to the compute time. + if (profile->compute_time_ns() == 0) { + profile->set_compute_time_ns(profile->compute_and_transfer_time_ns()); + } + } + return result_handles; } @@ -715,14 +803,16 @@ tensorflow::Status Service::ExecuteParallel(const ExecuteParallelRequest* arg, // Execute the generated executables in parallel and return the device // handles for each computation's output. + ExecutionProfile profile; TF_ASSIGN_OR_RETURN( std::vector outputs, ExecuteParallelAndRegisterResult(executable_ptrs, all_arguments, execute_backend_.get(), device_handles, - computation_names)); + computation_names, &profile)); for (const GlobalDataHandle& output : outputs) { ExecuteResponse response; *response.mutable_output() = output; + *response.mutable_profile() = profile; *result->add_responses() = response; } @@ -1082,8 +1172,9 @@ tensorflow::Status Service::IsConstant(const IsConstantRequest* arg, return InvalidArgument("computations may not be empty"); } - TF_ASSIGN_OR_RETURN(bool is_constant, - user_computation->IsConstant(arg->operand())); + TF_ASSIGN_OR_RETURN( + bool is_constant, + user_computation->IsConstant(arg->operand(), arg->num_parameters())); result->set_is_constant(is_constant); return tensorflow::Status::OK(); @@ -1101,8 +1192,9 @@ tensorflow::Status Service::ComputeConstant(const ComputeConstantRequest* arg, return InvalidArgument("computations may not be empty"); } - TF_ASSIGN_OR_RETURN(bool is_constant, - user_computation->IsConstant(arg->operand())); + TF_ASSIGN_OR_RETURN( + bool is_constant, + user_computation->IsConstant(arg->operand(), arg->parameters_size())); if (!is_constant) { return InvalidArgument("Operand to ComputeConstant depends on parameter."); } @@ -1141,8 +1233,18 @@ tensorflow::Status Service::ComputeConstant(const ComputeConstantRequest* arg, /*include_unreachable_instructions=*/ false)); + std::vector parameters(arg->parameters_size()); + for (int64 i = 0; i < arg->parameters_size(); ++i) { + parameters[i] = Literal(arg->parameters(i)); + } + std::vector parameter_ptrs; + std::transform(parameters.begin(), parameters.end(), + std::back_inserter(parameter_ptrs), + [](const Literal& literal) { return &literal; }); + HloEvaluator evaluator; - TF_ASSIGN_OR_RETURN(auto result_literal, evaluator.Evaluate(*module, {})); + TF_ASSIGN_OR_RETURN(auto result_literal, + evaluator.Evaluate(*module, parameter_ptrs)); // Since the shape_with_output_layout option in ExecutionOption is // non-effective to the Evaluator results, explicit relayout here. if (arg->has_output_layout()) { diff --git a/tensorflow/compiler/xla/service/service.h b/tensorflow/compiler/xla/service/service.h index 2452259f73..6646be2e9a 100644 --- a/tensorflow/compiler/xla/service/service.h +++ b/tensorflow/compiler/xla/service/service.h @@ -327,7 +327,8 @@ class Service : public ServiceInterface { arguments, Backend* backend, tensorflow::gtl::ArraySlice device_handles, - tensorflow::gtl::ArraySlice result_tags); + tensorflow::gtl::ArraySlice result_tags, + ExecutionProfile* profile); // Convenience function for adding a function to a user computation. template diff --git a/tensorflow/compiler/xla/service/user_computation.cc b/tensorflow/compiler/xla/service/user_computation.cc index 006c814996..e9d182509b 100644 --- a/tensorflow/compiler/xla/service/user_computation.cc +++ b/tensorflow/compiler/xla/service/user_computation.cc @@ -1482,14 +1482,15 @@ UserComputation::ComputeProgramShape( namespace { -// A visitor which checks whether an operation is a compile-time constant. That -// is, the operation does not depend on any parameter instructions. The visitor -// walks the computation starting at a given operation and sets is_constant to -// false iff a parameter or RNG operation is encountered. -void ConstantVisitor(const SessionComputation& session_computation, - const ComputationDataHandle& handle, - std::set* visited, bool* is_constant) { - if (visited->count(handle.handle()) != 0 || !*is_constant) { +// A visitor which checks whether an operation is pure functional meaning that +// it doesn't depend on any parameter with an index higher then num_parameters. +// The visitor walks the computation starting at a given operation and sets +// is_functional to false iff a parameter or RNG operation is encountered. +void PureFunctionalVisitor(const SessionComputation& session_computation, + const ComputationDataHandle& handle, + int64 num_parameters, std::set* visited, + bool* is_functional) { + if (visited->count(handle.handle()) != 0 || !*is_functional) { return; } @@ -1497,7 +1498,7 @@ void ConstantVisitor(const SessionComputation& session_computation, session_computation.requests().at(handle.handle()); switch (request.request().op_case()) { case OpRequest::kRngRequest: - *is_constant = false; + *is_functional = false; break; case OpRequest::kConstantRequest: @@ -1506,41 +1507,43 @@ void ConstantVisitor(const SessionComputation& session_computation, case OpRequest::kGetTupleElementRequest: { const GetTupleElementRequest& get_tuple_element_request = request.request().get_tuple_element_request(); - ConstantVisitor(session_computation, get_tuple_element_request.operand(), - visited, is_constant); + PureFunctionalVisitor(session_computation, + get_tuple_element_request.operand(), num_parameters, + visited, is_functional); break; } case OpRequest::kSliceRequest: { const SliceRequest& slice_request = request.request().slice_request(); - ConstantVisitor(session_computation, slice_request.operand(), visited, - is_constant); + PureFunctionalVisitor(session_computation, slice_request.operand(), + num_parameters, visited, is_functional); break; } case OpRequest::kDynamicSliceRequest: { const DynamicSliceRequest& dynamic_slice_request = request.request().dynamic_slice_request(); - ConstantVisitor(session_computation, dynamic_slice_request.operand(), - visited, is_constant); - ConstantVisitor(session_computation, - dynamic_slice_request.start_indices(), visited, - is_constant); + PureFunctionalVisitor(session_computation, + dynamic_slice_request.operand(), num_parameters, + visited, is_functional); + PureFunctionalVisitor(session_computation, + dynamic_slice_request.start_indices(), + num_parameters, visited, is_functional); break; } case OpRequest::kDynamicUpdateSliceRequest: { const DynamicUpdateSliceRequest& dynamic_update_slice_request = request.request().dynamic_update_slice_request(); - ConstantVisitor(session_computation, - dynamic_update_slice_request.operand(), visited, - is_constant); - ConstantVisitor(session_computation, - dynamic_update_slice_request.update(), visited, - is_constant); - ConstantVisitor(session_computation, - dynamic_update_slice_request.start_indices(), visited, - is_constant); + PureFunctionalVisitor(session_computation, + dynamic_update_slice_request.operand(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, + dynamic_update_slice_request.update(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, + dynamic_update_slice_request.start_indices(), + num_parameters, visited, is_functional); break; } @@ -1549,7 +1552,8 @@ void ConstantVisitor(const SessionComputation& session_computation, request.request().concatenate_request(); for (const ComputationDataHandle& handle : concatenate_request.operands()) { - ConstantVisitor(session_computation, handle, visited, is_constant); + PureFunctionalVisitor(session_computation, handle, num_parameters, + visited, is_functional); } break; } @@ -1557,61 +1561,63 @@ void ConstantVisitor(const SessionComputation& session_computation, case OpRequest::kConvolveRequest: { const ConvolveRequest& convolve_request = request.request().convolve_request(); - ConstantVisitor(session_computation, convolve_request.lhs(), visited, - is_constant); - ConstantVisitor(session_computation, convolve_request.rhs(), visited, - is_constant); + PureFunctionalVisitor(session_computation, convolve_request.lhs(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, convolve_request.rhs(), + num_parameters, visited, is_functional); break; } case OpRequest::kCrossReplicaSumRequest: { // TODO(b/33009255): Implmement constant folding for cross replica sum. - *is_constant = false; + *is_functional = false; break; } case OpRequest::kInfeedRequest: { - *is_constant = false; + *is_functional = false; break; } case OpRequest::kOutfeedRequest: { - *is_constant = false; + *is_functional = false; break; } case OpRequest::kCallRequest: { const CallRequest& call_request = request.request().call_request(); for (const ComputationDataHandle& handle : call_request.operands()) { - ConstantVisitor(session_computation, handle, visited, is_constant); + PureFunctionalVisitor(session_computation, handle, num_parameters, + visited, is_functional); } // TODO(b/32495713): We aren't checking the to_apply computation itself, // so we conservatively say that computations containing the Call op - // cannot be constant. We cannot set is_constant=false in other similar + // cannot be constant. We cannot set is_functional=false in other similar // cases since we're already relying on IsConstant to return true. - *is_constant = false; + *is_functional = false; break; } case OpRequest::kCustomCallRequest: { - *is_constant = false; + *is_functional = false; break; } case OpRequest::kSendRequest: { - *is_constant = false; + *is_functional = false; break; } case OpRequest::kRecvRequest: { - *is_constant = false; + *is_functional = false; break; } case OpRequest::kMapRequest: { const MapRequest& map_request = request.request().map_request(); for (const ComputationDataHandle& handle : map_request.operands()) { - ConstantVisitor(session_computation, handle, visited, is_constant); + PureFunctionalVisitor(session_computation, handle, num_parameters, + visited, is_functional); } // TODO(b/32495713): We aren't checking the to_apply computation itself. break; @@ -1619,10 +1625,10 @@ void ConstantVisitor(const SessionComputation& session_computation, case OpRequest::kReduceRequest: { const ReduceRequest& reduce_request = request.request().reduce_request(); - ConstantVisitor(session_computation, reduce_request.operand(), visited, - is_constant); - ConstantVisitor(session_computation, reduce_request.init_value(), visited, - is_constant); + PureFunctionalVisitor(session_computation, reduce_request.operand(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, reduce_request.init_value(), + num_parameters, visited, is_functional); // TODO(b/32495713): We aren't checking the to_apply computation itself. break; } @@ -1630,10 +1636,12 @@ void ConstantVisitor(const SessionComputation& session_computation, case OpRequest::kReduceWindowRequest: { const ReduceWindowRequest& reduce_window_request = request.request().reduce_window_request(); - ConstantVisitor(session_computation, reduce_window_request.operand(), - visited, is_constant); - ConstantVisitor(session_computation, reduce_window_request.init_value(), - visited, is_constant); + PureFunctionalVisitor(session_computation, + reduce_window_request.operand(), num_parameters, + visited, is_functional); + PureFunctionalVisitor(session_computation, + reduce_window_request.init_value(), num_parameters, + visited, is_functional); // TODO(b/32495713): We aren't checking the to_apply computation itself. break; } @@ -1641,13 +1649,15 @@ void ConstantVisitor(const SessionComputation& session_computation, case OpRequest::kSelectAndScatterRequest: { const SelectAndScatterRequest& select_and_scatter_request = request.request().select_and_scatter_request(); - ConstantVisitor(session_computation, select_and_scatter_request.operand(), - visited, is_constant); - ConstantVisitor(session_computation, select_and_scatter_request.source(), - visited, is_constant); - ConstantVisitor(session_computation, - select_and_scatter_request.init_value(), visited, - is_constant); + PureFunctionalVisitor(session_computation, + select_and_scatter_request.operand(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, + select_and_scatter_request.source(), num_parameters, + visited, is_functional); + PureFunctionalVisitor(session_computation, + select_and_scatter_request.init_value(), + num_parameters, visited, is_functional); // TODO(b/32495713): We aren't checking the select and scatter // computations themselves. break; @@ -1656,76 +1666,80 @@ void ConstantVisitor(const SessionComputation& session_computation, case OpRequest::kBroadcastRequest: { const BroadcastRequest& broadcast_request = request.request().broadcast_request(); - ConstantVisitor(session_computation, broadcast_request.operand(), visited, - is_constant); + PureFunctionalVisitor(session_computation, broadcast_request.operand(), + num_parameters, visited, is_functional); break; } case OpRequest::kReshapeRequest: { const ReshapeRequest& reshape_request = request.request().reshape_request(); - ConstantVisitor(session_computation, reshape_request.operand(), visited, - is_constant); + PureFunctionalVisitor(session_computation, reshape_request.operand(), + num_parameters, visited, is_functional); break; } case OpRequest::kReverseRequest: { const ReverseRequest& reverse_request = request.request().reverse_request(); - ConstantVisitor(session_computation, reverse_request.operand(), visited, - is_constant); + PureFunctionalVisitor(session_computation, reverse_request.operand(), + num_parameters, visited, is_functional); break; } case OpRequest::kPadRequest: { const PadRequest& pad_request = request.request().pad_request(); - ConstantVisitor(session_computation, pad_request.operand(), visited, - is_constant); - ConstantVisitor(session_computation, pad_request.padding_value(), visited, - is_constant); + PureFunctionalVisitor(session_computation, pad_request.operand(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, pad_request.padding_value(), + num_parameters, visited, is_functional); break; } case OpRequest::kParameterRequest: { - *is_constant = false; + const ParameterRequest& parameter_request = + request.request().parameter_request(); + if (parameter_request.parameter() >= num_parameters) { + *is_functional = false; + } break; } case OpRequest::kConvertRequest: { const ConvertRequest& convert_request = request.request().convert_request(); - ConstantVisitor(session_computation, convert_request.operand(), visited, - is_constant); + PureFunctionalVisitor(session_computation, convert_request.operand(), + num_parameters, visited, is_functional); break; } case OpRequest::kWhileRequest: { const WhileRequest& while_request = request.request().while_request(); - ConstantVisitor(session_computation, while_request.init(), visited, - is_constant); + PureFunctionalVisitor(session_computation, while_request.init(), + num_parameters, visited, is_functional); // TODO(b/32495713): We aren't checking the condition and body // computations themselves. - *is_constant = false; + *is_functional = false; break; } case OpRequest::kTernaryOpRequest: { const TernaryOpRequest& ternary_op_request = request.request().ternary_op_request(); - ConstantVisitor(session_computation, ternary_op_request.lhs(), visited, - is_constant); - ConstantVisitor(session_computation, ternary_op_request.rhs(), visited, - is_constant); - ConstantVisitor(session_computation, ternary_op_request.ehs(), visited, - is_constant); + PureFunctionalVisitor(session_computation, ternary_op_request.lhs(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, ternary_op_request.rhs(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, ternary_op_request.ehs(), + num_parameters, visited, is_functional); break; } case OpRequest::kTransposeRequest: { const TransposeRequest& transpose_request = request.request().transpose_request(); - ConstantVisitor(session_computation, transpose_request.operand(), visited, - is_constant); + PureFunctionalVisitor(session_computation, transpose_request.operand(), + num_parameters, visited, is_functional); break; } @@ -1734,7 +1748,8 @@ void ConstantVisitor(const SessionComputation& session_computation, request.request().variadic_op_request(); for (const ComputationDataHandle& handle : variadic_op_request.operands()) { - ConstantVisitor(session_computation, handle, visited, is_constant); + PureFunctionalVisitor(session_computation, handle, num_parameters, + visited, is_functional); } break; } @@ -1742,67 +1757,74 @@ void ConstantVisitor(const SessionComputation& session_computation, case OpRequest::kUnaryOpRequest: { const UnaryOpRequest& unary_op_request = request.request().unary_op_request(); - ConstantVisitor(session_computation, unary_op_request.operand(), visited, - is_constant); + PureFunctionalVisitor(session_computation, unary_op_request.operand(), + num_parameters, visited, is_functional); break; } case OpRequest::kBatchNormTrainingRequest: { const BatchNormTrainingRequest& batch_norm_training_request = request.request().batch_norm_training_request(); - ConstantVisitor(session_computation, - batch_norm_training_request.operand(), visited, - is_constant); - ConstantVisitor(session_computation, batch_norm_training_request.scale(), - visited, is_constant); - ConstantVisitor(session_computation, batch_norm_training_request.offset(), - visited, is_constant); + PureFunctionalVisitor(session_computation, + batch_norm_training_request.operand(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, + batch_norm_training_request.scale(), num_parameters, + visited, is_functional); + PureFunctionalVisitor(session_computation, + batch_norm_training_request.offset(), + num_parameters, visited, is_functional); break; } case OpRequest::kBatchNormInferenceRequest: { const BatchNormInferenceRequest& batch_norm_inference_request = request.request().batch_norm_inference_request(); - ConstantVisitor(session_computation, - batch_norm_inference_request.operand(), visited, - is_constant); - ConstantVisitor(session_computation, batch_norm_inference_request.scale(), - visited, is_constant); - ConstantVisitor(session_computation, - batch_norm_inference_request.offset(), visited, - is_constant); - ConstantVisitor(session_computation, batch_norm_inference_request.mean(), - visited, is_constant); - ConstantVisitor(session_computation, - batch_norm_inference_request.variance(), visited, - is_constant); + PureFunctionalVisitor(session_computation, + batch_norm_inference_request.operand(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, + batch_norm_inference_request.scale(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, + batch_norm_inference_request.offset(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, + batch_norm_inference_request.mean(), num_parameters, + visited, is_functional); + PureFunctionalVisitor(session_computation, + batch_norm_inference_request.variance(), + num_parameters, visited, is_functional); break; } case OpRequest::kBatchNormGradRequest: { const BatchNormGradRequest& batch_norm_grad_request = request.request().batch_norm_grad_request(); - ConstantVisitor(session_computation, batch_norm_grad_request.operand(), - visited, is_constant); - ConstantVisitor(session_computation, batch_norm_grad_request.scale(), - visited, is_constant); - ConstantVisitor(session_computation, batch_norm_grad_request.mean(), - visited, is_constant); - ConstantVisitor(session_computation, batch_norm_grad_request.variance(), - visited, is_constant); - ConstantVisitor(session_computation, - batch_norm_grad_request.grad_output(), visited, - is_constant); + PureFunctionalVisitor(session_computation, + batch_norm_grad_request.operand(), num_parameters, + visited, is_functional); + PureFunctionalVisitor(session_computation, + batch_norm_grad_request.scale(), num_parameters, + visited, is_functional); + PureFunctionalVisitor(session_computation, batch_norm_grad_request.mean(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, + batch_norm_grad_request.variance(), num_parameters, + visited, is_functional); + PureFunctionalVisitor(session_computation, + batch_norm_grad_request.grad_output(), + num_parameters, visited, is_functional); break; } case OpRequest::kBinaryOpRequest: { const BinaryOpRequest& binary_op_request = request.request().binary_op_request(); - ConstantVisitor(session_computation, binary_op_request.lhs(), visited, - is_constant); - ConstantVisitor(session_computation, binary_op_request.rhs(), visited, - is_constant); + PureFunctionalVisitor(session_computation, binary_op_request.lhs(), + num_parameters, visited, is_functional); + PureFunctionalVisitor(session_computation, binary_op_request.rhs(), + num_parameters, visited, is_functional); break; } @@ -1817,8 +1839,8 @@ void ConstantVisitor(const SessionComputation& session_computation, } // namespace -StatusOr UserComputation::IsConstant( - const ComputationDataHandle& handle) { +StatusOr UserComputation::IsConstant(const ComputationDataHandle& handle, + int64 num_parameters) { tensorflow::mutex_lock lock(mutex_); // Verify that the handle is valid. @@ -1829,7 +1851,8 @@ StatusOr UserComputation::IsConstant( bool is_constant = true; std::set visited; - ConstantVisitor(session_computation_, handle, &visited, &is_constant); + PureFunctionalVisitor(session_computation_, handle, num_parameters, &visited, + &is_constant); return is_constant; } diff --git a/tensorflow/compiler/xla/service/user_computation.h b/tensorflow/compiler/xla/service/user_computation.h index dabf68e298..ac879ce55a 100644 --- a/tensorflow/compiler/xla/service/user_computation.h +++ b/tensorflow/compiler/xla/service/user_computation.h @@ -250,9 +250,11 @@ class UserComputation { StatusOr> ComputeProgramShape( VersionedComputationHandle::Version version) const; - // Returns true if the given data handle does not depend on any - // parameters. That is, the value can be computed at compile time. - StatusOr IsConstant(const ComputationDataHandle& handle); + // Returns true if the given data handle does not depend on any parameter with + // index higher then num_parameters. That is, the value can be computed at + // compile time if we know the first num_parameters arguments. + StatusOr IsConstant(const ComputationDataHandle& handle, + int64 num_parameters); // Returns the output shape of the operation indicated by the given handle. StatusOr GetShape(const ComputationDataHandle& handle); diff --git a/tensorflow/compiler/xla/tests/compute_constant_test.cc b/tensorflow/compiler/xla/tests/compute_constant_test.cc index b2e9743af7..d423c78476 100644 --- a/tensorflow/compiler/xla/tests/compute_constant_test.cc +++ b/tensorflow/compiler/xla/tests/compute_constant_test.cc @@ -71,24 +71,27 @@ class ComputeConstantTest : public ::testing::Test { StatusOr> ComputeConstantLiteral( Client* client, const ComputationDataHandle& operand, - ComputationBuilder* builder, Layout* output_layout = nullptr) { - TF_ASSIGN_OR_RETURN(auto computed, - builder->ComputeConstant(operand, output_layout)); + ComputationBuilder* builder, Layout* output_layout = nullptr, + tensorflow::gtl::ArraySlice parameters = {}) { + TF_ASSIGN_OR_RETURN(auto computed, builder->ComputeConstant( + operand, output_layout, parameters)); return std::move(computed); } template - StatusOr ComputeConstantScalar(Client* client, - const ComputationDataHandle& operand, - ComputationBuilder* builder) { - TF_ASSIGN_OR_RETURN(auto literal, - ComputeConstantLiteral(client, operand, builder)); + StatusOr ComputeConstantScalar( + Client* client, const ComputationDataHandle& operand, + ComputationBuilder* builder, + tensorflow::gtl::ArraySlice parameters = {}) { + TF_ASSIGN_OR_RETURN( + auto literal, + ComputeConstantLiteral(client, operand, builder, nullptr, parameters)); return literal->Get({}); } bool IsConstant(const ComputationDataHandle& operand, - ComputationBuilder* builder) { - StatusOr result = builder->IsConstant(operand); + ComputationBuilder* builder, int64 num_parameters = 0) { + StatusOr result = builder->IsConstant(operand, num_parameters); EXPECT_TRUE(result.ok()) << result.status(); return result.ok() ? result.ValueOrDie() : false; } @@ -138,7 +141,25 @@ TEST_F(ComputeConstantTest, ScalarRng) { } } -TEST_F(ComputeConstantTest, DirectParam) { +TEST_F(ComputeConstantTest, Param) { + for (ClientType client_type : client_types) { + Client* client = ClientOrDie(platform_, client_type); + ComputationBuilder b(client, TestName()); + auto param = b.Parameter(0, ShapeUtil::MakeShape(F32, {}), "lhs"); + auto computation = b.Add(param, b.ConstantR0(1.5f)); + + std::vector arguments; + arguments.emplace_back(*Literal::CreateR0(42.5f)); + EXPECT_TRUE(IsConstant(computation, &b, arguments.size())); + + auto value = + ComputeConstantScalar(client, computation, &b, arguments); + ASSERT_TRUE(value.ok()) << value.status(); + EXPECT_EQ(value.ValueOrDie(), 44.0f); + } +} + +TEST_F(ComputeConstantTest, DirectParamMissing) { for (ClientType client_type : client_types) { Client* client = ClientOrDie(platform_, client_type); ComputationBuilder b(client, TestName()); @@ -152,7 +173,7 @@ TEST_F(ComputeConstantTest, DirectParam) { } } -TEST_F(ComputeConstantTest, IndirectParam) { +TEST_F(ComputeConstantTest, IndirectParamMissing) { for (ClientType client_type : client_types) { Client* client = ClientOrDie(platform_, client_type); ComputationBuilder b(client, TestName()); diff --git a/tensorflow/compiler/xla/tests/while_test.cc b/tensorflow/compiler/xla/tests/while_test.cc index 71a1b0abee..3b29a2eb9e 100644 --- a/tensorflow/compiler/xla/tests/while_test.cc +++ b/tensorflow/compiler/xla/tests/while_test.cc @@ -357,6 +357,111 @@ TEST_F(WhileTest, WhileWithVectorResultIntoTuple) { ComputeAndCompareTuple(&builder, *expected, {}, ErrorSpec(0.0001)); } +// TODO(b/63003356): 11-06-2017: fails on all back-ends with incorrect result. +TEST_F(WhileTest, DISABLED_WhileWithPermutationAndTupleResult) { + std::vector shape_elements = { + ShapeUtil::MakeShape(S32, {}), ShapeUtil::MakeShape(F32, {3}), + ShapeUtil::MakeShape(F32, {3}), ShapeUtil::MakeShape(F32, {3})}; + Shape result_shape = ShapeUtil::MakeTupleShape(shape_elements); + + // Create a computation for the condition. + // Repeat for N iterations. + const int N = 2; + Computation condition; + { + ComputationBuilder builder(client_, "condition"); + auto prev = builder.Parameter(0, result_shape, "prev"); + auto iteration = builder.GetTupleElement(prev, 0); + builder.Gt(builder.ConstantR0(N), iteration); + condition = builder.Build().ConsumeValueOrDie(); + } + + // Create a computation for the body. + // Add 1 to the iteration variable and permute the weights. + Computation body; + { + ComputationBuilder builder(client_, "body"); + auto prev = builder.Parameter(0, result_shape, "prev"); + auto iteration = builder.GetTupleElement(prev, 0); + auto w1 = builder.GetTupleElement(prev, 1); + auto w2 = builder.GetTupleElement(prev, 2); + auto w3 = builder.GetTupleElement(prev, 3); + auto result = builder.Tuple( + {builder.Add(iteration, builder.ConstantR0(1)), w3, w1, w2}); + body = builder.Build().ConsumeValueOrDie(); + } + + // Create a While node with computations for the condition and the body. + ComputationBuilder builder(client_, "while"); + auto init = builder.Tuple( + {builder.ConstantR0(0), builder.ConstantR1(3, 1.f), + builder.ConstantR1(3, 2.f), builder.ConstantR1(3, 3.f)}); + auto result = builder.While(condition, body, init); + VLOG(2) << "result = " + << ShapeUtil::HumanString( + *builder.GetShape(result).ConsumeValueOrDie()); + + auto expected_counter = Literal::CreateR0(N); + auto expected_w1 = Literal::CreateR1({1.0f, 1.0f, 1.0f}); + auto expected_w2 = Literal::CreateR1({2.0f, 2.0f, 2.0f}); + auto expected_w3 = Literal::CreateR1({3.0f, 3.0f, 3.0f}); + auto expected = Literal::MakeTuple({expected_counter.get(), expected_w2.get(), + expected_w3.get(), expected_w1.get()}); + VLOG(2) << "expected = " << ShapeUtil::HumanString(expected->shape()); + ComputeAndCompareTuple(&builder, *expected, {}, ErrorSpec(0.0001)); +} + +// TODO(b/63003356): 11-06-2017: fails on all back-ends with incorrect result. +TEST_F(WhileTest, DISABLED_WhileWithPermutationAndVectorResult) { + std::vector shape_elements = { + ShapeUtil::MakeShape(S32, {}), ShapeUtil::MakeShape(F32, {3}), + ShapeUtil::MakeShape(F32, {3}), ShapeUtil::MakeShape(F32, {3})}; + Shape result_shape = ShapeUtil::MakeTupleShape(shape_elements); + + // Create a computation for the condition. + // Repeat for N iterations. + const int N = 2; + Computation condition; + { + ComputationBuilder builder(client_, "condition"); + auto prev = builder.Parameter(0, result_shape, "prev"); + auto iteration = builder.GetTupleElement(prev, 0); + builder.Gt(builder.ConstantR0(N), iteration); + condition = builder.Build().ConsumeValueOrDie(); + } + + // Create a computation for the body. + // Add 1 to the iteration variable permute the weights. + Computation body; + { + ComputationBuilder builder(client_, "body"); + auto prev = builder.Parameter(0, result_shape, "prev"); + auto iteration = builder.GetTupleElement(prev, 0); + auto w1 = builder.GetTupleElement(prev, 1); + auto w2 = builder.GetTupleElement(prev, 2); + auto w3 = builder.GetTupleElement(prev, 3); + auto result = builder.Tuple( + {builder.Add(iteration, builder.ConstantR0(1)), w3, w1, w2}); + body = builder.Build().ConsumeValueOrDie(); + } + + // Create a While node with computations for the condition and the body. + ComputationBuilder builder(client_, "while"); + auto init = builder.Tuple( + {builder.ConstantR0(0), builder.ConstantR1(3, 1.f), + builder.ConstantR1(3, 2.f), builder.ConstantR1(3, 3.f)}); + auto xla_while = builder.While(condition, body, init); + + auto add12 = builder.Add(builder.GetTupleElement(xla_while, 1), + builder.GetTupleElement(xla_while, 2)); + auto result = builder.Add(add12, builder.GetTupleElement(xla_while, 3)); + VLOG(2) << "result = " + << ShapeUtil::HumanString( + *builder.GetShape(result).ConsumeValueOrDie()); + std::vector expected = {6.f, 6.f, 6.f}; + ComputeAndCompareR1(&builder, expected, {}, ErrorSpec(0.0001)); +} + // Tests a while node when the result type T is a Tuple. // // tuple> result(0, vector(10, 0.0f)); diff --git a/tensorflow/compiler/xla/tools/parser/hlo_parser.cc b/tensorflow/compiler/xla/tools/parser/hlo_parser.cc index 5de73ee866..6c2e37e3b5 100644 --- a/tensorflow/compiler/xla/tools/parser/hlo_parser.cc +++ b/tensorflow/compiler/xla/tools/parser/hlo_parser.cc @@ -58,6 +58,7 @@ class HloParser { string* root_name); bool ParseInstruction(HloComputation::Builder* builder, string* root_name); bool ParseSharding(HloInstruction* instruction); + bool ParseControlPredecessors(HloInstruction* instruction); bool ParseLiteral(std::unique_ptr* literal, const Shape& shape); bool ParseTupleLiteral(std::unique_ptr* literal, const Shape& shape); bool ParseNonTupleLiteral(std::unique_ptr* literal, @@ -436,10 +437,35 @@ bool HloParser::ParseInstruction(HloComputation::Builder* builder, return TokenError(StrCat("parsing not yet implemented for op: ", HloOpcodeString(opcode))); } - // Parse "sharding=". - if (lexer_.GetKind() == TokKind::kComma) { - if (!ParseSharding(instruction)) { - return false; + + bool has_sharding = false; + bool has_control = false; + while (EatIfPresent(TokKind::kComma)) { + string attribute_name; + if (!ParseAttributeName(&attribute_name)) { + return TokenError("expects ', sharding=' or ', control-predecessors='"); + } + + if (attribute_name == "sharding") { + // Parse "sharding=". + if (has_sharding) { + return TokenError("expects at most 1 'sharding='"); + } + has_sharding = true; + if (!ParseSharding(instruction)) { + return false; + } + } else if (attribute_name == "control-predecessors") { + // Parse "control-predecessors" + if (has_control) { + return TokenError("expects at most 1 'control-predecessors='"); + } + has_control = true; + if (!ParseControlPredecessors(instruction)) { + return false; + } + } else { + return TokenError(StrCat("unexpected attribute: ", attribute_name)); } } @@ -449,15 +475,6 @@ bool HloParser::ParseInstruction(HloComputation::Builder* builder, // ::= '{' 'replicated'? 'maximal'? ('device=' int)? shape? ('devices=' ('[' // dims ']')* device_list)? '}' dims ::= int_list device_list ::= int_list bool HloParser::ParseSharding(HloInstruction* instruction) { - if (!ParseToken(TokKind::kComma, - "expects ',' in front of an extra attribute")) { - return false; - } - string attribute_name; - if (!ParseAttributeName(&attribute_name) || attribute_name != "sharding") { - return TokenError("expects attribute name: sharding"); - } - if (!ParseToken(TokKind::kLbrace, "expected '{' to start sharding attribute")) { return false; @@ -577,6 +594,34 @@ bool HloParser::ParseSharding(HloInstruction* instruction) { return true; } +// '{' name+ '}' +bool HloParser::ParseControlPredecessors(HloInstruction* instruction) { + if (!ParseToken(TokKind::kLbrace, + "expects '{' at the beginning of control predecessors")) { + return false; + } + do { + string name; + if (!ParseName(&name)) { + return TokenError("expects a control predecessor"); + } + HloInstruction* pre = + tensorflow::gtl::FindPtrOrNull(instruction_pool_, name); + if (!pre) { + return TokenError( + StrCat("control predecessor ", name, " is not defined: ")); + } + Status status = pre->AddControlDependencyTo(instruction); + if (!status.ok()) { + return TokenError(StrCat("error adding control dependency for: ", name, + " status: ", status.ToString())); + } + } while (EatIfPresent(TokKind::kComma)); + + return ParseToken(TokKind::kRbrace, + "expects '}' at the end of control predecessors"); +} + bool HloParser::SetValueInLiteral(int64 value, int64 linear_index, Literal* literal) { const Shape& shape = literal->shape(); diff --git a/tensorflow/compiler/xla/tools/parser/hlo_parser_test.cc b/tensorflow/compiler/xla/tools/parser/hlo_parser_test.cc index e065af7da6..359256f064 100644 --- a/tensorflow/compiler/xla/tools/parser/hlo_parser_test.cc +++ b/tensorflow/compiler/xla/tools/parser/hlo_parser_test.cc @@ -214,7 +214,7 @@ R"(HloModule TwoSendRecvBothWayRecvFist_module: ENTRY %TwoSendRecvBothWayRecvFist.v3 () -> f32[] { %recv = f32[] recv(), channel_id=15, sharding={maximal device=1} ROOT %constant = f32[] constant(2.1), sharding={maximal device=0} - %send = () send(f32[] %constant), channel_id=16, sharding={maximal device=0} + %send = () send(f32[] %constant), channel_id=16, sharding={maximal device=0}, control-predecessors={%recv} } )" diff --git a/tensorflow/compiler/xla/xla.proto b/tensorflow/compiler/xla/xla.proto index ce3c3eee68..710bb6ff25 100644 --- a/tensorflow/compiler/xla/xla.proto +++ b/tensorflow/compiler/xla/xla.proto @@ -361,6 +361,7 @@ message WaitForExecutionResponse { message IsConstantRequest { ComputationHandle computation = 1; ComputationDataHandle operand = 2; + int64 num_parameters = 3; } message IsConstantResponse { @@ -371,6 +372,7 @@ message ComputeConstantRequest { ComputationHandle computation = 1; ComputationDataHandle operand = 2; Layout output_layout = 3; + repeated LiteralProto parameters = 4; } message ComputeConstantResponse { diff --git a/tensorflow/contrib/batching/adaptive_shared_batch_scheduler.h b/tensorflow/contrib/batching/adaptive_shared_batch_scheduler.h index a0606427a5..6ed177e001 100644 --- a/tensorflow/contrib/batching/adaptive_shared_batch_scheduler.h +++ b/tensorflow/contrib/batching/adaptive_shared_batch_scheduler.h @@ -399,7 +399,7 @@ ASBSQueue::~ASBSQueue() { template Status ASBSQueue::Schedule(std::unique_ptr* task) { - bool added_new_batch = false; + ASBSBatch* new_batch = nullptr; size_t size = (*task)->size(); if (size > options_.max_batch_size) { return errors::InvalidArgument("Task size ", size, @@ -418,15 +418,14 @@ Status ASBSQueue::Schedule(std::unique_ptr* task) { current_batch_ = nullptr; } if (!current_batch_) { - added_new_batch = true; num_enqueued_batches_++; - current_batch_ = + current_batch_ = new_batch = new ASBSBatch(this, scheduler_->GetEnv()->NowMicros()); } current_batch_->AddTask(std::move(*task)); num_enqueued_tasks_++; } - if (added_new_batch) scheduler_->AddBatch(current_batch_); + if (new_batch != nullptr) scheduler_->AddBatch(new_batch); return Status::OK(); } diff --git a/tensorflow/contrib/boosted_trees/python/training/functions/gbdt_batch.py b/tensorflow/contrib/boosted_trees/python/training/functions/gbdt_batch.py index 4d9fd75323..cebe3474ca 100644 --- a/tensorflow/contrib/boosted_trees/python/training/functions/gbdt_batch.py +++ b/tensorflow/contrib/boosted_trees/python/training/functions/gbdt_batch.py @@ -208,7 +208,7 @@ def extract_features(features, feature_columns): if tensor.dtype == dtypes.float32: if len(tensor.shape) > 1 and tensor.shape[1] > 1: unstacked = array_ops.unstack(tensor, axis=1) - for i in xrange(len(unstacked)): + for i in range(len(unstacked)): dense_float_names.append(_FEATURE_NAME_TEMPLATE % (key, i)) dense_floats.append(array_ops.reshape(unstacked[i], [-1, 1])) else: diff --git a/tensorflow/contrib/cmake/tf_python.cmake b/tensorflow/contrib/cmake/tf_python.cmake index 68234911a3..4b60460cb2 100755 --- a/tensorflow/contrib/cmake/tf_python.cmake +++ b/tensorflow/contrib/cmake/tf_python.cmake @@ -224,6 +224,7 @@ add_python_module("tensorflow/python/grappler") add_python_module("tensorflow/python/keras") add_python_module("tensorflow/python/keras/activations") add_python_module("tensorflow/python/keras/applications") +add_python_module("tensorflow/python/keras/applications/inception_resnet_v2") add_python_module("tensorflow/python/keras/applications/inception_v3") add_python_module("tensorflow/python/keras/applications/mobilenet") add_python_module("tensorflow/python/keras/applications/resnet50") diff --git a/tensorflow/contrib/data/__init__.py b/tensorflow/contrib/data/__init__.py index 6c46acf204..824ac4298f 100644 --- a/tensorflow/contrib/data/__init__.py +++ b/tensorflow/contrib/data/__init__.py @@ -30,6 +30,7 @@ See the @{$datasets$Importing Data} Programmer's Guide for an overview. @@make_saveable_from_iterator @@read_batch_features @@unbatch +@@parallel_interleave @@rejection_resample @@sloppy_interleave @@ -50,6 +51,7 @@ from tensorflow.contrib.data.python.ops.dataset_ops import get_single_element from tensorflow.contrib.data.python.ops.enumerate_ops import enumerate_dataset from tensorflow.contrib.data.python.ops.error_ops import ignore_errors from tensorflow.contrib.data.python.ops.grouping import group_by_window +from tensorflow.contrib.data.python.ops.interleave_ops import parallel_interleave from tensorflow.contrib.data.python.ops.interleave_ops import sloppy_interleave from tensorflow.contrib.data.python.ops.iterator_ops import make_saveable_from_iterator from tensorflow.contrib.data.python.ops.readers import FixedLengthRecordDataset diff --git a/tensorflow/contrib/eager/python/examples/mnist/mnist.py b/tensorflow/contrib/eager/python/examples/mnist/mnist.py index 3dd920415d..bfb7d5a900 100644 --- a/tensorflow/contrib/eager/python/examples/mnist/mnist.py +++ b/tensorflow/contrib/eager/python/examples/mnist/mnist.py @@ -191,9 +191,9 @@ def main(_): train_dir = None test_dir = None summary_writer = tf.contrib.summary.create_summary_file_writer( - train_dir, flush_secs=10) + train_dir, flush_millis=10000) test_summary_writer = tf.contrib.summary.create_summary_file_writer( - test_dir, flush_secs=10, name='test') + test_dir, flush_millis=10000, name='test') checkpoint_prefix = os.path.join(FLAGS.checkpoint_dir, 'ckpt') with tf.device(device): diff --git a/tensorflow/contrib/eager/python/examples/rnn_colorbot/rnn_colorbot.py b/tensorflow/contrib/eager/python/examples/rnn_colorbot/rnn_colorbot.py index 318962c634..609cbd2877 100644 --- a/tensorflow/contrib/eager/python/examples/rnn_colorbot/rnn_colorbot.py +++ b/tensorflow/contrib/eager/python/examples/rnn_colorbot/rnn_colorbot.py @@ -248,9 +248,9 @@ def main(_): log_dir = os.path.join(FLAGS.dir, "summaries") tf.gfile.MakeDirs(log_dir) train_summary_writer = tf.contrib.summary.create_summary_file_writer( - os.path.join(log_dir, "train"), flush_secs=10) + os.path.join(log_dir, "train"), flush_millis=10000) test_summary_writer = tf.contrib.summary.create_summary_file_writer( - os.path.join(log_dir, "eval"), flush_secs=10, name="eval") + os.path.join(log_dir, "eval"), flush_millis=10000, name="eval") with tf.device(device): for epoch in range(FLAGS.num_epochs): diff --git a/tensorflow/contrib/estimator/BUILD b/tensorflow/contrib/estimator/BUILD index a0f83ac105..6eb2cfdaca 100644 --- a/tensorflow/contrib/estimator/BUILD +++ b/tensorflow/contrib/estimator/BUILD @@ -7,6 +7,7 @@ package( licenses(["notice"]) # Apache 2.0 load("//tensorflow:tensorflow.bzl", "py_test") +load("//tensorflow:tensorflow.bzl", "cuda_py_test") filegroup( name = "all_files", @@ -30,6 +31,7 @@ py_library( ":head", ":logit_fns", ":multi_head", + ":replicate_model_fn", "//tensorflow/python:util", ], ) @@ -227,9 +229,69 @@ py_test( "//tensorflow/python:string_ops", "//tensorflow/python/estimator:metric_keys", "//tensorflow/python/estimator:model_fn", - "//tensorflow/python/estimator:prediction_keys", + "//tensorflow/python/ops/losses", "//tensorflow/python/saved_model:signature_constants", "//third_party/py/numpy", "@six_archive//:six", ], ) + +py_library( + name = "replicate_model_fn", + srcs = [ + "python/estimator/replicate_model_fn.py", + ], + srcs_version = "PY2AND3", + deps = [ + "//tensorflow/core:protos_all_py", + "//tensorflow/python:array_ops", + "//tensorflow/python:control_flow_ops", + "//tensorflow/python:device", + "//tensorflow/python:device_lib", + "//tensorflow/python:framework_ops", + "//tensorflow/python:gradients", + "//tensorflow/python:math_ops", + "//tensorflow/python:platform", + "//tensorflow/python:state_ops", + "//tensorflow/python:training", + "//tensorflow/python:variable_scope", + "//tensorflow/python:variables", + "//tensorflow/python/estimator:export_output", + "//tensorflow/python/estimator:model_fn", + "//tensorflow/python/estimator:util", + "@six_archive//:six", + ], +) + +cuda_py_test( + name = "replicate_model_fn_test", + size = "small", + srcs = ["python/estimator/replicate_model_fn_test.py"], + additional_deps = [ + "//tensorflow/python/estimator", + "//tensorflow/python/estimator:dnn", + "//tensorflow/python/estimator:export_export", + "//tensorflow/python/estimator:export_output", + "//tensorflow/python/estimator:model_fn", + "//tensorflow/python/estimator:numpy_io", + "//tensorflow/python/estimator:optimizers", + "//tensorflow/python/estimator:prediction_keys", + "//tensorflow/python/feature_column", + "//tensorflow/python/ops/losses", + "//tensorflow/python/saved_model:signature_constants", + "//tensorflow/python:array_ops", + "//tensorflow/python:client_testlib", + "//tensorflow/python:control_flow_ops", + "//tensorflow/python:framework_for_generated_wrappers", + "//tensorflow/python:framework_test_lib", + "//tensorflow/python:math_ops", + "//tensorflow/python:metrics", + "//tensorflow/python:platform", + "//tensorflow/python:summary", + "//tensorflow/python:training", + "//tensorflow/python:variable_scope", + "//tensorflow/python:variables", + ":replicate_model_fn", + ], + tags = ["requires-gpu-sm35"], +) diff --git a/tensorflow/contrib/estimator/python/estimator/replicate_model_fn.py b/tensorflow/contrib/estimator/python/estimator/replicate_model_fn.py new file mode 100644 index 0000000000..7005a647db --- /dev/null +++ b/tensorflow/contrib/estimator/python/estimator/replicate_model_fn.py @@ -0,0 +1,470 @@ +# Copyright 2017 The TensorFlow Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +"""Utilities to replicate model_fn's over local GPUs. + +This file contains util that allow to replicate `Estimator.model_fn` over +GPUs. Replicated version of a `model_fn` is returned that can subsequently +be used with `Estimator`. +""" + +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function + +import copy + +import six + +from tensorflow.core.framework import node_def_pb2 +from tensorflow.python.client import device_lib +from tensorflow.python.estimator import model_fn as model_fn_lib +from tensorflow.python.estimator import util +from tensorflow.python.estimator.export import export_output as export_output_lib +from tensorflow.python.framework import device as framework_device +from tensorflow.python.framework import ops as ops_lib +from tensorflow.python.ops import array_ops +from tensorflow.python.ops import control_flow_ops +from tensorflow.python.ops import gradients as gradients_lib +from tensorflow.python.ops import math_ops +from tensorflow.python.ops import state_ops +from tensorflow.python.ops import variable_scope +from tensorflow.python.ops import variables as variables_lib +from tensorflow.python.platform import tf_logging +from tensorflow.python.training import training_util + + +def replicate_model_fn(model_fn, optimizer_fn, devices=None): + """Replicate `Estimator.model_fn` over GPUs within a single host. + + The given `model_fn` specifies a single forward pass of a model. To replicate + such a model over GPUs, each GPU gets its own instance of the forward pass + (a.k.a. a tower). The input features and labels get sharded into the chunks + that correspond to the number of GPUs. Each tower computes its own loss based + on its input. For each such loss, gradients are computed. After that, the + available losses are summed to form aggregated loss. The available + gradients are summed too. Then, they update weights using the specified + optimizer. + + If `devices` are `None`, then all available GPUs are going to be used for + replication. If no GPUs are available, then the model is going to be + placed on the CPU. + + Two modes of local replication over available GPUs are supported: + 1) If exactly 1 GPU is detected, then variables and operations are placed + onto GPU. + 2) If more than 1 GPU is detected, then variables are going to be placed on + the CPU. Replicas of operations are placed on each individual GPU. + + Here is an example of how one might use their `model_fn` to run over GPUs: + ```python + def optimizer_fn(): + return tf.train.GradientDescentOptimizer(learning_rate=0.001) + ... + def model_fn(...): # See `model_fn` in `Estimator`. + loss = ... + if mode == tf.estimator.ModeKeys.TRAIN: + # See the section below on `EstimatorSpec.train_op`. + return EstimatorSpec(mode=mode, loss=loss, train_op=tf.noop()) + + # No change for `ModeKeys.EVAL` or `ModeKeys.PREDICT`. + return EstimatorSpec(...) + ... + classifier = tf.estimator.Estimator( + model_fn=replicate_model_fn.replicate_model_fn(model_fn, optimizer_fn)) + ``` + + On `EstimatorSpec.train_op`: + `model_fn` returns `EstimatorSpec.train_op` for + `tf.estimator.GraphKeys.TRAIN`. It is typically derived using an optimizer. + `replicate_model_fn` ignores the returned `EstimatorSpec.train_op`, so there + is no need to use an optimizer inside the user's `model_fn`. The + `EstimatorSpec.loss` subgraph is going to be executed, while + `EstimatorSpec.train_op` isn't going to be executed. One could pass + `train_op=tf.noop()` to `EstimatorSpec`. + + On sharding input features and labels: + Input features and labels are split for consumption by each tower. They are + split across the dimension 0. Features and labels need to be batch major. + + On reduction algorithms: + Certain algorithms were chosen for aggregating results of computations on + multiple towers: + - Losses from all towers are reduced using sum. + - Gradients are reduced using sum for each trainable variable. + - `eval_metrics_ops` are reduced per metric using `reduce_mean`. + - `EstimatorSpec.predictions` and `EstimatorSpec.export_outputs` are + reduced using concatenation. + - For all other fields of `EstimatorSpec` the values of the first tower + are taken. + + On replication of variables: + Variables are not duplicated between towers. Instead, they are placed on a + single device as defined above and shared across towers. + + Other current limitations: + - `predictions` are not supported for `ModeKeys.EVAL`. That is required for + `tf.contrib.estimator.add_metrics`. + + Args: + model_fn: `model_fn` as defined in `Estimator`. See the section above about + the train_op argument of `EstimatorSpec`. + optimizer_fn: a function that returns an optimizer instance. The function + may accept one `params` argument. This is the `params` argument as + defined by `Estimator`. See the `Estimator` documentation for details. + devices: Optional list of devices to replicate the model across. This + argument can be used to replice only on the subset of available GPUs. + If `None`, then all available GPUs are going to be used for replication. + If no GPUs are available, then the model is going to be placed on the CPU. + + Returns: + A replicated version of the supplied `model_fn`. Returned function that + conforms to the requirements of `Estimator`'s `model_fn` and can be used + instead of the supplied `model_fn`. + """ + if not devices: + devices = _get_local_devices('GPU') or _get_local_devices('CPU') + + is_a_single_gpu_case = len(devices) == 1 and 'GPU' in devices[0] + local_ps_device = '/{}:0'.format('GPU' if is_a_single_gpu_case else 'CPU') + + tf_logging.info('Replicating the `model_fn` across {}. Local parameter ' + 'server device is going to be {}.'.format( + devices, local_ps_device)) + + def replicated_model_fn(mode, features, labels, params=None, config=None): + """Replicated version of `model_fn` to be used instead.""" + feature_shards, label_shards = _split_batch( + features, labels, len(devices), device=local_ps_device) + tower_specs = _get_loss_towers( + model_fn=model_fn, + mode=mode, + features=feature_shards, + labels=label_shards, + params=params, + config=config, + devices=devices, + local_ps_device=local_ps_device) + + if mode == model_fn_lib.ModeKeys.TRAIN: + train_op = _minimize_towers(tower_specs, + _call_optimizer_fn(optimizer_fn, params)) + return _train_spec( + tower_specs, train_op, aggregation_device=local_ps_device) + elif mode == model_fn_lib.ModeKeys.EVAL: + return _eval_spec(tower_specs, aggregation_device=local_ps_device) + elif mode == model_fn_lib.ModeKeys.PREDICT: + return _predict_spec(tower_specs, aggregation_device=local_ps_device) + + return replicated_model_fn + + +def _get_local_devices(device_type): + local_device_protos = device_lib.list_local_devices() + return [ + device.name + for device in local_device_protos + if device.device_type == device_type + ] + + +def _split_batch(features, labels, number_of_shards, device): + """Split input features and labes into batches.""" + + def split_dictionary(dictionary): + shards = [{} for _ in range(number_of_shards)] + for name, tensor in six.iteritems(dictionary): + for i, shard in enumerate(array_ops.split(tensor, number_of_shards)): + shards[i][name] = shard + return shards + + with ops_lib.name_scope('split_inputs'): + with ops_lib.device(device): + if isinstance(features, dict): + feature_shards = split_dictionary(features) + else: + feature_shards = array_ops.split(features, number_of_shards) + + if labels is None: + label_shards = None + elif isinstance(labels, dict): + label_shards = split_dictionary(labels) + else: + label_shards = array_ops.split(labels, number_of_shards) + return feature_shards, label_shards + + +_DEFAULT_NAME_SCOPE_PATTERN = 'tower_{}' + + +def _get_loss_towers(model_fn, + mode, + features, + labels, + params, + config, + devices, + local_ps_device, + name_scope_pattern=_DEFAULT_NAME_SCOPE_PATTERN): + """Replicate the loss computation across devices.""" + tower_specs = [] + + model_fn_args = util.fn_args(model_fn) + optional_params = {} + if 'params' in model_fn_args: + optional_params['params'] = copy.deepcopy(params) + if 'config' in model_fn_args: + optional_params['config'] = copy.deepcopy(config) + + for i, device in enumerate(devices): + is_the_first_tower = (i == 0) + + device_setter = _local_device_setter( + worker_device=device, ps_device=local_ps_device) + + # We would like to preserve the names of the variables and ops that a user + # might be relying on. Names with prefix are going to resolve to variables + # and ops of the first tower. + name_scope = name_scope_pattern + if is_the_first_tower: + name_scope = '' + + with variable_scope.variable_scope('', reuse=not is_the_first_tower): + with ops_lib.name_scope(name_scope.format(i)): + with ops_lib.device(device_setter): + labels_shard = None + if labels: + labels_shard = labels[i] + + tower_specs.append( + model_fn( + mode=mode, + features=features[i], + labels=labels_shard, + **optional_params)) + return tower_specs + + +def _local_device_setter(ps_device, worker_device): + """A device setter that puts distributes Var/Ops to PS/workers.""" + ps_ops = ['Variable', 'VariableV2', 'VarHandleOp'] + + def local_device_chooser(op): + current_device = framework_device.DeviceSpec.from_string(op.device or '') + + node_def = op if isinstance(op, node_def_pb2.NodeDef) else op.node_def + if node_def.op in ps_ops: + ps_device_spec = framework_device.DeviceSpec.from_string( + '{}'.format(ps_device)) + + ps_device_spec.merge_from(current_device) + return ps_device_spec.to_string() + else: + worker_device_spec = framework_device.DeviceSpec.from_string( + worker_device or '') + worker_device_spec.merge_from(current_device) + return worker_device_spec.to_string() + + return local_device_chooser + + +def _minimize_towers(tower_specs, optimizer): + """Aggregate and apply gradients for computed losses.""" + grad_lists = {} + for tower_spec in tower_specs: + with ops_lib.device(tower_spec.loss.device): + variables = variables_lib.trainable_variables() + gradients = gradients_lib.gradients(tower_spec.loss, variables) + + for var, grad in zip(variables, gradients): + if grad is not None: + grad_lists.setdefault(var, []).append(grad) + + aggregated_grads = [] + with ops_lib.name_scope('gradient_aggregating'): + for var, grads in six.iteritems(grad_lists): + grad = _compute_sum_on_device(grads, var.device) + aggregated_grads.append((grad, var)) + + train_op = optimizer.apply_gradients( + aggregated_grads, global_step=training_util.get_global_step()) + + return train_op + + +def _call_optimizer_fn(optimizer_fn, params): + arguments = {} + optimizer_fn_arguments = util.fn_args(optimizer_fn) + if 'params' in optimizer_fn_arguments: + arguments['params'] = params + return optimizer_fn(**arguments) + + +def _compute_sum_on_device(values, device, name=None): + with ops_lib.device(device): + return math_ops.add_n(values, name=name) + + +def _train_spec(tower_specs, + train_op, + aggregation_device, + aggregated_loss_name='loss'): + """Populate replicated EstimatorSpec for `GraphKeys.TRAIN`.""" + estimator_spec = tower_specs[0]._asdict() + estimator_spec['mode'] = model_fn_lib.ModeKeys.TRAIN + estimator_spec['train_op'] = train_op + estimator_spec['loss'] = _compute_sum_on_device( + [spec.loss for spec in tower_specs], aggregation_device, + aggregated_loss_name) + return model_fn_lib.EstimatorSpec(**estimator_spec) + + +def _eval_spec(tower_specs, aggregation_device, aggregated_loss_name='loss'): + """Populate replicated EstimatorSpec for `GraphKeys.EVAL`.""" + estimator_spec = tower_specs[0]._asdict() + estimator_spec['mode'] = model_fn_lib.ModeKeys.EVAL + estimator_spec['loss'] = _compute_sum_on_device( + [spec.loss for spec in tower_specs], aggregation_device, + aggregated_loss_name) + + eval_metric_ops_lists = {} + for tower_spec in tower_specs: + metrics = tower_spec.eval_metric_ops or {} + for name, (_, update_op) in six.iteritems(metrics): + update_ops = eval_metric_ops_lists.setdefault(name, ([])) + update_ops.append(update_op) + + eval_metric_ops = {} + for name, (metric_tensor, _) in six.iteritems(tower_specs[0].eval_metric_ops): + with ops_lib.control_dependencies(eval_metric_ops_lists[name]): + # This operation reduces local variables across all metrics, yet is + # called for every metric. This is redundant and it's done because + # it is hard to know what local variables correspond to what metric. + # Estimator is going to execute all `reduced_update_op`s as part of + # a group inside a single `Session.run()` call, which will avoid duplicate + # computation. + reduced_update_op = _reduce_metric_variables(len(tower_specs)) + eval_metric_ops[name] = (metric_tensor, reduced_update_op) + + estimator_spec['eval_metric_ops'] = eval_metric_ops + return model_fn_lib.EstimatorSpec(**estimator_spec) + + +def _reduce_metric_variables(number_of_towers): + """Aggregate local variables used in metrics into the first tower.""" + if number_of_towers == 1: + return control_flow_ops.no_op() + + metric_variables = ops_lib.get_collection(ops_lib.GraphKeys.METRIC_VARIABLES) + variables_per_tower = len(metric_variables) // number_of_towers + + if len(metric_variables) % number_of_towers != 0: + raise ValueError( + 'Different `EstimatorSpec.eval_metric_ops` across `model_fn()` calls.' + ' Expected {} local variables, but got {} instead.'.format( + variables_per_tower * number_of_towers, len(metric_variables))) + + # `metric_variables` has the size of `variables_per_tower` x + # number_of_towers. Each tower is produced by calling the same model_fn. + # First `variables_per_tower` correspond to the first tower. Each such + # variable has an replica at the `(variables_per_tower * i)` position, where + # `i` is `[1.. number_of_towers]`. We are going to add values from replicas + # to each variable of the first tower. We then zero out replica values, so + # that `_reduce_metric_variables` operation is idempotent. If a metric + # is then computed based on local variables from the first tower, then the + # resulting metric is an estimate for all `number_of_towers` towers. + ops = [] + for i in range(0, variables_per_tower): + next_replica_id = i + variables_per_tower + replicas = [ + metric_variables[replica_id] + for replica_id in range(next_replica_id, len(metric_variables), + variables_per_tower) + ] # `replicas` doesn't contain the first-tower variable. + + reduce_op = state_ops.assign_add(metric_variables[i], + math_ops.add_n(replicas)) + + with ops_lib.control_dependencies([reduce_op]): + for replica in replicas: + zeros_for_replica = array_ops.zeros( + array_ops.shape(replica), dtype=replica.dtype) + zero_out_replica_op = state_ops.assign(replica, zeros_for_replica) + ops.append(zero_out_replica_op) + + return control_flow_ops.group(*ops) + + +def _predict_spec(tower_specs, aggregation_device): + """Populate replicated EstimatorSpec for `GraphKeys.PREDICT`.""" + estimator_spec = tower_specs[0]._asdict() + estimator_spec['mode'] = model_fn_lib.ModeKeys.PREDICT + + with ops_lib.device(aggregation_device): + estimator_spec['predictions'] = _concat_tensor_dicts( + *[tower_spec.predictions for tower_spec in tower_specs]) + + export_outputs_dict = _dict_concat( + *[tower_spec.export_outputs for tower_spec in tower_specs]) + + export_outputs = {} + for name, export_output_list in six.iteritems(export_outputs_dict): + if isinstance(export_output_list[0], export_output_lib.PredictOutput): + export_outputs[name] = export_output_lib.PredictOutput( + outputs=_concat_tensor_dicts(*[ + export_output.outputs for export_output in export_output_list + ])) + elif isinstance(export_output_list[0], + export_output_lib.RegressionOutput): + export_outputs[name] = export_output_lib.RegressionOutput( + value=array_ops.concat( + [export_output.value for export_output in export_output_list], + axis=0)) + elif isinstance(export_output_list[0], + export_output_lib.ClassificationOutput): + scores = None + if export_output_list[0].scores is not None: + scores = array_ops.concat( + [export_output.scores for export_output in export_output_list], + axis=0) + + classes = None + if export_output_list[0].classes is not None: + classes = array_ops.stack( + [export_output.classes for export_output in export_output_list], + axis=0) + + export_outputs[name] = export_output_lib.ClassificationOutput( + scores=scores, classes=classes) + + estimator_spec['export_outputs'] = export_outputs + return model_fn_lib.EstimatorSpec(**estimator_spec) + + +def _concat_tensor_dicts(*tensor_dicts): + return { + name: array_ops.concat(tensors, axis=0, name=name) + for name, tensors in six.iteritems(_dict_concat(*tensor_dicts)) + } + + +def _dict_concat(*dicts): + list_dict = {} + for d in dicts: + if d is None: + continue + + for k, v in six.iteritems(d): + list_dict.setdefault(k, []).append(v) + return list_dict diff --git a/tensorflow/contrib/estimator/python/estimator/replicate_model_fn_test.py b/tensorflow/contrib/estimator/python/estimator/replicate_model_fn_test.py new file mode 100644 index 0000000000..10b47fba5a --- /dev/null +++ b/tensorflow/contrib/estimator/python/estimator/replicate_model_fn_test.py @@ -0,0 +1,901 @@ +# Copyright 2017 The TensorFlow Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +"""Tests for utilities that replicate `Estimator.model_fn` over GPUs.""" + +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function + +import re +import shutil +import tempfile +import numpy as np +import six + +from tensorflow.contrib.estimator.python.estimator import replicate_model_fn +from tensorflow.python.estimator import estimator as estimator_lib +from tensorflow.python.estimator import model_fn as model_fn_lib +from tensorflow.python.estimator.canned import dnn +from tensorflow.python.estimator.canned import optimizers +from tensorflow.python.estimator.canned import prediction_keys +from tensorflow.python.estimator.export import export +from tensorflow.python.estimator.export import export_output +from tensorflow.python.estimator.inputs import numpy_io +from tensorflow.python.feature_column import feature_column +from tensorflow.python.framework import constant_op +from tensorflow.python.framework import dtypes +from tensorflow.python.framework import ops as ops_lib +from tensorflow.python.framework import test_util +from tensorflow.python.ops import array_ops +from tensorflow.python.ops import control_flow_ops +from tensorflow.python.ops import math_ops +from tensorflow.python.ops import metrics as metrics_lib +from tensorflow.python.ops import variable_scope +from tensorflow.python.ops import variables +from tensorflow.python.ops.losses import losses +from tensorflow.python.platform import gfile +from tensorflow.python.platform import test +from tensorflow.python.saved_model import signature_constants +from tensorflow.python.summary.writer import writer_cache +from tensorflow.python.training import gradient_descent + + +class DNNClassifierIntegrationTest(test_util.TensorFlowTestCase): + + def setUp(self): + self._model_dir = tempfile.mkdtemp() + + def test_complete_flow(self): + n_classes = 3 + input_dimension = 2 + batch_size = 12 + + data = np.linspace( + 0., n_classes - 1., batch_size * input_dimension, dtype=np.float32) + x_data = data.reshape(batch_size, input_dimension) + y_data = np.reshape(self._as_label(data[:batch_size]), (batch_size, 1)) + train_input_fn = numpy_io.numpy_input_fn( + x={'x': x_data}, + y=y_data, + batch_size=batch_size, + num_epochs=None, + shuffle=True) + eval_input_fn = numpy_io.numpy_input_fn( + x={'x': x_data}, y=y_data, batch_size=batch_size, shuffle=False) + predict_input_fn = numpy_io.numpy_input_fn( + x={'x': x_data}, batch_size=batch_size, shuffle=False) + + feature_columns = [ + feature_column.numeric_column('x', shape=(input_dimension,)) + ] + + estimator = dnn.DNNClassifier( + hidden_units=(2, 2), + feature_columns=feature_columns, + n_classes=n_classes, + model_dir=self._model_dir) + + def optimizer_fn(): + return optimizers.get_optimizer_instance('Adagrad', learning_rate=0.05) + + # TODO(isaprykin): Switch Estimator to use allow_soft_placement=True + # during export_savedmodel and then switch this test to replicate over + # GPUs instead of CPUs. + estimator = estimator_lib.Estimator( + model_fn=replicate_model_fn.replicate_model_fn( + estimator.model_fn, + optimizer_fn, + devices=['/cpu:0', '/cpu:0', '/cpu:0']), + model_dir=estimator.model_dir, + config=estimator.config, + params=estimator.params) + + num_steps = 10 + estimator.train(train_input_fn, steps=num_steps) + + scores = estimator.evaluate(eval_input_fn) + self.assertEqual(num_steps, scores[ops_lib.GraphKeys.GLOBAL_STEP]) + self.assertIn('loss', six.iterkeys(scores)) + + predicted_proba = np.array([ + x[prediction_keys.PredictionKeys.PROBABILITIES] + for x in estimator.predict(predict_input_fn) + ]) + self.assertAllEqual((batch_size, n_classes), predicted_proba.shape) + + feature_spec = feature_column.make_parse_example_spec(feature_columns) + serving_input_receiver_fn = export.build_parsing_serving_input_receiver_fn( + feature_spec) + export_dir = estimator.export_savedmodel(tempfile.mkdtemp(), + serving_input_receiver_fn) + self.assertTrue(gfile.Exists(export_dir)) + + def _as_label(self, data_in_float): + return np.rint(data_in_float).astype(np.int64) + + def tearDown(self): + if self._model_dir: + writer_cache.FileWriterCache.clear() + shutil.rmtree(self._model_dir) + + +class ReplicateModelTest(test_util.TensorFlowTestCase): + + def model_fn(self, mode, features, labels, params): + c = variable_scope.get_variable( + 'c', + initializer=constant_op.constant(10, dtype=dtypes.float64), + dtype=dtypes.float64) + + predictions = math_ops.multiply(features, c) + + loss = None + if mode is not model_fn_lib.ModeKeys.PREDICT: + loss = losses.absolute_difference( + labels=labels, + predictions=predictions, + reduction=losses.Reduction.SUM) + loss = math_ops.reduce_sum(loss) + + metrics = { + 'accuracy': metrics_lib.accuracy(labels, predictions), + 'auc': metrics_lib.auc(labels, predictions) + } + + return model_fn_lib.EstimatorSpec( + mode=mode, + loss=loss, + eval_metric_ops=metrics, + predictions={'probabilities': predictions}, + train_op=control_flow_ops.no_op()) # This train_op isn't actually used. + + def optimizer_fn(self, params): + return gradient_descent.GradientDescentOptimizer(params['learning_rate']) + + @property + def params(self): + params = {} + params['learning_rate'] = 1.0 + return params + + def test_train(self): + features = np.array([[1.0], [2.0]]) + labels = np.array([[1.0], [2.0]]) + + with self.test_session() as session: + replicated_model_fn = replicate_model_fn.replicate_model_fn( + self.model_fn, self.optimizer_fn, devices=['/gpu:0', '/gpu:1']) + estimator_spec = replicated_model_fn(model_fn_lib.ModeKeys.TRAIN, + features, labels, self.params) + session.run(variables.global_variables_initializer()) + + # loss = feature * c - label + total_loss = (1.0 * 10 - 1.0) + (2.0 * 10 - 2.0) + self.assertEqual(total_loss, session.run(estimator_spec.loss)) + + # loss' of c is 3. + # new value of c = 10 - learning rate * 3 = 7.0. + session.run(estimator_spec.train_op) + with variable_scope.variable_scope('', reuse=True): + c = variable_scope.get_variable('c', dtype=dtypes.float64) + self.assertEqual(7.0, session.run(c)) + + def test_train_spec_with_optimizer_without_params(self): + + def optimizer_fn_without_params(): + return gradient_descent.GradientDescentOptimizer(learning_rate=1.0) + + features = np.array([[1.0], [2.0]]) + labels = np.array([[1.0], [2.0]]) + + with self.test_session() as session: # pylint: disable=unused-variable + replicated_model_fn = replicate_model_fn.replicate_model_fn( + self.model_fn, + optimizer_fn_without_params, + devices=['/gpu:0', '/gpu:1']) + # This call is going to fail if `replicated_model_fn` is still passing + # `params` inside `optimizer_fn`, even though the latter doesn't take any: + estimator_spec = replicated_model_fn(model_fn_lib.ModeKeys.TRAIN, + features, labels, self.params) + del estimator_spec + + def test_eval(self): + features = np.array([[0.01], [0.002]]) + labels = np.array([[0.01], [0.02]]) + + with self.test_session() as session: + replicated_model_fn = replicate_model_fn.replicate_model_fn( + self.model_fn, self.optimizer_fn, devices=['/gpu:0', '/gpu:1']) + estimator_spec = replicated_model_fn(model_fn_lib.ModeKeys.EVAL, features, + labels, self.params) + session.run(variables.local_variables_initializer()) + session.run(variables.global_variables_initializer()) + + accuracy, a = estimator_spec.eval_metric_ops['accuracy'] + auc, b = estimator_spec.eval_metric_ops['auc'] + + session.run([a, b]) + accuracy = session.run(accuracy) + auc = session.run(auc) + + # Accuracy is 0.0 (no match) in the first tower. + # Accuracy is 1.0 (match) in the second tower, since the feature + # times weight "c" happened to be equal to the label. + total_loss = ((0.01 * 10 - 0.01) + (0.002 * 10 - 0.02)) + + self.assertNear((0.0 + 1.0) / 2.0, accuracy, 0.01) + self.assertEqual(0, auc) + self.assertNear(total_loss, session.run(estimator_spec.loss), 0.01) + + def test_predict(self): + features = np.array([[0.01], [0.002]]) + labels = np.array([[0.01], [0.02]]) + + with self.test_session() as session: + replicated_model_fn = replicate_model_fn.replicate_model_fn( + self.model_fn, self.optimizer_fn, devices=['/gpu:0', '/gpu:1']) + estimator_spec = replicated_model_fn(model_fn_lib.ModeKeys.PREDICT, + features, labels, self.params) + session.run(variables.global_variables_initializer()) + + self.assertAllClose({ + 'probabilities': np.array([[0.1], [0.02]]) + }, session.run(estimator_spec.predictions)) + + def test_train_single_tower(self): + features = np.array([[1.0], [2.0]]) + labels = np.array([[1.0], [2.0]]) + + with self.test_session() as session: + replicated_model_fn = replicate_model_fn.replicate_model_fn( + self.model_fn, self.optimizer_fn) + estimator_spec = replicated_model_fn(model_fn_lib.ModeKeys.TRAIN, + features, labels, self.params) + session.run(variables.global_variables_initializer()) + + # loss = feature * c - label + total_loss = (1.0 * 10 - 1.0) + (2.0 * 10 - 2.0) + self.assertEqual(total_loss, session.run(estimator_spec.loss)) + + # loss' of c is 3. + # new value of c = 10 - learning rate * 3 = 7.0. + session.run(estimator_spec.train_op) + with variable_scope.variable_scope('', reuse=True): + c = variable_scope.get_variable('c', dtype=dtypes.float64) + self.assertEqual(7.0, session.run(c)) + + def test_eval_single_tower(self): + features = np.array([[0.01], [0.002]]) + labels = np.array([[0.01], [0.02]]) + + with self.test_session() as session: + replicated_model_fn = replicate_model_fn.replicate_model_fn( + self.model_fn, self.optimizer_fn, devices=['/gpu:0']) + estimator_spec = replicated_model_fn(model_fn_lib.ModeKeys.EVAL, features, + labels, self.params) + session.run(variables.local_variables_initializer()) + session.run(variables.global_variables_initializer()) + + accuracy, a = estimator_spec.eval_metric_ops['accuracy'] + auc, b = estimator_spec.eval_metric_ops['auc'] + + session.run([a, b]) + accuracy = session.run(accuracy) + auc = session.run(auc) + + # Accuracy is 0.0 (no match) in the first tower. + # Accuracy is 1.0 (match) in the second tower, since the feature + # times weight "c" happened to be equal to the label. + total_loss = ((0.01 * 10 - 0.01) + (0.002 * 10 - 0.02)) + + self.assertNear((0.0 + 1.0) / 2.0, accuracy, 0.01) + self.assertEqual(0, auc) + self.assertNear(total_loss, session.run(estimator_spec.loss), 0.01) + + def test_predict_single_tower(self): + features = np.array([[0.01], [0.002]]) + labels = np.array([[0.01], [0.02]]) + + with self.test_session() as session: + replicated_model_fn = replicate_model_fn.replicate_model_fn( + self.model_fn, self.optimizer_fn, devices=['/gpu:0']) + estimator_spec = replicated_model_fn(model_fn_lib.ModeKeys.PREDICT, + features, labels, self.params) + session.run(variables.global_variables_initializer()) + + self.assertAllClose({ + 'probabilities': np.array([[0.1], [0.02]]) + }, session.run(estimator_spec.predictions)) + + +class GetLossTowersTest(test_util.TensorFlowTestCase): + + def model_fn(self, mode, features, labels, params): + c = variable_scope.get_variable( + 'c', + initializer=constant_op.constant(0.25, dtype=dtypes.float64), + dtype=dtypes.float64) + + predictions = math_ops.add(np.array([0.1, 0.2, 0.3, features[0]]), c) + labels = np.array([0.1, 0.2, 0.3, labels[0]]) + + loss = losses.absolute_difference( + labels=labels, predictions=predictions, reduction=losses.Reduction.SUM) + + return model_fn_lib.EstimatorSpec(mode=mode, loss=math_ops.reduce_sum(loss)) + + def test_gradients_are_computed(self): + with self.test_session() as session: + tower_specs = replicate_model_fn._get_loss_towers( + self.model_fn, + mode=None, + features=[[0.6], [1.6]], + labels=[[0.6], [0.6]], + params=None, + config=None, + devices=['/gpu:0', '/gpu:1'], + local_ps_device='/gpu:0', + name_scope_pattern='test_tower_{}') + session.run(variables.global_variables_initializer()) + + self.assertEqual(len(tower_specs), 2) + + self.assertEqual('/device:GPU:0', tower_specs[0].loss.device) + self.assertEqual('Sum:0', tower_specs[0].loss.name) + self.assertEqual(1.0, session.run(tower_specs[0].loss)) + + self.assertEqual('/device:GPU:1', tower_specs[1].loss.device) + self.assertEqual('test_tower_1/Sum:0', tower_specs[1].loss.name) + # The input batch for the second tower had a loss that is 1.0 + # bigger: 0.6 vs 1.6. + self.assertEqual(2.0, session.run(tower_specs[1].loss)) + + self.assertEqual(1, len(variables.global_variables())) + self.assertEqual(1, len(variables.trainable_variables())) + + with variable_scope.variable_scope('', reuse=True): + c = variable_scope.get_variable('c', dtype=dtypes.float64) + self.assertEqual(0.25, session.run(c)) + + +class SplitBatchTest(test_util.TensorFlowTestCase): + + def evaluate_shards(self, first_list, second_list): + evaluate_items = lambda x: x.eval() + return list(map(evaluate_items, first_list)), list( + map(evaluate_items, second_list)) + + def test_simple_half_split(self): + with self.test_session() as session: # pylint: disable=unused-variable + features = [0.0, 1.0, 2.0, 3.0] + labels = [10.0, 11.0, 12.0, 13.0] + feature_shards, label_shards = replicate_model_fn._split_batch( + features, labels, 2, device='/gpu:0') + + feature_shards, label_shards = self.evaluate_shards( + feature_shards, label_shards) + + self.assertAllEqual([[0.0, 1.0], [2.0, 3.0]], feature_shards) + self.assertAllEqual([[10.0, 11.0], [12.0, 13.0]], label_shards) + + def test_to_each_their_own(self): + with self.test_session() as session: # pylint: disable=unused-variable + features = [0.0, 1.0, 2.0, 3.0] + labels = [10.0, 11.0, 12.0, 13.0] + feature_shards, label_shards = replicate_model_fn._split_batch( + features, labels, 4, device='/gpu:0') + + feature_shards, label_shards = self.evaluate_shards( + feature_shards, label_shards) + + self.assertAllEqual([[0.0], [1.0], [2.0], [3.0]], feature_shards) + self.assertAllEqual([[10.0], [11.0], [12.0], [13.0]], label_shards) + + def test_one_batch(self): + with self.test_session() as session: # pylint: disable=unused-variable + features = [0.0, 1.0, 2.0, 3.0] + labels = [10.0, 11.0, 12.0, 13.0] + feature_shards, label_shards = replicate_model_fn._split_batch( + features, labels, 1, device='/gpu:0') + + feature_shards, label_shards = self.evaluate_shards( + feature_shards, label_shards) + + self.assertAllEqual([[0.0, 1.0, 2.0, 3.0]], feature_shards) + self.assertAllEqual([[10.0, 11.0, 12.0, 13.0]], label_shards) + + def test_half_split_in_dictionary(self): + with self.test_session() as session: # pylint: disable=unused-variable + features = {'first': [0.0, 1.0, 2.0, 3.0], 'second': [4.0, 5.0, 6.0, 7.0]} + labels = [10.0, 11.0, 12.0, 13.0] + + feature_shards, label_shards = replicate_model_fn._split_batch( + features, labels, 2, device='/gpu:0') + + self.assertAllEqual([0.0, 1.0], feature_shards[0]['first'].eval()) + self.assertAllEqual([4.0, 5.0], feature_shards[0]['second'].eval()) + self.assertAllEqual([2.0, 3.0], feature_shards[1]['first'].eval()) + self.assertAllEqual([6.0, 7.0], feature_shards[1]['second'].eval()) + self.assertAllEqual([10.0, 11.0], label_shards[0].eval()) + self.assertAllEqual([12.0, 13.0], label_shards[1].eval()) + + def test_one_batch_in_dictionary(self): + with self.test_session() as session: # pylint: disable=unused-variable + features = {'first': [0.0, 1.0, 2.0, 3.0], 'second': [4.0, 5.0, 6.0, 7.0]} + labels = [10.0, 11.0, 12.0, 13.0] + + feature_shards, label_shards = replicate_model_fn._split_batch( + features, labels, 1, device='/gpu:0') + + self.assertAllEqual([0.0, 1.0, 2.0, 3.0], + feature_shards[0]['first'].eval()) + self.assertAllEqual([4.0, 5.0, 6.0, 7.0], + feature_shards[0]['second'].eval()) + self.assertAllEqual([10.0, 11.0, 12.0, 13.0], label_shards[0].eval()) + + def test_feature_and_label_dictionaries(self): + with self.test_session() as session: # pylint: disable=unused-variable + features = {'first': [0.0, 1.0, 2.0, 3.0], 'second': [4.0, 5.0, 6.0, 7.0]} + labels = {'first': [10.0, 11.0], 'second': [12.0, 13.0]} + + feature_shards, label_shards = replicate_model_fn._split_batch( + features, labels, 2, device='/gpu:0') + + self.assertAllEqual([0.0, 1.0], feature_shards[0]['first'].eval()) + self.assertAllEqual([4.0, 5.0], feature_shards[0]['second'].eval()) + self.assertAllEqual([2.0, 3.0], feature_shards[1]['first'].eval()) + self.assertAllEqual([6.0, 7.0], feature_shards[1]['second'].eval()) + self.assertAllEqual([10.0], label_shards[0]['first'].eval()) + self.assertAllEqual([12.0], label_shards[0]['second'].eval()) + self.assertAllEqual([11], label_shards[1]['first'].eval()) + self.assertAllEqual([13.0], label_shards[1]['second'].eval()) + + +class TrainSpecTest(test_util.TensorFlowTestCase): + + expected_predictions = {} + + def create_estimator_spec(self, loss): + return model_fn_lib.EstimatorSpec( + mode=model_fn_lib.ModeKeys.TRAIN, + loss=loss, + train_op=loss, # Not used; currently required. + predictions=self.expected_predictions) + + def create_constant_loss(self, loss_value): + return constant_op.constant(loss_value, dtype=dtypes.float64) + + def test_example(self): + with self.test_session() as session: + tower_losses = list(map(self.create_constant_loss, [2, 4, 6])) + tower_specs = list(map(self.create_estimator_spec, tower_losses)) + + expected_train_op = tower_losses[1] + + estimator_spec = replicate_model_fn._train_spec( + tower_specs, expected_train_op, aggregation_device='/gpu:0') + + self.assertEqual(expected_train_op, estimator_spec.train_op) + self.assertEqual(2 + 4 + 6, session.run(estimator_spec.loss)) + self.assertEqual(self.expected_predictions, estimator_spec.predictions) + + +class EvalSpecTest(test_util.TensorFlowTestCase): + + def create_estimator_spec(self, loss, metrics): + return model_fn_lib.EstimatorSpec( + mode=model_fn_lib.ModeKeys.EVAL, loss=loss, eval_metric_ops=metrics) + + def create_constant_loss(self, loss_value): + return constant_op.constant(loss_value, dtype=dtypes.float64) + + def create_eval_metrics(self, noise): + predictions = np.array([0.1, 0.2, 0.3, 0.6 + noise]) + labels = np.array([0.1, 0.2, 0.3, 0.6]) + + metrics = { + 'accuracy': metrics_lib.accuracy(labels, predictions), + 'auc': metrics_lib.auc(labels, predictions) + } + return metrics + + def test_example(self): + with self.test_session() as session: + tower_losses = map(self.create_constant_loss, [2, 4, 6]) + tower_metrics = map(self.create_eval_metrics, [0, 0.2, 0.3]) + tower_specs = [ + self.create_estimator_spec(l, m) + for l, m in zip(tower_losses, tower_metrics) + ] + session.run(variables.local_variables_initializer()) + + estimator_spec = replicate_model_fn._eval_spec( + tower_specs, aggregation_device='/device:GPU:0') + + accuracy, a = estimator_spec.eval_metric_ops['accuracy'] + auc, b = estimator_spec.eval_metric_ops['auc'] + + self.assertEqual('/device:CPU:0', accuracy.device) + self.assertEqual('/device:CPU:0', auc.device) + + session.run([a, b]) + accuracy = session.run(accuracy) + auc = session.run(auc) + + self.assertNear((12 - 2) / 12, accuracy, 0.01) + self.assertEqual(0, auc) + self.assertEqual(2 + 4 + 6, session.run(estimator_spec.loss)) + + def test_handles_single_tower(self): + with self.test_session() as session: + tower_losses = map(self.create_constant_loss, [5]) + tower_metrics = map(self.create_eval_metrics, [0.2]) + tower_specs = [ + self.create_estimator_spec(l, m) + for l, m in zip(tower_losses, tower_metrics) + ] + session.run(variables.local_variables_initializer()) + + estimator_spec = replicate_model_fn._eval_spec( + tower_specs, aggregation_device='/device:GPU:0') + + accuracy, a = estimator_spec.eval_metric_ops['accuracy'] + auc, b = estimator_spec.eval_metric_ops['auc'] + + self.assertEqual('/device:CPU:0', accuracy.device) + self.assertEqual('/device:CPU:0', auc.device) + + session.run([a, b]) + accuracy = session.run(accuracy) + auc = session.run(auc) + + self.assertNear((4 - 1) / 4, accuracy, 0.01) + self.assertEqual(0, auc) + self.assertEqual(5, session.run(estimator_spec.loss)) + + +class PredictSpecTest(test_util.TensorFlowTestCase): + + def model_fn(self, mode, features, labels, params): + c = variable_scope.get_variable( + 'c', + initializer=constant_op.constant(0.25, dtype=dtypes.float64), + dtype=dtypes.float64) + + predictions = math_ops.add(np.array([features[0], features[0]]), c) + + return model_fn_lib.EstimatorSpec( + mode=model_fn_lib.ModeKeys.PREDICT, + predictions={ + 'probabilities': predictions + }) + + def test_example(self): + with self.test_session() as session: + tower_specs = replicate_model_fn._get_loss_towers( + self.model_fn, + mode=None, + features=[[0.1], [0.2]], + labels=[[], []], + params=None, + config=None, + devices=['/gpu:0', '/gpu:1'], + local_ps_device='/gpu:0', + ) + session.run(variables.global_variables_initializer()) + + estimator_spec = replicate_model_fn._predict_spec( + tower_specs, aggregation_device='/gpu:0') + + self.assertEqual('/device:GPU:0', + estimator_spec.predictions['probabilities'].device) + self.assertAllClose({ + 'probabilities': np.array([0.35, 0.35, 0.45, 0.45]) + }, session.run(estimator_spec.predictions)) + + +class ReduceMetricVariablesTest(test_util.TensorFlowTestCase): + + def create_metric_variable(self, initial_value, name): + return variable_scope.variable( + initial_value, + trainable=False, + collections=[ops_lib.GraphKeys.METRIC_VARIABLES], + validate_shape=True, + name=name) + + def create_tower_metrics(self, tower_id): + with variable_scope.variable_scope('', reuse=(tower_id != 0)): + self.create_metric_variable(1.3 * (tower_id + 1), 'total') + self.create_metric_variable(2.3 * (tower_id + 1), 'count') + self.create_metric_variable( + np.array([3.3, 3.5, 3.7]) * (tower_id + 1), 'total') + + def test_example(self): + with self.test_session() as session: + for tower_id in range(3): + self.create_tower_metrics(tower_id) + + session.run( + variables.variables_initializer( + ops_lib.get_collection(ops_lib.GraphKeys.METRIC_VARIABLES))) + + session.run( + replicate_model_fn._reduce_metric_variables(number_of_towers=3)) + + # 1st tower = 1.3, 2.3, [3.3, 3.5, 3.7] + # 2nd tower = 2.6, 4.6, [6.6, 7.0, 7.4] + # 3rd tower = 3.9, 6.9, [9.9, 10.5, 11.1] + # Reduced = 7.8, 13.8, [19.8, 21.0, 22.2] + # Towers are accumulated in the first tower. + local_metrics = session.run( + ops_lib.get_collection(ops_lib.GraphKeys.METRIC_VARIABLES)) + + self.assertNear(7.8, local_metrics[0], 0.01) + self.assertNear(13.8, local_metrics[1], 0.01) + self.assertAllClose([19.8, 21., 22.1], local_metrics[2], 0.01) + self.assertNear(0.0, local_metrics[3], 0.01) + self.assertNear(0.0, local_metrics[4], 0.01) + self.assertAllClose([0.0, 0.0, 0.0], local_metrics[5], 0.01) + self.assertNear(0.0, local_metrics[6], 0.01) + self.assertNear(0.0, local_metrics[7], 0.01) + self.assertAllClose([0.0, 0.0, 0.0], local_metrics[8], 0.01) + + def test_reduce_is_idempotent(self): + with self.test_session() as session: + for tower_id in range(3): + self.create_tower_metrics(tower_id) + + session.run( + variables.variables_initializer( + ops_lib.get_collection(ops_lib.GraphKeys.METRIC_VARIABLES))) + + for _ in range(20): + session.run( + replicate_model_fn._reduce_metric_variables(number_of_towers=3)) + + local_metrics = session.run( + ops_lib.get_collection(ops_lib.GraphKeys.METRIC_VARIABLES)) + + self.assertNear(7.8, local_metrics[0], 0.01) + self.assertNear(13.8, local_metrics[1], 0.01) + self.assertAllClose([19.8, 21., 22.1], local_metrics[2], 0.01) + self.assertNear(0.0, local_metrics[3], 0.01) + self.assertNear(0.0, local_metrics[4], 0.01) + self.assertAllClose([0.0, 0.0, 0.0], local_metrics[5], 0.01) + self.assertNear(0.0, local_metrics[6], 0.01) + self.assertNear(0.0, local_metrics[7], 0.01) + self.assertAllClose([0.0, 0.0, 0.0], local_metrics[8], 0.01) + + def test_handles_single_tower(self): + with self.test_session() as session: + self.create_tower_metrics(0) + session.run( + variables.variables_initializer( + ops_lib.get_collection(ops_lib.GraphKeys.METRIC_VARIABLES))) + + session.run( + replicate_model_fn._reduce_metric_variables(number_of_towers=1)) + + local_metrics = session.run( + ops_lib.get_collection(ops_lib.GraphKeys.METRIC_VARIABLES)) + + self.assertNear(1.3, local_metrics[0], 0.01) + self.assertNear(2.3, local_metrics[1], 0.01) + self.assertAllClose([3.3, 3.5, 3.7], local_metrics[2], 0.01) + + def test_doesnt_accept_uneven_number_of_variables(self): + with self.test_session() as session: + for tower_id in range(3): + self.create_tower_metrics(tower_id) + self.create_metric_variable(-1.0, 'oddball') + + session.run( + variables.variables_initializer( + ops_lib.get_collection(ops_lib.GraphKeys.METRIC_VARIABLES))) + + with self.assertRaisesRegexp(ValueError, ''): + session.run( + replicate_model_fn._reduce_metric_variables(number_of_towers=3)) + + +class MergeExportOutputsTest(test_util.TensorFlowTestCase): + + def optimizer_fn(self): + return gradient_descent.GradientDescentOptimizer(1.0) + + def model_fn(self, mode, features, labels, params): + c = variable_scope.get_variable( + 'c', + initializer=constant_op.constant(10, dtype=dtypes.float64), + dtype=dtypes.float64) + + predictions = {'probabilities': math_ops.multiply(features, c)} + loss = losses.absolute_difference( + labels=labels, + predictions=predictions['probabilities'], + reduction=losses.Reduction.SUM) + + metrics = { + 'accuracy': metrics_lib.accuracy(labels, predictions['probabilities']), + 'auc': metrics_lib.auc(labels, predictions['probabilities']) + } + tensor_string_repr = str(features) + classes = constant_op.constant( + re.search('(split_inputs/split:[0-9])', tensor_string_repr).group(1), + dtype=dtypes.string) + + export_outputs = { + signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY: + export_output.PredictOutput(predictions), + 'classification_output': + export_output.ClassificationOutput(predictions['probabilities'], + classes), + 'classification_scores': + export_output.ClassificationOutput( + scores=predictions['probabilities']), + 'classification_classes': + export_output.ClassificationOutput(classes=classes), + 'regression_output': + export_output.RegressionOutput(predictions['probabilities']), + } + + return model_fn_lib.EstimatorSpec( + mode=mode, + loss=math_ops.reduce_sum(loss), + eval_metric_ops=metrics, + predictions=predictions, + train_op=loss, # This train_op isn't actually used. + export_outputs=export_outputs) + + def replicate_estimator_spec(self, session): + features = np.array([0.01, 0.002]) + labels = np.array([0.01, 0.02]) + + replicated_model_fn = replicate_model_fn.replicate_model_fn( + self.model_fn, self.optimizer_fn, devices=['/gpu:0', '/gpu:1']) + estimator_spec = replicated_model_fn(model_fn_lib.ModeKeys.PREDICT, + features, labels, {}) + session.run(variables.global_variables_initializer()) + return estimator_spec + + def test_merde_predict_output(self): + with self.test_session() as session: + estimator_spec = self.replicate_estimator_spec(session) + self.assertAllClose( + { + 'probabilities': np.array([0.1, 0.02]) + }, + session.run(estimator_spec.export_outputs[ + signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY].outputs)) + + def test_merge_classification_output_scores_classes(self): + with self.test_session() as session: + estimator_spec = self.replicate_estimator_spec(session) + self.assertAllClose( + [0.1, 0.02], + session.run( + estimator_spec.export_outputs['classification_output'].scores)) + self.assertAllEqual( + [b'split_inputs/split:0', b'split_inputs/split:1'], + session.run( + estimator_spec.export_outputs['classification_output'].classes)) + + def test_merge_classification_output_scores(self): + with self.test_session() as session: + estimator_spec = self.replicate_estimator_spec(session) + self.assertAllClose( + [0.1, 0.02], + session.run( + estimator_spec.export_outputs['classification_scores'].scores)) + self.assertEqual( + None, estimator_spec.export_outputs['classification_scores'].classes) + + def test_merge_classification_output_classes(self): + with self.test_session() as session: + estimator_spec = self.replicate_estimator_spec(session) + self.assertAllEqual( + [b'split_inputs/split:0', b'split_inputs/split:1'], + session.run( + estimator_spec.export_outputs['classification_classes'].classes)) + self.assertEqual( + None, estimator_spec.export_outputs['classification_classes'].scores) + + def test_merge_regression_output(self): + with self.test_session() as session: + estimator_spec = self.replicate_estimator_spec(session) + self.assertAllClose( + [0.1, 0.02], + session.run(estimator_spec.export_outputs['regression_output'].value)) + + +class GetLocalDevicesTest(test_util.TensorFlowTestCase): + + def test_there_is_at_least_a_cpu(self): + self.assertTrue(replicate_model_fn._get_local_devices('CPU')) + + def test_there_is_no_xpu(self): + self.assertFalse( + replicate_model_fn._get_local_devices('XPU')) # XPU doesn't exist. + + def test_whether_there_is_a_gpu(self): + self.assertEqual( + len(replicate_model_fn._get_local_devices('GPU')), + test.is_gpu_available()) + + +class LocalDeviceSetterTest(test_util.TensorFlowTestCase): + + def test_vars_are_on_ps_but_ops_are_on_workers(self): + local_device_setter = replicate_model_fn._local_device_setter( + ps_device='/device:GPU:3', worker_device='/device:GPU:2') + + with ops_lib.device(local_device_setter): + c = variables.Variable(0.01) + self.assertEqual('/device:GPU:3', c.device) + + cc = variables.Variable(0.02) + self.assertEqual('/device:GPU:3', cc.device) + + ccc = variables.Variable(0.03) + self.assertEqual('/device:GPU:3', ccc.device) + + c_op = array_ops.concat(c, axis=0) + self.assertEqual('/device:GPU:2', c_op.device) + + cc_op = array_ops.concat(cc, axis=0) + self.assertEqual('/device:GPU:2', cc_op.device) + + +class ComputeSumWithDevicePlacementTest(test_util.TensorFlowTestCase): + + def test_example(self): + with self.test_session() as session: + total = replicate_model_fn._compute_sum_on_device( + [1.0, 2.0, 3.0, 4.0], device='/device:GPU:0', name='test_sum') + + self.assertEqual('/device:GPU:0', total.device) + self.assertEqual('test_sum', total.op.name) + self.assertEqual(10.0, session.run(total)) + + +class ConcatTensorDictsTest(test_util.TensorFlowTestCase): + + def test_example(self): + tensor_dicts = [ + { + 'a': np.array([1.0, 2.0]), + 'b': np.array([11.0]), + 'c': np.array([21.0]), + }, + { + 'a': np.array([3.0]), + 'b': np.array([12.0, 13.0]), + }, + { + 'b': np.array([14.0]), + }, + ] + + with self.test_session() as session: + self.assertAllClose({ + 'a': np.array([1.0, 2.0, 3.0]), + 'b': np.array([11.0, 12.0, 13.0, 14.0]), + 'c': np.array([21.0]), + }, session.run(replicate_model_fn._concat_tensor_dicts(*tensor_dicts))) + + +if __name__ == '__main__': + test.main() diff --git a/tensorflow/contrib/framework/BUILD b/tensorflow/contrib/framework/BUILD index 891425fd8c..e8dad886a1 100644 --- a/tensorflow/contrib/framework/BUILD +++ b/tensorflow/contrib/framework/BUILD @@ -24,6 +24,7 @@ tf_custom_op_py_library( "python/framework/__init__.py", "python/framework/checkpoint_utils.py", "python/framework/experimental.py", + "python/framework/graph_util.py", "python/framework/tensor_util.py", "python/ops/__init__.py", "python/ops/accumulate_n_v2.py", @@ -32,6 +33,7 @@ tf_custom_op_py_library( "python/ops/checkpoint_ops.py", "python/ops/ops.py", "python/ops/prettyprint_ops.py", + "python/ops/sort_ops.py", "python/ops/variables.py", ], dso = [ @@ -231,6 +233,17 @@ py_test( ], ) +py_test( + name = "graph_util_test", + srcs = ["python/framework/graph_util_test.py"], + srcs_version = "PY2AND3", + deps = [ + ":framework_py", + "//tensorflow/python:client_testlib", + "//tensorflow/python:platform", + ], +) + py_test( name = "tensor_util_test", srcs = ["python/framework/tensor_util_test.py"], @@ -307,6 +320,20 @@ py_test( ], ) +py_test( + name = "sort_ops_test", + size = "medium", + srcs = ["python/ops/sort_ops_test.py"], + srcs_version = "PY2AND3", + deps = [ + ":framework_py", + "//tensorflow/python:array_ops", + "//tensorflow/python:client_testlib", + "//tensorflow/python:random_ops", + "//third_party/py/numpy", + ], +) + filegroup( name = "all_files", srcs = glob( diff --git a/tensorflow/contrib/framework/__init__.py b/tensorflow/contrib/framework/__init__.py index 8421ba7c04..3f59261183 100644 --- a/tensorflow/contrib/framework/__init__.py +++ b/tensorflow/contrib/framework/__init__.py @@ -79,6 +79,8 @@ See the @{$python/contrib.framework} guide. @@load_embedding_initializer @@load_linear_multiclass_bias_initializer @@load_variable_slot_initializer + +@@sort """ from __future__ import absolute_import diff --git a/tensorflow/contrib/framework/python/framework/__init__.py b/tensorflow/contrib/framework/python/framework/__init__.py index c8e6a46854..2d49771ab7 100644 --- a/tensorflow/contrib/framework/python/framework/__init__.py +++ b/tensorflow/contrib/framework/python/framework/__init__.py @@ -21,6 +21,7 @@ from __future__ import print_function # pylint: disable=wildcard-import from tensorflow.contrib.framework.python.framework.checkpoint_utils import * from tensorflow.contrib.framework.python.framework.experimental import experimental +from tensorflow.contrib.framework.python.framework.graph_util import * from tensorflow.contrib.framework.python.framework.tensor_util import * # pylint: enable=wildcard-import from tensorflow.python.util import decorator_utils diff --git a/tensorflow/contrib/framework/python/framework/graph_util.py b/tensorflow/contrib/framework/python/framework/graph_util.py new file mode 100644 index 0000000000..8ab8711db4 --- /dev/null +++ b/tensorflow/contrib/framework/python/framework/graph_util.py @@ -0,0 +1,128 @@ +# Copyright 2017 The TensorFlow Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +"""Helpers to manipulate a tensor graph in python. +""" + +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function +import copy +import six + +# pylint: disable=unused-import +from tensorflow.core.framework import graph_pb2 +from tensorflow.core.framework import node_def_pb2 +from tensorflow.python.framework.graph_util_impl import _assert_nodes_are_present +from tensorflow.python.framework.graph_util_impl import _bfs_for_reachable_nodes +from tensorflow.python.framework.graph_util_impl import _extract_graph_summary +from tensorflow.python.framework.graph_util_impl import _node_name + +__all__ = ["fuse_op"] + + +def fuse_op(graph_def, input_nodes, output_nodes, output_dtypes, + output_quantized, op_name, op_type): + """Fuse subgraph between input_nodes and output_nodes into a single custom op. + + Args: + graph_def: A graph_pb2.GraphDef proto. + input_nodes: input nodes to the subgraph to be fused. + output_nodes: output nodes to the subgraph to be fused. + output_dtypes: A list of output datatypes for the custom op + output_quantized: A boolean flag that indicates if output is quantized + op_name: fused op name. + op_type: fused op type. + Returns: + The GraphDef of the new graph. + + Raises: + TypeError: If 'graph_def' is not a graph_pb2.GraphDef proto. + """ + + if not isinstance(graph_def, graph_pb2.GraphDef): + raise TypeError("graph_def must be a graph_pb2.GraphDef proto.") + + if isinstance(input_nodes, six.string_types): + raise TypeError("input_nodes must be a list.") + + if isinstance(output_nodes, six.string_types): + raise TypeError("output_nodes must be a list.") + + name_to_input_name, name_to_node, name_to_seq_num = _extract_graph_summary( + graph_def) + _assert_nodes_are_present(name_to_node, input_nodes + output_nodes) + + # Nodes upto and including input_nodes + reachable_by_input = _bfs_for_reachable_nodes(input_nodes, name_to_input_name) + # Nodes upto and including output_nodes + reachable_by_output = _bfs_for_reachable_nodes(output_nodes, + name_to_input_name) + + # Set of nodes in the list input_nodes + input_nodes_set = set(input_nodes) + + # Set of nodes in the list output_nodes + output_nodes_set = set(output_nodes) + + nodes_post_output = [] + for node in graph_def.node: + n = _node_name(node.name) + if n in reachable_by_output: + if n not in reachable_by_input and n not in output_nodes_set: + # n is between input and output, i.e., part of the fused op + next_to_visit = [n] + while next_to_visit: + cur_node = next_to_visit[0] + del next_to_visit[0] + if cur_node in reachable_by_input and cur_node not in input_nodes_set: + raise TypeError("Node %s uses input %s not in input_nodes." % + (n, cur_node)) + if cur_node not in input_nodes_set: + next_to_visit += name_to_input_name[cur_node] + else: + nodes_post_output.append(n) + + # Add all nodes upto the input nodes + out = graph_pb2.GraphDef() + reachable_by_input_sorted = sorted( + list(reachable_by_input), key=lambda n: name_to_seq_num[n]) + for node in reachable_by_input_sorted: + out.node.extend([copy.deepcopy(name_to_node[node])]) + + # Add the custom op + new_node = node_def_pb2.NodeDef() + for node in input_nodes: + new_node.input.append(node) + new_node.attr["_output_types"].list.type[:] = output_dtypes + new_node.attr["_output_quantized"].b = output_quantized + new_node.op = op_type + new_node.name = op_name + out.node.extend([new_node]) + + # Add the nodes in the output of the custom op + for index, n in enumerate(output_nodes): + assert len(name_to_node[n].input) == 1 + new_node = copy.deepcopy(name_to_node[n]) + del new_node.input[:] + new_node.input.append(op_name + (":" + str(index) if index != 0 else "")) + out.node.extend([new_node]) + + # Add the nodes post output_nodes + for n in nodes_post_output: + out.node.extend([copy.deepcopy(name_to_node[n])]) + + out.library.CopyFrom(graph_def.library) + out.versions.CopyFrom(graph_def.versions) + return out diff --git a/tensorflow/contrib/framework/python/framework/graph_util_test.py b/tensorflow/contrib/framework/python/framework/graph_util_test.py new file mode 100644 index 0000000000..87b992e22e --- /dev/null +++ b/tensorflow/contrib/framework/python/framework/graph_util_test.py @@ -0,0 +1,61 @@ +# Copyright 2017 The TensorFlow Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +"""@graph_util tests.""" +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function + +from tensorflow.contrib.framework.python.framework import graph_util +from tensorflow.core.framework import graph_pb2 +from tensorflow.core.framework import node_def_pb2 +from tensorflow.core.framework import types_pb2 +from tensorflow.python.platform import test + + +def GetNewNode(name, op, input_nodes): + new_node = node_def_pb2.NodeDef() + new_node.op = op + new_node.name = name + for node in input_nodes: + new_node.input.append(node) + return new_node + + +class GraphUtilTest(test.TestCase): + + def testGraphUtil(self): + graph_def = graph_pb2.GraphDef() + node_a = GetNewNode('A', 'Placeholder', []) + node_b = GetNewNode('B', 'Op1', ['A']) + node_c = GetNewNode('C', 'Op1', ['B']) + node_d = GetNewNode('D', 'Op1', ['C']) + node_e = GetNewNode('E', 'Op1', ['D']) + graph_def.node.extend([node_a, node_b, node_c, node_d, node_e]) + fused_graph_def = graph_util.fuse_op( + graph_def, ['A'], ['D'], [types_pb2.DT_FLOAT], True, 'FusedOp', 'Op2') + self.assertEqual(len(fused_graph_def.node), 4) + self.assertEqual(fused_graph_def.node[0].name, 'A') + self.assertEqual(fused_graph_def.node[1].name, 'FusedOp') + self.assertEqual(fused_graph_def.node[1].input[0], 'A') + self.assertEqual(fused_graph_def.node[1].op, 'Op2') + self.assertEqual(fused_graph_def.node[1].attr['_output_quantized'].b, True) + self.assertEqual(fused_graph_def.node[1].attr['_output_types'].list.type, + [types_pb2.DT_FLOAT]) + self.assertEqual(fused_graph_def.node[2].name, 'D') + self.assertEqual(fused_graph_def.node[3].name, 'E') + + +if __name__ == '__main__': + test.main() diff --git a/tensorflow/contrib/framework/python/ops/__init__.py b/tensorflow/contrib/framework/python/ops/__init__.py index edef37cf0c..685bb94779 100644 --- a/tensorflow/contrib/framework/python/ops/__init__.py +++ b/tensorflow/contrib/framework/python/ops/__init__.py @@ -24,5 +24,6 @@ from tensorflow.contrib.framework.python.ops.arg_scope import * from tensorflow.contrib.framework.python.ops.checkpoint_ops import * from tensorflow.contrib.framework.python.ops.ops import * from tensorflow.contrib.framework.python.ops.prettyprint_ops import * +from tensorflow.contrib.framework.python.ops.sort_ops import * from tensorflow.contrib.framework.python.ops.variables import * # pylint: enable=wildcard-import diff --git a/tensorflow/contrib/framework/python/ops/sort_ops.py b/tensorflow/contrib/framework/python/ops/sort_ops.py new file mode 100644 index 0000000000..8f62f0ea7b --- /dev/null +++ b/tensorflow/contrib/framework/python/ops/sort_ops.py @@ -0,0 +1,113 @@ +# Copyright 2017 The TensorFlow Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +"""Support for sorting tensors. + +@@sort +""" + +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function + +from tensorflow.python.framework import ops as framework_ops +from tensorflow.python.framework import tensor_util +from tensorflow.python.ops import array_ops +from tensorflow.python.ops import math_ops +from tensorflow.python.ops import nn_ops + + +def sort(values, axis=-1, direction='ASCENDING', name=None): + """Sorts a tensor. + + Args: + values: 1-D or higher numeric `Tensor`. + axis: The axis along which to sort. The default is -1, which sorts the last + axis. + direction: The direction in which to sort the values (`'ASCENDING'` or + `'DESCENDING'`). + name: Optional name for the operation. + + Returns: + A `Tensor` with the same dtype and shape as `values`, with the elements + sorted along the given `axis`. + + Raises: + ValueError: If axis is not a constant scalar, or the direction is invalid. + """ + with framework_ops.name_scope(name, 'sort'): + if direction not in _SORT_IMPL: + raise ValueError('%s should be one of %s' % + (direction, ', '.join(sorted(_SORT_IMPL.keys())))) + # Axis must be an integer, not a Tensor. + axis = framework_ops.convert_to_tensor(axis, name='axis') + axis_static = tensor_util.constant_value(axis) + if axis.shape.ndims != 0 or axis_static is None: + raise ValueError('axis must be a constant scalar') + axis_static = int(axis_static) # Avoids NumPy casting error + + values = framework_ops.convert_to_tensor(values, name='values') + + return _SORT_IMPL[direction](values, axis_static) + + +def _descending_sort(values, axis): + """Sorts values in reverse using `top_k`. + + Args: + values: Tensor of numeric values. + axis: Index of the axis which values should be sorted along. + + Returns: + The sorted values. + """ + k = array_ops.shape(values)[axis] + rank = array_ops.rank(values) + # Fast path: sorting the last axis. + if axis == -1 or axis + 1 == values.get_shape().ndims: + return nn_ops.top_k(values, k)[0] + + # Otherwise, transpose the array. Swap axes `axis` and `rank - 1`. + if axis < 0: + # Make axis a Tensor with the real axis index if needed. + axis += rank + transposition = array_ops.concat( + [ + # Axes up to axis are unchanged. + math_ops.range(axis), + # Swap axis and rank - 1. + [rank - 1], + # Axes in [axis + 1, rank - 1) are unchanged. + math_ops.range(axis + 1, rank - 1), + # Swap axis and rank - 1. + [axis] + ], + axis=0) + top_k_input = array_ops.transpose(values, transposition) + values, unused_indices = nn_ops.top_k(top_k_input, k) + # transposition contains a single cycle of length 2 (swapping 2 elements), + # so it is an involution (it is its own inverse). + return array_ops.transpose(values, transposition) + + +def _ascending_sort(values, axis): + # Negate the values to get the ascending order from descending sort. + values_or_indices = _descending_sort(-values, axis) + return -values_or_indices + + +_SORT_IMPL = { + 'ASCENDING': _ascending_sort, + 'DESCENDING': _descending_sort, +} diff --git a/tensorflow/contrib/framework/python/ops/sort_ops_test.py b/tensorflow/contrib/framework/python/ops/sort_ops_test.py new file mode 100644 index 0000000000..d08ae502f1 --- /dev/null +++ b/tensorflow/contrib/framework/python/ops/sort_ops_test.py @@ -0,0 +1,95 @@ +# Copyright 2017 The TensorFlow Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# ============================================================================== +"""Tests for the sort wrapper.""" + +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function + +import numpy as np + +from tensorflow.contrib.framework.python.ops import sort_ops +from tensorflow.python.framework import constant_op +from tensorflow.python.framework import dtypes +from tensorflow.python.framework import errors +from tensorflow.python.ops import array_ops +from tensorflow.python.ops import random_ops +from tensorflow.python.platform import test + + +class SortTest(test.TestCase): + + def testRandom_lowDimensionality(self): + self._testRandom_lowDimensionality(negative_axis=False) + + def testRandom_lowDimensionality_negative(self): + self._testRandom_lowDimensionality(negative_axis=True) + + def _testRandom_lowDimensionality(self, negative_axis): + np.random.seed(42) + for _ in range(20): + rank = np.random.randint(1, 3) + shape = [np.random.randint(0, 20) for _ in range(rank)] + arr = np.random.random(shape) + sort_axis = np.random.choice(rank) + if negative_axis: + sort_axis = -1 - sort_axis + with self.test_session(): + self.assertAllEqual( + np.sort(arr, axis=sort_axis), + sort_ops.sort(constant_op.constant(arr), axis=sort_axis).eval()) + + def testRandom_highDimensionality(self): + np.random.seed(100) + for _ in range(20): + rank = np.random.randint(5, 15) + shape = [np.random.randint(1, 4) for _ in range(rank)] + arr = np.random.random(shape) + sort_axis = np.random.choice(rank) + with self.test_session(): + self.assertAllEqual( + np.sort(arr, axis=sort_axis), + sort_ops.sort(constant_op.constant(arr), axis=sort_axis).eval()) + + def testScalar(self): + # Create an empty scalar where the static shape is unknown. + zeros_length_1 = array_ops.zeros( + random_ops.random_uniform([1], minval=0, maxval=1, dtype=dtypes.int32), + dtype=dtypes.int32) + scalar = array_ops.zeros(zeros_length_1) + + sort = sort_ops.sort(scalar) + with self.test_session(): + with self.assertRaises(errors.InvalidArgumentError): + sort.eval() + + def testNegativeOutOfBounds_staticShape(self): + arr = constant_op.constant([3, 4, 5]) + with self.assertRaises(ValueError): + sort_ops.sort(arr, axis=-4) + + def testDescending(self): + arr = np.random.random((10, 5, 5)) + with self.test_session(): + self.assertAllEqual( + np.sort(arr, axis=0)[::-1], + sort_ops.sort( + constant_op.constant(arr), + axis=0, + direction='DESCENDING').eval()) + + +if __name__ == '__main__': + test.main() diff --git a/tensorflow/contrib/summary/summary_ops.py b/tensorflow/contrib/summary/summary_ops.py index ecfa6baeff..56e3198593 100644 --- a/tensorflow/contrib/summary/summary_ops.py +++ b/tensorflow/contrib/summary/summary_ops.py @@ -246,8 +246,8 @@ def image(name, tensor, bad_color=None, max_images=3, family=None): """Writes an image summary if possible.""" def function(tag, scope): - if bad_color is None: - bad_color_ = constant_op.constant([255, 0, 0, 255], dtype=dtypes.uint8) + bad_color_ = (constant_op.constant([255, 0, 0, 255], dtype=dtypes.uint8) + if bad_color is None else bad_color) # Note the identity to move the tensor to the CPU. return gen_summary_ops.write_image_summary( context.context().summary_writer_resource, diff --git a/tensorflow/contrib/tpu/profiler/BUILD b/tensorflow/contrib/tpu/profiler/BUILD index f6309e2e72..0e1fca3d3c 100644 --- a/tensorflow/contrib/tpu/profiler/BUILD +++ b/tensorflow/contrib/tpu/profiler/BUILD @@ -95,3 +95,10 @@ tf_proto_library_cc( cc_api_version = 2, visibility = ["//visibility:public"], ) + +tf_proto_library_cc( + name = "tf_op_stats_proto", + srcs = ["tf_op_stats.proto"], + cc_api_version = 2, + visibility = ["//visibility:public"], +) diff --git a/tensorflow/contrib/tpu/profiler/tf_op_stats.proto b/tensorflow/contrib/tpu/profiler/tf_op_stats.proto new file mode 100644 index 0000000000..5b2dbb3124 --- /dev/null +++ b/tensorflow/contrib/tpu/profiler/tf_op_stats.proto @@ -0,0 +1,127 @@ +// This proto describes the format of tensorflow operation level stats for +// profiling (in tensorboard) purpose. + +syntax = "proto2"; + +package tensorflow.tpu; + +// Result proto for OpMetrics. +message OpMetricsResult { + // True if this OP is executed on the device; False if it is executed on the + // host. + optional bool on_device = 1; + reserved 2; // was uint32 id. + // Name of this OP. + optional string name = 3; + // Rank of this OP. + optional uint64 rank = 4; + // The starting time in cycles of the last instance of this OP executed. + optional double last_starttime_in_cycles = 5; + // The ending time in cycles of the last instance of this OP executed. + optional double last_endtime_in_cycles = 6; + // If this OP (say A), is an immediate child of another OP (say B), this field + // stores the sum of duration in microseconds of A inside B. If A appears more + // than once in B, the duration of all A's appearances will be added together. + // This sum will be reset after the self-time of B is calculated so that it + // can be reused for a new parent OP. + optional double sum_of_duration_in_us_as_children = 7; + // Number of instances that this OP occurred. + optional uint64 occurrences = 8; + // Total time in microseconds spent in this OP (accumulated + // over all of its occurrences). + optional double total_time_in_us = 9; + // Total self time in microseconds spent in this OP + // (accumulated over all of its occurrences). + optional double total_self_time_in_us = 10; + // The total self time as a fraction of sum of all OP's + // total self time on the host. + optional double host_total_self_time_as_fraction_of_all_op_time = 11; + // Cumulative total self time in fraction on the host. + optional double host_cumulative_total_self_time_as_fraction_of_all_op_time = + 12; + // The total self time as a fraction of sum of all OP's + // total self time on the device. + optional double device_total_self_time_as_fraction_of_all_op_time = 13; + // Cumulative total self time in fraction on the device. + optional double device_cumulative_total_self_time_as_fraction_of_all_op_time = + 14; + // Total number of FLOPs incurred by this OP. + optional double total_flops = 15; + // Total time in microseconds that the MXU is occupied by this OP. + optional double total_bytes_accessed = 16; + // Total time in microseconds that the MXU is occupied by this OP. + optional double mxu_occupancy_in_us = 17; + // Total time in microseconds that the XU is occupied by this OP. + optional double xu_occupancy_in_us = 18; + // Total DMA access stall time in microseconds. + optional double total_dma_stall_in_us = 19; +} + +// Result proto for OpMetricsDb. +message OpMetricsDbResult { + // A bunch of OpMetricsResults. + repeated OpMetricsResult metrics_db = 1; +} + +// Result proto for StepInfo. +message StepInfoResult { + // The (micro) step number. + optional uint32 step_num = 1; + // The step duration in picoseconds. + optional uint64 duration_ps = 2; + // The infeed duration in picoseconds. + // Can turn into a map if we want a variable number of ops. + optional uint64 infeed_duration_ps = 3; +} + +// Result proto for a sequence of steps. +message StepSequenceResult { + // A sequence of StepInfoResults. + repeated StepInfoResult step_sequence = 1; +} + +// Result proto for a StepDatabase. +message StepDatabaseResult { + // A map from core_id to StepSequenceResult. + map step_sequence_per_core = 1; +} + +// Result proto for Dashboard data. +message DashboardResult { + // The total iteration time in nanoseconds. + optional double iteration_time_ns = 1; + // The total number of iterations. + optional int32 num_iterations = 2; + // The total computation time in nanoseconds. + optional double computation_time_ns = 3; + // The total number of computations. + optional int32 num_computations = 4; +} + +// Result proto for HloExtraInfo. +message HloExtraInfoResult { + // Category of the HLO op given by the compiler. + optional string category = 1; + // The long name of the HLO that includes the dimensions. + optional string long_name = 2; +} + +// Result proto for HloExtraInfoMap. +message HloExtraInfoMapResult { + // A map from HLO name to HloExtraInfo. + map hlo_extrainfo_map = 1; +} + +// Result proto for TfStatsHelper. +message TfOpStats { + // The result for the TF-metric database. + optional OpMetricsDbResult tf_metrics_db = 1; + // The result for the HLO-metric database. + optional OpMetricsDbResult hlo_metrics_db = 2; + // The result for the step database. + optional StepDatabaseResult step_db = 3; + // The result for the TPU dashboard. + optional DashboardResult dashboard = 4; + // The result for the HloExtraInfoMap. + optional HloExtraInfoMapResult hlo_extrainfo_map = 5; +} diff --git a/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py b/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py index 5a3b831429..060b3f9129 100644 --- a/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py +++ b/tensorflow/contrib/tpu/python/tpu/tpu_estimator.py @@ -66,7 +66,7 @@ _CROSS_REPLICA_SUM_OP = 'CrossReplicaSum' _RESERVED_PARAMS_KEYS = [_BATCH_SIZE_KEY] # TODO(b/65703635): Flip the value and remove all dead code. -_WRAP_INPUT_FN_INTO_WHILE_LOOP = False +_WRAP_INPUT_FN_INTO_WHILE_LOOP = True def _create_global_step(graph): diff --git a/tensorflow/core/BUILD b/tensorflow/core/BUILD index 7b535da0b2..9530af637e 100644 --- a/tensorflow/core/BUILD +++ b/tensorflow/core/BUILD @@ -1414,16 +1414,19 @@ LIB_INTERNAL_PUBLIC_HEADERS = tf_additional_lib_hdrs() + [ "platform/tracing.h", ] +# Replicated for lib_internal and lib_internal_impl. +LIB_INTERNAL_DEFINES = (tf_additional_lib_defines() + [ + "TF_USE_SNAPPY", + ] + tf_additional_verbs_lib_defines() + + tf_additional_mpi_lib_defines() + + tf_additional_gdr_lib_defines()) + cc_library( name = "lib_internal", srcs = LIB_INTERNAL_PRIVATE_HEADERS, hdrs = LIB_INTERNAL_PUBLIC_HEADERS, copts = tf_copts(), - defines = tf_additional_lib_defines() + [ - "TF_USE_SNAPPY", - ] + tf_additional_verbs_lib_defines() + - tf_additional_mpi_lib_defines() + - tf_additional_gdr_lib_defines(), + defines = LIB_INTERNAL_DEFINES, linkopts = select({ "//tensorflow:freebsd": [], "//tensorflow:windows": [], @@ -1477,6 +1480,7 @@ cc_library( ), hdrs = LIB_INTERNAL_PUBLIC_HEADERS, copts = tf_copts(), + defines = LIB_INTERNAL_DEFINES, deps = tf_additional_lib_deps() + [ ":lib_hash_crc32c_accelerate_internal", ":lib_proto_parsing", diff --git a/tensorflow/core/api_def/api_test.cc b/tensorflow/core/api_def/api_test.cc index ceeb172fa0..d95d958d5a 100644 --- a/tensorflow/core/api_def/api_test.cc +++ b/tensorflow/core/api_def/api_test.cc @@ -46,92 +46,218 @@ constexpr char kDefaultApiDefDir[] = "tensorflow/core/api_def/base_api"; constexpr char kOverridesFilePath[] = "tensorflow/cc/ops/op_gen_overrides.pbtxt"; -constexpr char kApiDefFileFormat[] = "api_def_%c.pbtxt"; -constexpr char kAlphabet[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"; +constexpr char kApiDefFileFormat[] = "api_def_%s.pbtxt"; +constexpr char kApiDefFilePattern[] = "api_def_*.pbtxt"; -// Get map from first character to ApiDefs for ops -// that start with that character. -std::unordered_map GenerateApiDef( - const OpList& ops, const OpGenOverrides& overrides) { +void FillBaseApiDef(ApiDef* api_def, const OpDef& op) { + api_def->set_graph_op_name(op.name()); + // Add arg docs + for (auto& input_arg : op.input_arg()) { + if (!input_arg.description().empty()) { + auto* api_def_in_arg = api_def->add_in_arg(); + api_def_in_arg->set_name(input_arg.name()); + api_def_in_arg->set_description(input_arg.description()); + } + } + for (auto& output_arg : op.output_arg()) { + if (!output_arg.description().empty()) { + auto* api_def_out_arg = api_def->add_out_arg(); + api_def_out_arg->set_name(output_arg.name()); + api_def_out_arg->set_description(output_arg.description()); + } + } + // Add attr docs + for (auto& attr : op.attr()) { + if (!attr.description().empty()) { + auto* api_def_attr = api_def->add_attr(); + api_def_attr->set_name(attr.name()); + api_def_attr->set_description(attr.description()); + } + } + // Add docs + api_def->set_summary(op.summary()); + api_def->set_description(op.description()); +} + +// Checks if arg1 should be before arg2 according to ordering in args. +bool CheckArgBefore(const ApiDef::Arg* arg1, const ApiDef::Arg* arg2, + const protobuf::RepeatedPtrField& args) { + for (auto& arg : args) { + if (arg.name() == arg2->name()) { + return false; + } else if (arg.name() == arg1->name()) { + return true; + } + } + return false; +} + +// Checks if attr1 should be before attr2 according to ordering in op_def. +bool CheckAttrBefore(const ApiDef::Attr* attr1, const ApiDef::Attr* attr2, + const OpDef& op_def) { + for (auto& attr : op_def.attr()) { + if (attr.name() == attr2->name()) { + return false; + } else if (attr.name() == attr1->name()) { + return true; + } + } + return false; +} + +// Applies renames to args. +void ApplyArgOverrides( + protobuf::RepeatedPtrField* args, + const protobuf::RepeatedPtrField& renames, + const protobuf::RepeatedPtrField& op_args, + const string& op_name) { + for (auto& rename : renames) { + // First check if rename is valid. + bool valid = false; + for (const auto& op_arg : op_args) { + if (op_arg.name() == rename.from()) { + valid = true; + } + } + QCHECK(valid) << rename.from() << " is not a valid argument for " + << op_name; + bool found_arg = false; + // If Arg is already in ApiDef, just update it. + for (int i = 0; i < args->size(); ++i) { + auto* arg = args->Mutable(i); + if (arg->name() == rename.from()) { + arg->set_rename_to(rename.to()); + found_arg = true; + break; + } + } + if (!found_arg) { // not in ApiDef, add a new arg. + auto* new_arg = args->Add(); + new_arg->set_name(rename.from()); + new_arg->set_rename_to(rename.to()); + } + } + // We don't really need a specific order here right now. + // However, it is clearer if order follows OpDef. + std::sort(args->pointer_begin(), args->pointer_end(), + [&](ApiDef::Arg* arg1, ApiDef::Arg* arg2) { + return CheckArgBefore(arg1, arg2, op_args); + }); +} + +// Returns existing attribute with the given name if such +// attribute exists. Otherwise, adds a new attribute and returns it. +ApiDef::Attr* FindOrAddAttr(ApiDef* api_def, const string attr_name) { + // If Attr is already in ApiDef, just update it. + for (int i = 0; i < api_def->attr_size(); ++i) { + auto* attr = api_def->mutable_attr(i); + if (attr->name() == attr_name) { + return attr; + } + } + // Add a new Attr. + auto* new_attr = api_def->add_attr(); + new_attr->set_name(attr_name); + return new_attr; +} + +// Applies renames and default values to attributes. +void ApplyAttrOverrides(ApiDef* api_def, const OpGenOverride& op_override, + const OpDef& op_def) { + for (auto& attr_rename : op_override.attr_rename()) { + auto* attr = FindOrAddAttr(api_def, attr_rename.from()); + attr->set_rename_to(attr_rename.to()); + } + + for (auto& attr_default : op_override.attr_default()) { + auto* attr = FindOrAddAttr(api_def, attr_default.name()); + *(attr->mutable_default_value()) = attr_default.value(); + } + // We don't really need a specific order here right now. + // However, it is clearer if order follows OpDef. + std::sort(api_def->mutable_attr()->pointer_begin(), + api_def->mutable_attr()->pointer_end(), + [&](ApiDef::Attr* attr1, ApiDef::Attr* attr2) { + return CheckAttrBefore(attr1, attr2, op_def); + }); +} + +void ApplyOverridesToApiDef(ApiDef* api_def, const OpDef& op, + const OpGenOverride& op_override) { + // Fill ApiDef with data based on op and op_override. + // Set visibility + if (op_override.skip()) { + api_def->set_visibility(ApiDef_Visibility_SKIP); + } else if (op_override.hide()) { + api_def->set_visibility(ApiDef_Visibility_HIDDEN); + } + // Add endpoints + if (!op_override.rename_to().empty()) { + api_def->add_endpoint()->set_name(op_override.rename_to()); + } else if (!op_override.alias().empty()) { + api_def->add_endpoint()->set_name(op.name()); + } + + for (auto& alias : op_override.alias()) { + auto* endpoint = api_def->add_endpoint(); + endpoint->set_name(alias); + } + + ApplyArgOverrides(api_def->mutable_in_arg(), op_override.input_rename(), + op.input_arg(), api_def->graph_op_name()); + ApplyArgOverrides(api_def->mutable_out_arg(), op_override.output_rename(), + op.output_arg(), api_def->graph_op_name()); + ApplyAttrOverrides(api_def, op_override, op); +} + +// Get map from ApiDef file path to corresponding ApiDefs proto. +std::unordered_map GenerateApiDef( + const string& api_def_dir, const OpList& ops, + const OpGenOverrides& overrides) { std::unordered_map name_to_override; for (const auto& op_override : overrides.op()) { name_to_override[op_override.name()] = op_override; } - std::unordered_map api_defs_map; + std::unordered_map api_defs_map; for (const auto& op : ops.op()) { CHECK(!op.name().empty()) << "Encountered empty op name: %s" << op.DebugString(); - const char file_id = toupper(op.name()[0]); - CHECK(isalpha(file_id)) << "Unexpected op name: " << op.name(); - ApiDef* api_def = api_defs_map[file_id].add_op(); - api_def->set_graph_op_name(op.name()); + string file_path = io::JoinPath(api_def_dir, kApiDefFileFormat); + file_path = strings::Printf(file_path.c_str(), op.name().c_str()); + ApiDef* api_def = api_defs_map[file_path].add_op(); + FillBaseApiDef(api_def, op); if (name_to_override.find(op.name()) != name_to_override.end()) { - const auto& op_override = name_to_override[op.name()]; - // Set visibility - if (op_override.skip()) { - api_def->set_visibility(ApiDef_Visibility_SKIP); - } else if (op_override.hide()) { - api_def->set_visibility(ApiDef_Visibility_HIDDEN); - } - // Add endpoints - if (!op_override.rename_to().empty()) { - auto* endpoint = api_def->add_endpoint(); - endpoint->set_name(op_override.rename_to()); - } else { - auto* endpoint = api_def->add_endpoint(); - endpoint->set_name(op.name()); - } - for (auto& alias : op_override.alias()) { - auto* endpoint = api_def->add_endpoint(); - endpoint->set_name(alias); - } - // Add attributes - for (auto& attr : op.attr()) { - auto* api_def_attr = api_def->add_attr(); - api_def_attr->set_name(attr.name()); - for (auto& attr_override : op_override.attr_default()) { - if (attr.name() == attr_override.name()) { - *(api_def_attr->mutable_default_value()) = attr_override.value(); - } - } - for (auto& attr_rename : op_override.attr_rename()) { - if (attr.name() == attr_rename.from()) { - api_def_attr->set_rename_to(attr_rename.to()); - } - } - } - } else { - auto* endpoint = api_def->add_endpoint(); - endpoint->set_name(op.name()); + ApplyOverridesToApiDef(api_def, op, name_to_override[op.name()]); } - // Add docs - api_def->set_summary(op.summary()); - api_def->set_description(op.description()); } return api_defs_map; } -// Reads golden api defs file with the given suffix. -string GetGoldenApiDefsStr(Env* env, const string& api_files_dir, char suffix) { - string file_path = strings::Printf( - io::JoinPath(api_files_dir, kApiDefFileFormat).c_str(), suffix); - if (env->FileExists(file_path).ok()) { +// Reads golden ApiDef files and returns a map from file name to ApiDef file +// contents. +std::unordered_map GetGoldenApiDefs( + Env* env, const string& api_files_dir) { + std::vector matching_paths; + TF_CHECK_OK(env->GetMatchingPaths( + io::JoinPath(api_files_dir, kApiDefFilePattern), &matching_paths)); + + std::unordered_map file_path_to_api_def; + for (auto& file_path : matching_paths) { string file_contents; - TF_EXPECT_OK(ReadFileToString(env, file_path, &file_contents)); - return file_contents; + TF_CHECK_OK(ReadFileToString(env, file_path, &file_contents)); + file_path_to_api_def[file_path] = file_contents; } - return ""; + return file_path_to_api_def; } void RunApiTest(bool update_api_def, const string& api_files_dir) { // Read C++ overrides file - string overrides_file_contents; + OpGenOverrides overrides; Env* env = Env::Default(); - TF_EXPECT_OK( - ReadFileToString(env, kOverridesFilePath, &overrides_file_contents)); + TF_EXPECT_OK(ReadTextProto(env, kOverridesFilePath, &overrides)); // Read all ops OpList ops; @@ -139,29 +265,22 @@ void RunApiTest(bool update_api_def, const string& api_files_dir) { const std::vector multi_line_fields = {"description"}; // Get expected ApiDefs - OpGenOverrides overrides; - auto new_api_defs_map = GenerateApiDef(ops, overrides); + const auto new_api_defs_map = GenerateApiDef(api_files_dir, ops, overrides); bool updated_at_least_one_file = false; + const auto golden_api_defs_map = GetGoldenApiDefs(env, api_files_dir); - for (char c : kAlphabet) { - string golden_api_defs_str = GetGoldenApiDefsStr(env, api_files_dir, c); - string new_api_defs_str = new_api_defs_map[c].DebugString(); + for (auto new_api_entry : new_api_defs_map) { + const auto& file_path = new_api_entry.first; + const auto& golden_api_defs_str = golden_api_defs_map.at(file_path); + string new_api_defs_str = new_api_entry.second.DebugString(); new_api_defs_str = PBTxtToMultiline(new_api_defs_str, multi_line_fields); if (golden_api_defs_str == new_api_defs_str) { continue; } if (update_api_def) { - string output_file_path = - io::JoinPath(api_files_dir, strings::Printf(kApiDefFileFormat, c)); - if (new_api_defs_str.empty()) { - std::cout << "Deleting " << output_file_path << "..." << std::endl; - TF_EXPECT_OK(env->DeleteFile(output_file_path)); - } else { - std::cout << "Updating " << output_file_path << "..." << std::endl; - TF_EXPECT_OK( - WriteStringToFile(env, output_file_path, new_api_defs_str)); - } + std::cout << "Updating " << file_path << "..." << std::endl; + TF_EXPECT_OK(WriteStringToFile(env, file_path, new_api_defs_str)); updated_at_least_one_file = true; } else { EXPECT_EQ(golden_api_defs_str, new_api_defs_str) @@ -170,6 +289,21 @@ void RunApiTest(bool update_api_def, const string& api_files_dir) { } } + for (const auto& golden_api_entry : golden_api_defs_map) { + const auto& file_path = golden_api_entry.first; + if (new_api_defs_map.find(file_path) == new_api_defs_map.end()) { + if (update_api_def) { + std::cout << "Deleting " << file_path << "..." << std::endl; + TF_EXPECT_OK(env->DeleteFile(file_path)); + updated_at_least_one_file = true; + } else { + EXPECT_EQ("", golden_api_entry.second) + << "To update golden API files, run " + << "tensorflow/core/api_def/update_api_def.sh."; + } + } + } + if (update_api_def && !updated_at_least_one_file) { std::cout << "Api def files are already up to date." << std::endl; } diff --git a/tensorflow/core/api_def/base_api/api_def_A.pbtxt b/tensorflow/core/api_def/base_api/api_def_A.pbtxt deleted file mode 100644 index 8193d1bc62..0000000000 --- a/tensorflow/core/api_def/base_api/api_def_A.pbtxt +++ /dev/null @@ -1,670 +0,0 @@ -op { - graph_op_name: "Abort" - endpoint { - name: "Abort" - } - summary: "Raise a exception to abort the process when called." - description: <= 2." -} -op { - graph_op_name: "AdjustContrastv2" - endpoint { - name: "AdjustContrastv2" - } - summary: "Adjust the contrast of one or more images." - description: < [2.0132, 1.056] -``` - -@compatibility(numpy) -Equivalent to np.angle. -@end_compatibility -END -} -op { - graph_op_name: "Any" - endpoint { - name: "Any" - } - summary: "Computes the \"logical or\" of elements across dimensions of a tensor." - description: < l1 else 0.0 -accum = accum_new -END -} -op { - graph_op_name: "ApplyFtrlV2" - endpoint { - name: "ApplyFtrlV2" - } - summary: "Update \'*var\' according to the Ftrl-proximal scheme." - description: < l1 else 0.0 -accum = accum_new -END -} -op { - graph_op_name: "ApplyGradientDescent" - endpoint { - name: "ApplyGradientDescent" - } - summary: "Update \'*var\' by subtracting \'alpha\' * \'delta\' from it." -} -op { - graph_op_name: "ApplyMomentum" - endpoint { - name: "ApplyMomentum" - } - summary: "Update \'*var\' according to the momentum scheme. Set use_nesterov = True if you" - description: <= 2." +} diff --git a/tensorflow/core/api_def/base_api/api_def_AdjustContrastv2.pbtxt b/tensorflow/core/api_def/base_api/api_def_AdjustContrastv2.pbtxt new file mode 100644 index 0000000000..429a5e4434 --- /dev/null +++ b/tensorflow/core/api_def/base_api/api_def_AdjustContrastv2.pbtxt @@ -0,0 +1,36 @@ +op { + graph_op_name: "AdjustContrastv2" + endpoint { + name: "AdjustContrast" + } + in_arg { + name: "images" + description: < [2.0132, 1.056] +``` + +@compatibility(numpy) +Equivalent to np.angle. +@end_compatibility +END +} diff --git a/tensorflow/core/api_def/base_api/api_def_Any.pbtxt b/tensorflow/core/api_def/base_api/api_def_Any.pbtxt new file mode 100644 index 0000000000..09fd4e0b60 --- /dev/null +++ b/tensorflow/core/api_def/base_api/api_def_Any.pbtxt @@ -0,0 +1,42 @@ +op { + graph_op_name: "Any" + endpoint { + name: "Any" + } + endpoint { + name: "ReduceAny" + } + in_arg { + name: "input" + description: < l1 else 0.0 +accum = accum_new +END +} diff --git a/tensorflow/core/api_def/base_api/api_def_ApplyFtrlV2.pbtxt b/tensorflow/core/api_def/base_api/api_def_ApplyFtrlV2.pbtxt new file mode 100644 index 0000000000..974f3adc19 --- /dev/null +++ b/tensorflow/core/api_def/base_api/api_def_ApplyFtrlV2.pbtxt @@ -0,0 +1,75 @@ +op { + graph_op_name: "ApplyFtrlV2" + in_arg { + name: "var" + description: < l1 else 0.0 +accum = accum_new +END +} diff --git a/tensorflow/core/api_def/base_api/api_def_ApplyGradientDescent.pbtxt b/tensorflow/core/api_def/base_api/api_def_ApplyGradientDescent.pbtxt new file mode 100644 index 0000000000..2f38ebd1b8 --- /dev/null +++ b/tensorflow/core/api_def/base_api/api_def_ApplyGradientDescent.pbtxt @@ -0,0 +1,35 @@ +op { + graph_op_name: "ApplyGradientDescent" + in_arg { + name: "var" + description: < -1. +END + } + attr { + name: "scientific" + description: < -1. +END + } + attr { + name: "fill" + description: < -1. If empty, pads with spaces. +Another typical value is '0'. String cannot be longer than 1 character. +END + } + summary: "Converts each entry in the given tensor to strings. Supports many numeric" + description: <