[mlir][bufferization] Remove allow-return-allocs and create-deallocs pass options, remove bufferization.escape attribute

This is the first commit in a series with the goal to rework the BufferDeallocation pass. Currently, this pass heavily relies on copies to perform correct deallocations, which leads to very slow code and potentially high memory usage. Additionally, there are unsupported cases such as returning memrefs which this series of commits aims to add support for as well. This first commit removes the deallocation capabilities of one-shot-bufferization.One-shot-bufferization should never deallocate any memrefs as this should be entirely handled by the buffer-deallocation pass going forward. This means the allow-return-allocs pass option will default to true now, create-deallocs defaults to false and they, as well as the escape attribute indicating whether a memref escapes the current region, will be removed. The documentation should w.r.t. these pass option changes should also be updated in this commit. Already reviewed in https://reviews.llvm.org/D156662

Author: maerhart

mlir/docs/Bufferization.md

@@ -266,42 +266,16 @@ must be inserted due to a RaW conflict. E.g.:
 In the above example, a buffer copy of buffer(`%another_tensor`) (with `%cst`
 inserted) is yielded from the "then" branch.
 
-In both examples, a buffer is allocated inside of a block and then yielded from
-the block. Deallocation of such buffers is tricky and not currently implemented
-in an efficient way. For this reason, One-Shot Bufferize must be explicitly
-configured with `allow-return-allocs` to support such IR.
-
-When running with `allow-return-allocs`, One-Shot Bufferize may introduce
-allocations that cannot be deallocated by One-Shot Bufferize yet. For that
-reason, `-buffer-deallocation` must be run after One-Shot Bufferize. This buffer
-deallocation pass resolves yields of newly allocated buffers with copies. E.g.,
-the `scf.if` example above would bufferize to IR similar to the following:
-
-```mlir
-%0 = scf.if %c -> (memref<?xf32>) {
- %1 = memref.alloc(...) : memref<?xf32>
- ...
- scf.yield %1 : memref<?xf32>
-} else {
- %2 = memref.alloc(...) : memref<?xf32>
- memref.copy %another_memref, %2
- scf.yield %2 : memref<?xf32>
-}
-```
-
-In the bufferized IR, both branches return a newly allocated buffer, so it does
-not matter which if-branch was taken. In both cases, the resulting buffer `%0`
-must be deallocated at some point after the `scf.if` (unless the `%0` is
-returned/yielded from its block).
-
-Note: Buffer allocations that are returned from a function are not deallocated,
-not even with `-buffer-deallocation`. It is the caller's responsibility to
-deallocate the buffer. In the future, this could be automated with allocation
-hoisting (across function boundaries) or reference counting.
-
-One-Shot Bufferize can be configured to leak all memory and not generate any
-buffer deallocations with `create-deallocs=0`. This can be useful for
-compatibility with legacy code that has its own method of deallocating buffers.
+Note: Buffer allocations that are returned from a function are not deallocated.
+It is the caller's responsibility to deallocate the buffer. For the full
+function boundary ABI for MemRefs w.r.t. buffer deallocation refer to the
+[*Function Boundary ABI*](#function-boundary-abi) section. In the future, this
+could be automated with allocation hoisting (across function boundaries) or
+reference counting.
+
+One-Shot Bufferize leaks all memory and does not generate any buffer
+deallocations. The `-buffer-deallocation-pipeline` has to be run afterwards to
+insert the deallocation operations.
 
 ## Ownership-based Buffer Deallocation
 

mlir/include/mlir/Dialect/Bufferization/IR/BufferizableOpInterface.h

@@ -361,10 +361,6 @@ struct BufferizationOptions {
  /// used.
  UnknownTypeConverterFn unknownTypeConverterFn = nullptr;
 
- /// Specifies whether dealloc ops should be generated along with alloc ops. If
- /// not, new memory allocations will leak.
- bool createDeallocs = true;
-
  /// Seed for the analysis fuzzer. If set to `0`, the fuzzer is deactivated.
  /// Should be used only with `testAnalysisOnly = true`.
  unsigned analysisFuzzerSeed = 0;
@@ -588,13 +584,9 @@ class AnalysisState {
 /// undefined contents is allocated.
 FailureOr<Value>
 allocateTensorForShapedValue(OpBuilder &b, Location loc, Value shapedValue,
- bool escape, const BufferizationOptions &options,
+ const BufferizationOptions &options,
  bool copy = true);
 
-/// Return `true` if the allocation of the given op is guaranteed to not escape
-/// the containing block.
-bool allocationDoesNotEscape(OpResult opResult);
-
 /// Lookup the buffer for the given value. If the value was not bufferized
 /// yet, wrap it in a ToMemrefOp. Otherwise, it is the result of a ToTensorOp,
 /// from which the memref operand is returned.
@@ -641,12 +633,6 @@ OpTy replaceOpWithNewBufferizedOp(RewriterBase &rewriter, Operation *op,
  return newOp;
 }
 
-/// Return `true` if the buffer of given OpResult should be deallocated. This
-/// function should be called during `BufferizableOpInterface::bufferize`
-/// implementations that allocate a new buffer for the given OpResult.
-bool shouldDeallocateOpResult(OpResult opResult,
- const BufferizationOptions &options);
-
 /// Return a MemRefType to which the type of the given value can be bufferized.
 ///
 /// If possible, op bufferization implementations should not use this function

mlir/include/mlir/Dialect/Bufferization/IR/BufferizationBase.td

@@ -60,16 +60,6 @@ def Bufferization_Dialect : Dialect {
  /// arguments during One-Shot Module Bufferize.
  constexpr const static ::llvm::StringLiteral
  kBufferLayoutAttrName = "bufferization.buffer_layout";
-
- /// Attribute name used to mark escaping behavior of buffer allocations.
- /// Escaping allocations cannot be deallocated in the same block and must
- /// be treated specially: They are currently deallocated with the
- /// BufferDeallocation pass.
- ///
- /// Note: Only ops with at least one OpResult that bufferizes to a buffer
- /// allocation (as per BufferizableOpInterface) may have this attribute.
- constexpr const static ::llvm::StringLiteral
- kEscapeAttrName = "bufferization.escape";
  }];
  let hasOperationAttrVerify = 1;
 }

mlir/include/mlir/Dialect/Bufferization/TransformOps/BufferizationTransformOps.td

@@ -82,10 +82,9 @@ def OneShotBufferizeOp
  let arguments = (
  ins TransformHandleTypeInterface:$target,
  OptionalAttr<LayoutMapOption>:$function_boundary_type_conversion,
- DefaultValuedAttr<BoolAttr, "false">:$allow_return_allocs,
+ DefaultValuedAttr<BoolAttr, "false">:$allow_return_allocs_from_loops,
  DefaultValuedAttr<BoolAttr, "false">:$allow_unknown_ops,
  DefaultValuedAttr<BoolAttr, "false">:$bufferize_function_boundaries,
- DefaultValuedAttr<BoolAttr, "true">:$create_deallocs,
  DefaultValuedAttr<BoolAttr, "false">:$test_analysis_only,
  DefaultValuedAttr<BoolAttr, "false">:$print_conflicts,
  DefaultValuedAttr<StrAttr, "\"memref.copy\"">:$memcpy_op);

mlir/include/mlir/Dialect/Bufferization/Transforms/OneShotAnalysis.h

@@ -28,9 +28,9 @@ struct OneShotBufferizationOptions : public BufferizationOptions {
 
  OneShotBufferizationOptions() = default;
 
- /// Specifies whether returning newly allocated memrefs should be allowed.
- /// Otherwise, a pass failure is triggered.
- bool allowReturnAllocs = false;
+ /// Specifies whether returning newly allocated memrefs from loops should be
+ /// allowed. Otherwise, a pass failure is triggered.
+ bool allowReturnAllocsFromLoops = false;
 
  /// Specifies whether the tensor IR should be annotated with alias sets.
  bool dumpAliasSets = false;

mlir/include/mlir/Dialect/Bufferization/Transforms/Passes.td

@@ -387,15 +387,9 @@ def OneShotBufferize : Pass<"one-shot-bufferize", "ModuleOp"> {
  example, `tensor.generate` is not in destination-passing style and always
  results in a new buffer allocation.
 
- One-Shot Bufferize deallocates all buffers that it allocates. Yielding newly
- allocated buffers from a block can lead to bad performance because
- additional buffer copies are often needed to make sure that every buffer
- allocation is also deallocated again. By default, such IR is rejected by
- One-Shot Bufferize. Such IR can be allowed with `allow-return-allocs`. In
- that case, the `-buffer-deallocation` pass should be run after One-Shot
- Bufferize. Note that new buffer allocations that are returned from a
- function can currently not be deallocated by `-buffer-deallocation` and
- leak.
+ One-Shot Bufferize does not deallocate any buffers that it allocates. The
+ `-buffer-deallocation` pass should be run after One-Shot Bufferize to insert
+ the deallocation operations necessary to eliminate memory leaks.
 
  One-Shot Bufferize will by default reject IR that contains non-bufferizable
  op, i.e., ops that do not implemement BufferizableOpInterface. Such IR can
@@ -462,9 +456,9 @@ def OneShotBufferize : Pass<"one-shot-bufferize", "ModuleOp"> {
  `test-analysis-only`.
  }];
  let options = [
- Option<"allowReturnAllocs", "allow-return-allocs", "bool",
- /*default=*/"false",
- "Allows returning/yielding new allocations from a block.">,
+ Option<"allowReturnAllocsFromLoops", "allow-return-allocs-from-loops",
+ "bool", /*default=*/"false",
+ "Allows returning/yielding new allocations from a loop.">,
  Option<"allowUnknownOps", "allow-unknown-ops", "bool",
  /*default=*/"false",
  "Allows unknown (not bufferizable) ops in the input IR.">,
@@ -479,9 +473,6 @@ def OneShotBufferize : Pass<"one-shot-bufferize", "ModuleOp"> {
  "Bufferize function boundaries (experimental).">,
  Option<"copyBeforeWrite", "copy-before-write", "bool", /*default=*/"false",
  "Skip the analysis. Make a buffer copy on every write.">,
- Option<"createDeallocs", "create-deallocs", "bool", /*default=*/"true",
- "Specify if buffers should be deallocated. For compatibility with "
- "core bufferization passes.">,
  ListOption<"dialectFilter", "dialect-filter", "std::string",
  "Restrict bufferization to ops from these dialects.">,
  Option<"dumpAliasSets", "dump-alias-sets", "bool", /*default=*/"false",
@@ -513,8 +504,6 @@ def OneShotBufferize : Pass<"one-shot-bufferize", "ModuleOp"> {
  let statistics = [
  Statistic<"numBufferAlloc", "num-buffer-alloc",
  "Number of buffer allocations">,
- Statistic<"numBufferDealloc", "num-buffer-dealloc",
- "Number of buffer deallocations">,
  Statistic<"numTensorInPlace", "num-tensor-in-place",
  "Number of in-place tensor OpOperands">,
  Statistic<"numTensorOutOfPlace", "num-tensor-out-of-place",

mlir/include/mlir/Dialect/SparseTensor/Pipelines/Passes.h

@@ -93,8 +93,7 @@ struct SparseCompilerOptions
  desc("Specify if the temporary buffers created by the sparse "
  "compiler should be deallocated. For compatibility with core "
  "bufferization passes. "
- "This option is only used when enable-runtime-library=false. "
- "See also create-deallocs for BufferizationOption."),
+ "This option is only used when enable-runtime-library=false."),
  init(true)};
 
  PassOptions::Option<int32_t> vectorLength{

mlir/lib/Dialect/Bufferization/IR/BufferizableOpInterface.cpp

@@ -140,27 +140,11 @@ Operation *bufferization::getOwnerOfValue(Value value) {
  return llvm::cast<BlockArgument>(value).getOwner()->getParentOp();
 }
 
-bool bufferization::allocationDoesNotEscape(OpResult opResult) {
-#ifndef NDEBUG
- auto bufferizableOp = opResult.getDefiningOp<BufferizableOpInterface>();
- assert(bufferizableOp && bufferizableOp.bufferizesToAllocation(opResult) &&
- "expected op that bufferizes to an allocation");
-#endif // NDEBUG
-
- Operation *op = opResult.getDefiningOp();
- // If there is no 'escape' attribute, we cannot say for sure.
- if (!op->hasAttr(BufferizationDialect::kEscapeAttrName))
- return false;
- auto attr =
- op->getAttrOfType<ArrayAttr>(BufferizationDialect::kEscapeAttrName);
- return !llvm::cast<BoolAttr>(attr[opResult.getResultNumber()]).getValue();
-}
-
 /// Create an AllocTensorOp for the given shaped value. If `copy` is set, the
 /// shaped value is copied. Otherwise, a tensor with undefined contents is
 /// allocated.
 FailureOr<Value> bufferization::allocateTensorForShapedValue(
- OpBuilder &b, Location loc, Value shapedValue, bool escape,
+ OpBuilder &b, Location loc, Value shapedValue,
  const BufferizationOptions &options, bool copy) {
  Value tensor;
  if (llvm::isa<RankedTensorType>(shapedValue.getType())) {
@@ -202,8 +186,6 @@ FailureOr<Value> bufferization::allocateTensorForShapedValue(
  // Create AllocTensorOp.
  auto allocTensorOp = b.create<AllocTensorOp>(loc, tensorType, dynamicSizes,
  copy ? tensor : Value());
- allocTensorOp->setAttr(BufferizationDialect::kEscapeAttrName,
- b.getBoolArrayAttr({escape}));
 
  // Add 'memory_space' attribute. Not needed if 'copy' operand is specified.
  if (copy)
@@ -224,10 +206,8 @@ LogicalResult BufferizableOpInterface::resolveTensorOpOperandConflicts(
  Operation *op = getOperation();
  SmallVector<OpOperand *> outOfPlaceOpOperands;
  DenseSet<OpOperand *> copiedOpOperands;
- DenseSet<OpOperand *> escapingOpOperandCopies;
  SmallVector<Value> outOfPlaceValues;
  DenseSet<Value> copiedOpValues;
- DenseSet<Value> escapingValueCopies;
 
  // Find all out-of-place OpOperands.
  for (OpOperand &opOperand : op->getOpOperands()) {
@@ -243,11 +223,6 @@ LogicalResult BufferizableOpInterface::resolveTensorOpOperandConflicts(
  // Is the result yielded from a block? Or are deallocations turned off
  // entirely? In either case, mark the allocation as "escaping", so that it
  // will not be deallocated.
- bool escape = !state.getOptions().createDeallocs ||
- llvm::any_of(aliasingValues, [&](AliasingValue a) {
- return state.isTensorYielded(a.value);
- });
-
  if (aliasingValues.getNumAliases() == 1 &&
  isa<OpResult>(aliasingValues.getAliases()[0].value) &&
  !state.bufferizesToMemoryWrite(opOperand) &&
@@ -265,24 +240,19 @@ LogicalResult BufferizableOpInterface::resolveTensorOpOperandConflicts(
  outOfPlaceValues.push_back(value);
  if (!state.canOmitTensorCopy(opOperand))
  copiedOpValues.insert(value);
- if (escape)
- escapingValueCopies.insert(value);
  } else {
  // In all other cases, make a copy of the OpOperand.
  outOfPlaceOpOperands.push_back(&opOperand);
  if (!state.canOmitTensorCopy(opOperand))
  copiedOpOperands.insert(&opOperand);
- if (escape)
- escapingOpOperandCopies.insert(&opOperand);
  }
  }
 
  // Insert copies of OpOperands.
  rewriter.setInsertionPoint(op);
  for (OpOperand *opOperand : outOfPlaceOpOperands) {
  FailureOr<Value> copy = allocateTensorForShapedValue(
- rewriter, op->getLoc(), opOperand->get(),
- escapingOpOperandCopies.contains(opOperand), state.getOptions(),
+ rewriter, op->getLoc(), opOperand->get(), state.getOptions(),
  copiedOpOperands.contains(opOperand));
  if (failed(copy))
  return failure();
@@ -293,8 +263,8 @@ LogicalResult BufferizableOpInterface::resolveTensorOpOperandConflicts(
  rewriter.setInsertionPointAfter(op);
  for (Value value : outOfPlaceValues) {
  FailureOr<Value> copy = allocateTensorForShapedValue(
- rewriter, op->getLoc(), value, escapingValueCopies.contains(value),
- state.getOptions(), copiedOpValues.count(value));
+ rewriter, op->getLoc(), value, state.getOptions(),
+ copiedOpValues.count(value));
  if (failed(copy))
  return failure();
  SmallVector<OpOperand *> uses = llvm::to_vector(
@@ -314,29 +284,6 @@ LogicalResult BufferizableOpInterface::resolveTensorOpOperandConflicts(
  return success();
 }
 
-bool bufferization::shouldDeallocateOpResult(
- OpResult opResult, const BufferizationOptions &options) {
- Operation *op = opResult.getOwner();
- assert(options.dynCastBufferizableOp(op).bufferizesToAllocation(opResult) &&
- "expected that op allocates");
-
- AnalysisState analysisState(options);
- if (op->hasAttr(BufferizationDialect::kEscapeAttrName)) {
- // AllocTensorOp has one result.
- ArrayAttr escapeAttr = llvm::cast<ArrayAttr>(
- op->getAttr(BufferizationDialect::kEscapeAttrName));
- return !llvm::cast<BoolAttr>(escapeAttr[0]).getValue();
- }
-
- // No "escape" annotation found.
- if (options.createDeallocs) {
- // Perform an ad-hoc analysis.
- return !analysisState.isTensorYielded(opResult);
- }
-
- return false;
-}
-
 //===----------------------------------------------------------------------===//
 // OpFilter
 //===----------------------------------------------------------------------===//

mlir/lib/Dialect/Bufferization/IR/BufferizationDialect.cpp

@@ -28,9 +28,6 @@ constexpr const ::llvm::StringLiteral BufferizationDialect::kWritableAttrName;
 constexpr const ::llvm::StringLiteral
  BufferizationDialect::kBufferLayoutAttrName;
 
-/// Attribute name used to mark escaping behavior of buffer allocations.
-constexpr const ::llvm::StringLiteral BufferizationDialect::kEscapeAttrName;
-
 //===----------------------------------------------------------------------===//
 // Bufferization Dialect Interfaces
 //===----------------------------------------------------------------------===//
@@ -108,38 +105,6 @@ BufferizationDialect::verifyOperationAttribute(Operation *op,
  NamedAttribute attr) {
  using bufferization::BufferizableOpInterface;
 
- if (attr.getName() == kEscapeAttrName) {
- auto arrayAttr = llvm::dyn_cast<ArrayAttr>(attr.getValue());
- if (!arrayAttr)
- return op->emitError() << "'" << kEscapeAttrName
- << "' is expected to be a bool array attribute";
- if (arrayAttr.size() != op->getNumResults())
- return op->emitError()
- << "'" << kEscapeAttrName
- << "' has wrong number of elements, expected "
- << op->getNumResults() << ", got " << arrayAttr.size();
- auto bufferizableOp = dyn_cast<BufferizableOpInterface>(op);
- if (!bufferizableOp)
- return op->emitError()
- << "'" << kEscapeAttrName << "' only valid on bufferizable ops";
- for (const auto &it : llvm::enumerate(arrayAttr)) {
- auto attr = it.value();
- auto boolAttr = llvm::dyn_cast<BoolAttr>(attr);
- if (!boolAttr)
- return op->emitError() << "'" << kEscapeAttrName
- << "' is expected to be a bool array attribute";
- if (!boolAttr.getValue())
- continue;
- if (!llvm::isa<TensorType>(op->getResult(it.index()).getType()))
- return op->emitError()
- << "'" << kEscapeAttrName << "' only valid for tensor results";
- if (!bufferizableOp.bufferizesToAllocation(op->getOpResult(it.index())))
- return op->emitError() << "'" << kEscapeAttrName
- << "' only valid for allocation results";
- }
- return success();
- }
-
  return op->emitError()
  << "attribute '" << attr.getName()
  << "' not supported as an op attribute by the bufferization dialect";

mlir/lib/Dialect/Bufferization/IR/BufferizationOps.cpp

@@ -187,20 +187,9 @@ LogicalResult AllocTensorOp::bufferize(RewriterBase &rewriter,
  return failure();
  }
 
- // Should the buffer be deallocated?
- bool dealloc =
- shouldDeallocateOpResult(llvm::cast<OpResult>(getResult()), options);
-
  // Replace op.
  replaceOpWithBufferizedValues(rewriter, getOperation(), *alloc);
 
- // Create buffer deallocation (if requested).
- if (!dealloc)
- return success();
-
- rewriter.setInsertionPoint(rewriter.getInsertionBlock()->getTerminator());
- if (failed(options.createDealloc(rewriter, loc, *alloc)))
- return failure();
  return success();
 }