From 24f88b319c88bfe55e8b2b683193a85842bdad88 Mon Sep 17 00:00:00 2001
From: yorksity <yorksity@outlook.com>
Date: Tue, 10 Oct 2023 14:36:23 +0800
Subject: [PATCH] [SPARK-45205][SQL] CommandResultExec to override iterator
 methods to avoid triggering multiple jobs
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

### What changes were proposed in this pull request?

After SPARK-35378 was changed, the execution of statements such as ‘show parititions test' became slower. The change point is that the execution process changes from ExecutedCommandEnec to CommandResultExec, but ExecutedCommandExec originally implemented the following method

override def executeToIterator(): Iterator[InternalRow] = sideEffectResult.iterator

CommandResultExec is not rewritten, so when the hasNext method is executed, a job process is created, resulting in increased time-consuming

### Why are the changes needed?

Improve performance when show partitions/tables.

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

Existing tests should cover this.

### Was this patch authored or co-authored using generative AI tooling?

No

Closes #43270 from yorksity/SPARK-45205.

Authored-by: yorksity <yorksity@outlook.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
(cherry picked from commit c9c99222e828d556552694dfb48c75bf0703a2c4)
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
---
 .../org/apache/spark/sql/execution/CommandResultExec.scala      | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/CommandResultExec.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/CommandResultExec.scala
index 5f38278d2dc67..45e3e41ab053d 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/CommandResultExec.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/CommandResultExec.scala
@@ -81,6 +81,8 @@ case class CommandResultExec(
     unsafeRows
   }
 
+  override def executeToIterator(): Iterator[InternalRow] = unsafeRows.iterator
+
   override def executeTake(limit: Int): Array[InternalRow] = {
     val taken = unsafeRows.take(limit)
     longMetric("numOutputRows").add(taken.size)