Improve performance issue for DELETE command

[zzet]

Before

 QUERY PLAN ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Delete on log_hierarchies (cost=4403.03..6923802.33 rows=169553156 width=6) (actual time=52728.567..52728.567 rows=0 loops=1) -> Seq Scan on log_hierarchies (cost=4403.03..6923802.33 rows=169553156 width=6) (actual time=52728.564..52728.564 rows=0 loops=1) Filter: ((hashed SubPlan 1) OR (descendant_id = 1022566)) Rows Removed by Filter: 338996939 SubPlan 1 -> Unique (cost=0.57..4403.03 rows=1 width=4) (actual time=0.085..0.085 rows=0 loops=1) -> Index Only Scan using log_anc_desc_idx on log_hierarchies log_hierarchies_1 (cost=0.57..4396.98 rows=2420 width=4) (actual time=0.085..0.085 rows=0 loops=1) Index Cond: (ancestor_id = 1022566) Heap Fetches: 0 Planning time: 0.223 ms Execution time: 52728.657 ms (11 rows) 

After

 QUERY PLAN ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Delete on log_hierarchies (cost=1.15..5006.48 rows=11962 width=34) (actual time=11.178..11.178 rows=0 loops=1) -> Nested Loop (cost=1.15..5006.48 rows=11962 width=34) (actual time=0.126..10.466 rows=561 loops=1) -> Subquery Scan on "ANY_subquery" (cost=0.57..4403.04 rows=1 width=32) (actual time=0.077..0.172 rows=33 loops=1) -> Unique (cost=0.57..4403.03 rows=1 width=4) (actual time=0.050..0.111 rows=33 loops=1) -> Index Only Scan using log_anc_desc_idx on log_hierarchies log_hierarchies_1 (cost=0.57..4396.98 rows=2420 width=4) (actual time=0.048..0.080 rows=33 loops=1) Index Cond: (ancestor_id = 1022533) Heap Fetches: 33 -> Index Scan using log_desc_idx on log_hierarchies (cost=0.57..483.82 rows=11962 width=10) (actual time=0.024..0.306 rows=17 loops=33) Index Cond: (descendant_id = "ANY_subquery".descendant_id) Planning time: 0.290 ms Execution time: 11.246 ms (11 rows) 

This commit fix issue with OR in condition. If we have OR we can't use indexes -> fullscan.

We can run 2 fast queries, because we run them in transaction.

[zzet]

similar PR: #96

[seuros]

Awesome work @zzet .

[seuros]

I wonder why travis didn't pickup the PR.

ping @mceachen

[zzet]

@seuros I updated commit:

explain analyze SELECT DISTINCT descendant_id FROM (SELECT descendant_id FROM "log_hierarchies" WHERE ancestor_id = 1022510 or descendant_id = 1022510 ) AS x; QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------ HashAggregate (cost=53382.86..53382.88 rows=2 width=4) (actual time=1.008..1.011 rows=22 loops=1) Group Key: log_hierarchies.descendant_id -> Bitmap Heap Scan on log_hierarchies (cost=354.58..53347.42 rows=14179 width=4) (actual time=0.218..0.661 rows=1708 loops=1) Recheck Cond: ((ancestor_id = 1022510) OR (descendant_id = 1022510)) Heap Blocks: exact=65 -> BitmapOr (cost=354.58..354.58 rows=14179 width=0) (actual time=0.202..0.202 rows=0 loops=1) -> Bitmap Index Scan on log_anc_desc_idx (cost=0.00..86.72 rows=2420 width=0) (actual time=0.045..0.045 rows=56 loops=1) Index Cond: (ancestor_id = 1022510) -> Bitmap Index Scan on log_desc_idx (cost=0.00..260.76 rows=11759 width=0) (actual time=0.152..0.152 rows=1687 loops=1) Index Cond: (descendant_id = 1022510) Planning time: 0.181 ms Execution time: 1.067 ms (12 rows) 

At now - one query.

[zzet]

@mceachen some problems with travis? https://travis-ci.org/mceachen/closure_tree/jobs/88097080#L181

[mceachen]

Thanks so much for the PR! Love the detail, especially the benchmark.
@martin-schmidt fixed the build which I just merged with master, and I'll cut 6.0.0 as soon as this PR builds green on master.