-12

The "Close Votes" review queue has a substantial backlog. And, it's reasonable to say a queue consistently that large is unmanageable.

As per this test performed on the review queue, playing the numbers game didn't work. By increasing the review limit to 60, it only resulted in an 11% decrease in the "Close Votes" queue and the "Low Quality" queue had a 74% reduction.

Since there is a 1k reputation difference between the "Close Votes" and the "Low Quality" queue, the number of people capable of reviewing may be a huge contributing factor to why the reduction varied so much between the two during the test mentioned above.

That's why I think that reducing the required reputation to review items in the higher-tier queues can be a practical solution to this problem.

Queues for 3k rep can be dropped to 2 or 2.5k

Queues for 2k rep can be dropped to 1 or 1.5k

An alternative to this: Allow lower rep users to participate in higher-tier queues if they meet a specific pass/fail ratio for all of their audits and have passed a specific number of audits.

Of course, this could also be handled by the number of reviews a user made that were helpful.

3
  • 6
    Correctly reviewing the "Close Votes" queue requires that you be able to cast close votes, which is a 3k reputation privilege, so in order to make this work, we'd have to lower that reputation privilege threshold. Although...maybe we could have a "Recommend Closure" flag, like for the "Low Quality Posts" review queue does for users that have not yet reached the "trusted users" privilege. Commented Feb 27, 2019 at 0:19
  • I like the premise of this idea. It's an interesting concept. Do we lower the threshold to cast close votes. For the other review queues I'd not worry, as they are under control (or were last time I looked). Commented Feb 27, 2019 at 7:58
  • I like the alternative suggestion Jake offered. Commented Feb 27, 2019 at 12:32

2 Answers 2

Reset to default
5

I don't think we should have this. One of the main reasons is that the reason for all the backlog in different queues isn't that there aren't enough users to review them. It's that there aren't enough users interested in reviewing them.

Queues for 2k rep can be dropped to 1 or 1.5k

As of right now, Low Quality Posts has 82 items that need review and Suggested Edits has 93. That's not that much of a backlog, so reducing the reputation requirements is not necessary.

Queues for 3k rep can be dropped to 2 or 2.5k

I disagree with this because:

  • That would require moving the "Cast close & reopen votes" privilege down to the same level of either "Edit questions and answers" or "Create tag synonyms".
  • While there is a significant backlog in Close Votes, it matters more that items are reviewed correctly than just reviewed and cleared from the queue. Bringing down the reputation requirement would mean people would need less experience to review Close and Reopen votes.

An alternative to this: Allow lower rep users to participate in higher-tier queues if they meet a specific pass/fail ratio for all of their audits and have passed a specific number of audits.

Review audits are simply there to make sure the reviewer is paying attention. They are incredibly easy to identify. Users could easily game this by reviewing only audits and getting to a 100% pass rate. Additionally, while it's great that users are paying attention, that doesn't mean they have enough experience for the higher queues. It just means that they are not robo-reviewers.

7
  • You make some great points on this, thanks for adding your thoughts. Although, the alternative could, like I said, be determined by the number of audits passed. I'm not sure why someone would go through and "game" like 250 audits and skip every other item just to be able to access a higher review queue. I just think we should incorporate helpful flags, reviews, etc. into a process that can get people who do not necessarily use SO for Q&A to be able to moderate the site in some way if that's what they're here for primarily. Commented Mar 6, 2019 at 17:31
  • 1
    The point about lack of interested users is technically true, but irrelevant: since the proportion of interested users with 3k+ is broadly similar to the proportion with 1-3k, reducing the threshold to increase the population size will bring in interested users at a similar rate. Also, this doesn't explain why it would be a problem to change the close vote privilege. Commented Mar 6, 2019 at 18:38
  • 2
    @Jake: I absolutely agree that it would be enormously helpful to incorporate evidence of good moderation into moderation privilege acquisition. More reviews would be completed faster and more accurately and by a better representation of dedicated users. But I don't think it's anywhere near as easy as you're suggesting, unfortunately. In fact, I've put a good few hours of thought into a system that should be able to handle this, but as it would require such substantial changes to SE software, have not yet posted it. Commented Mar 6, 2019 at 18:42
  • @NathanTuggy You're going to see much lower percentages of users interested in moderation at lower reputation levels. Although by far the more important point is as the second half, that those users aren't going to review as well, not that there are less of them. That's absolutely an explanation of why we shouldn't lower the privilege. Commented Mar 6, 2019 at 18:56
  • 2
    @Jake You appear to underestimate people's motivations for abusing reviews. Lots and lots of people have performed thousands of reviews each without actually reviewing the content (just mashing any button to complete the review) just to get a badge. Audits help with people like that, by they're still easy to defeat. Just assuming people won't abuse the system files in the face of overwhelming observed evidence of how people have reviewed over the course of the entire history of the review system. Commented Mar 6, 2019 at 18:58
  • @Servy I see what you mean, and it's unfortunate that people try to get badges like that. I wonder how realistic it would be to suggest introducing a "helpful ratio" parameter to receiving moderation badges. This would probably combat this but is a different post entirely. Commented Mar 6, 2019 at 19:56
  • @Jake I agree, it really is unfortunate how many people game the system. As you said, introducing a helpful ratio for badges is a completely different discussion. Feel free to post a new feature request if you wish. Commented Mar 6, 2019 at 19:58
4

You didn't allow for the humongous difference in the sizes of the queues when analyzing results:

  • The Close Votes had an 11% reduction in queue size, dropping from 9,500 per day to 8,400 per day
  • The Low Quality queue had a 74% reduction in queue size, dropping from 750 per day to 192 per day

The post doesn't report the number of reviews done and the number of completed reviews which would be needed to really see the impact of the changes. However, it can be seen that

  • in absolute numbers, the larger amount of eligible users in the Low Quality queue processed ~550 additional items per day while the smaller number of Close reviewers processed ~1'100 . The daily review limits were the same for both queues, so this cannot be written off to users capping.

So, your reasoning that the impact in the Low Quality queue was much higher doesn't stand.

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.