There is a quite a lot to break down there. It should probably be several different meta questions.

I like the suggested history and survey tags.

Should we be more aggressive with suggested edits to questions (and answers)? The suggestion part being key. It is most unpleasant to be edited wrongly.

That does mean a lot of moderation work however.

Dealing with highly voted wrong answers is hard. I don't think the stack overflow system allows for that very well.

Perhpas you can add a comment or a moderator sledgehammer edit to add a title "this answer is popular but considered wrong by experts in this community - see XYZ"

The real concern is your point:

"This is a real problem in the history of online programming languages communities: so many programmers have such strong (yet ultimately ill-informed) opinions that it becomes too tiresome to argue, and experts retreat to more closed spaces that they can more effectively moderate."

I feel like https://ai.stackexchange.com is at risk of going that way. We may be saved by being less popular and more niche despite "writing a programming language" being a thing lots of less experienced developers do as part of their journey.

Another way to combat that is with canonical question and answers maybe. But again this places a burden on experts to write them in the first place rather than say writing a blog post or an academic paper.

If we were to just harvest and curate such efforts you could argue we might as well be a search engine or an LLM rather than an effective SE community.

It would be good to establish a norm of more focused questions like this. That would also result in more questions as the overly-broad ones were able to air their constituent facets individually.

It would also be helpful for giving a stronger impression of seriousness to visitors. I know that there's been some doubt expressed about whether this is a real issue, and whether it really will put people off, but—as Alexis said here, and in other conversations—it is, and it will (and it does and it has). At the least, both of us have seen it.

That doesn't mean that the range of subjects the site covers is wrong, or that people are doing it wrong, but that by consciously tweaking the structure of questions and thus the sorts of answers they lead to we can set people up for success. The very broad questions don't do that well, and they end up with lots of very different answers crammed into them, user confusion, and still giving a confounding view of the site.

There is also a significant strain in the PL community that feels that "programming languages" means "type theory", and maybe stretches to compiler optimisation, and that's it—I don't want this site to be that. There is a continuum that spans theory and applied PL, another that spans academic, industrial, and hobbyist work, and another spanning design, implementation, and environs. There's scope for the Stack Exchange model to create a constructive mixed community that has been hard to find elsewhere, and to document knowledge that exists, but the site needs to attract and retain the full range of users for that to work.

I want to be able to make use of the expertise we have here, which is pretty wide-reaching precisely because there are many users outside of the "PL as type theory" milieu — for example, we have expertise here in 2D languages that just doesn't exist at all in academic literature, while concatenative languages are another area that has very little work of that sort. Esoteric languages are often pure explorations of a particular point in the design space that can be useful to inform wider language-design work. Similarly, we can all learn from people with domain expertise in other parts of the field, and it's great to draw more of them in. This blending is the real promise of the site, and picking a more constructive scope for each question is going to help that.

I have a few points to address specifically. These are somewhat recapping discussions from the chat.

A history tag

This would be great, and gives a useful focus to what I think can be useful questions about how and why things ended up as they did. Answers should be required to address the concrete history being asked about, and ideally have some backing references — though sometimes we'll be lucky enough to have primary-source answers, which are great too.

A survey tag

Survey questions have been controversial and suggested to be unsuited for the Stack Exchange model. I think it's plausible that there is a constructive scope for them on this site, for providing concrete instances of design or implementation patterns. To be useful, the question would need to circumscribe the PL technique being asked about and answers would identify previous instances.

This would be a meta-tag that doesn't describe what the question is "about", so it starts out with points against it, but several sites on the network make healthy use of tags like this to specify rules for the kinds of answer that are expected. We don't necessarily need this, but I don't think we should dismiss the idea either. The associated rule would be something like

Answers to this question must identify existing notable programming languages and how they incorporate the subject of the question. Speculative concepts and theories will be deleted.

It's possible that this isn't the best name for the tag, even though it's the most accurate one, because people don't know and won't learn what a survey is. I don't think literature-survey fits either, because it wouldn't be purely literary. The tag excerpt & wiki, and enforcement, should be able to stop it being treated as an opinion poll.

Human-computer-interaction questions

I am a big fan of taking account of human-factors elements of language design. I don't want to call these human-computer-interaction, because that's an enormous field of its own too, but questions explicitly directed at this area would be good. There are actually multiple dimensions within this: ergonomics, effectiveness, and programming interfaces all have good questions, and sometimes there can be the same topics addressed from multiple avenues. The static-or-dynamic typing question is one example of that: there are studies like the Hanenberg one you cited about how effective and efficient users are with one or the other, but also those focusing on the programmer experience, or on tooling that is enabled by type annotations. Questions about all of those things should be encouraged.

While answers should provide evidence rather than reckons, there are many forms that evidence can take, and we should be open to all of them. It's also the case that in many real-world situations the evidence is contradictory or contingent on other circumstances. We should expect better than guesswork, though.

Implementation questions masquerading as design

There are definitely plenty of questions that ask both, which is probably too broad. Often there are both design and implementation elements that are worth asking about: the static typing question again has meaningful parts on both sides. This part I'm probably the least convinced about—I think there are probably some cases, but more of these questions seem to fit into other buckets or can have their scope reigned in without changing the nature of the question.

I'm not keen to force everything into an implementation-shaped box.

Restructuring existing questions

I am uncertain about changing existing questions into these forms. Their current answers mostly won't address the new form of the question — some will, particularly the more comprehensive answers that the goal is preserving — but the thing about these broad questions is that they get lots of answers from different directions. In many cases, there are a couple of comprehensive answers that we definitely want to preserve (and so make the question match), and a lot of others that would no longer be answers to it. When there are two good directions to take the question, this is intractable. Perhaps we would delete them, perhaps they're fine just left there, or perhaps we'd want to historical-lock the old questions and let the (likely multiple) more focused versions emerge separately.

"What are the pros and cons of ..."

This form has been very popular, but is just not a good structure for a question in general. It's inviting lists of ideas labelled "pro" or "con", but that is not how language design works, is a false dichotomy, and doesn't admit the sorts of analysis that actually matter. We've all seen the evident LLM-generated answers to these passing by, but they're not so different.

Most of the time they are really multiple questions in one, that would better be split out as above into different versions for the different dimensions of the question: the history question, the survey question, the human-factors question, the implementation question, etc. Some however might be tightly-scoped enough that dividing them further doesn't help, but the structure of the question just doesn't set the answerer up to succeed.

I don't expect that there will be many questions left after taking a more principled take on them, but if there are some then at the least a better framing is "What are the trade-offs in ...": it explicitly seeks analysis and doesn't force answers into an artificial contrast that may not really exist. It also doesn't push towards the answers that currently end up listing something like "Readability" as both a pro and a con.

However, in most cases I think refocusing the question to what it's really asking about is going to lead to better questions, and more questions, and will also ensure that the asker understands what they're asking about. This shouldn't take on the same memetic form of pick-a-term-and-ask-for-its-tradeoffs, but some real questions about specific concrete problems may genuinely come down to this sort of informed analysis.

Substantiating answers

As well as questions that are too broad, another issue you identified is that there isn't a culture of expecting answers to provide evidence or rationale, and so answers are upvoted without any reason to believe they're true. Some of this is probably due to the nature of questions, where the breadth of them means there's just too much to talk about, and so answers stray outside what they've got support for; this part probably tidies itself up when the questions are more focused.

Others don't fit in this camp, though, and there are even answers that explicitly disclaim any knowledge or backing. We don't need every point to have a citation attached, but we do need something beyond guesses, wishes, and reckons, and we should ask that of our answers and our answerers. In the first instance that is voting from all of us, and careful thought given to our own answers we post.

There are other tools like NAA flags, post notices, and deletion policies, but before resorting to those we can collectively improve things ourselves. Often this will just mean doing a little research before submitting the answer and won't be a big disruption at all.

Personal experience on matters of readily-observable fact, like what features a language provides, or even primary-source comment on implementing some construct, is probably fine, while personal experience on matters of opinion generally won't be. Answers that just disclaim all relevant knowledge on factual questions shouldn't be encouraged. Questions should be posed to invite answers that have substance—again, setting the answerers up for success.

I think we will need further meta posts to sort out details on these things, and whether we really do want to change them, but at the high level getting to more focused questions would be a plus.

This discussion has been open for quite some time now, and it’s gotten some good feedback both here and in chat. This answer is intended to collect what appears to be the general consensus and to suggest some next steps.

Broad support for the history tag

People seem to generally agree that a history tag is a good idea. Michael Homer writes:

This would be great, and gives a useful focus to what I think can be useful questions about how and why things ended up as they did. Answers should be required to address the concrete history being asked about, and ideally have some backing references — though sometimes we'll be lucky enough to have primary-source answers, which are great too.

This perspective seems fairly representative. Since it seems to be unanimous, I have gone ahead and added a bullet explicitly mentioning history questions as on-topic on the relevant page of our help center.

Moderate support for survey questions, but also some concerns

People generally seem to agree that certain types of survey questions—such as literature survey questions explicitly seeking citations—are a good fit for the site. However, there are definitely some concerns as well. Michael Homer again writes:

Survey questions have been controversial and suggested to be unsuited for the Stack Exchange model. I think it's plausible that there is a constructive scope for them on this site, for providing concrete instances of design or implementation patterns. To be useful, the question would need to circumscribe the PL technique being asked about and answers would identify previous instances.

I agree that we ought to be cautious about these questions, as it is important to make very clear what requirements we expect these questions to uphold. I will commit to posting a followup question on meta to specifically discuss how we want to define those requirements and how we ought to moderate them.

No clear consensus on what to do about existing questions

On the subject of what we should do with existing questions, Michael writes:

I am uncertain about changing existing questions into these forms. Their current answers mostly won't address the new form of the question — some will, particularly the more comprehensive answers that the goal is preserving — but the thing about these broad questions is that they get lots of answers from different directions. In many cases, there are a couple of comprehensive answers that we definitely want to preserve (and so make the question match), and a lot of others that would no longer be answers to it. When there are two good directions to take the question, this is intractable. Perhaps we would delete them, perhaps they're fine just left there, or perhaps we'd want to historical-lock the old questions and let the (likely multiple) more focused versions emerge separately.

Bruce Adams raises some similar questions:

Should we be more aggressive with suggested edits to questions (and answers)? The suggestion part being key. It is most unpleasant to be edited wrongly.

That does mean a lot of moderation work however.

In the case where the question does not invalidate any answers, and they clearly fit the discussed format, people seem to mostly agree that editing them is reasonable. After all, in that case, we’re really just making what was already there more explicit. But we’ll need more meta discussions to decide what to do about existing questions that don’t fit so cleanly into the categories we want to include.

Limited agreement that we should expect more substantiation from answers

Michael writes at quite some length about how we might handle substantiating answers:

As well as questions that are too broad, another issue you identified is that there isn't a culture of expecting answers to provide evidence or rationale, and so answers are upvoted without any reason to believe they're true. Some of this is probably due to the nature of questions, where the breadth of them means there's just too much to talk about, and so answers stray outside what they've got support for; this part probably tidies itself up when the questions are more focused.

Others don't fit in this camp, though, and there are even answers that explicitly disclaim any knowledge or backing. We don't need every point to have a citation attached, but we do need something beyond guesses, wishes, and reckons, and we should ask that of our answers and our answerers. In the first instance that is voting from all of us, and careful thought given to our own answers we post.

There are other tools like NAA flags, post notices, and deletion policies, but before resorting to those we can collectively improve things ourselves. Often this will just mean doing a little research before submitting the answer and won't be a big disruption at all.

Personal experience on matters of readily-observable fact, like what features a language provides, or even primary-source comment on implementing some construct, is probably fine, while personal experience on matters of opinion generally won't be. Answers that just disclaim all relevant knowledge on factual questions shouldn't be encouraged. Questions should be posed to invite answers that have substance—again, setting the answerers up for success.

Bruce also notes that this is a tricky thing to deal with:

Dealing with highly voted wrong answers is hard. I don't think the stack overflow system allows for that very well.

Further discussion is required to decide what precisely to do, but there does seem to be consensus that we ought to hold answers to higher expectations going forward. Comment on answers if you do not think they are sufficiently substantiated, and if other mechanisms (mostly editing and voting) do not resolve the problem, feel free to flag the post, and we can take a look at it and decide what to do.

Next steps

Already, we’ve been continuing to have conversations—both here on meta and in chat—about what we’d like to do to answer some of the questions left open by this discussion. I’ve opened a couple of questions since this one seeking to refine our scope, and they seem to have produced a weak consensus, albeit with some dissent.

Broadly speaking, finding consensus on some of these issues seems challenging given we are still so small. Very often we just don’t have enough people participating on meta to have a healthy number of votes, but that seems okay for now. We can take this slowly and iteratively refine things as the site continues to grow.

I don't have a strong opinion, but some considerations:

If something could be clearly defined, it is not different from it has already existed

The details in language design might be important, but not that important. Many problems are solved in an ad-hoc manner, which may have some annoying consequences, but may not really break anything. As long as it is Turing-complete, people may find workarounds to the deficiencies. Stability is often preferred to improving the details. As a result, one developer of a serious practical language may choose to stick at one design, and live with it for the rest of their life. It would be too slow paced if we depend on them for acquiring the necessary experience. It would cost another developer of another similar language for any feedback of the previous developer's opinion of how it could have been done.

If it's not restricted to serious practical languages, it sounds less clever to require answerers to firstly publish their work as a complete language before posting the answer.

So I don't think it is a good idea to turn a general question that is not already about history to a question about specific or existing languages.

Language design itself is still open-ended

I believe there are still ideas like monads, coroutines, rvalues... not being invented. People would try various ways experimenting how to solve a problem. I myself asked a question, but ended up reinventing "multiple dispatch" and got the question closed. My bad for not doing prior research and / or not formulating the question well. But I don't think it's my bad for thinking about the less common situation. If I don't, there wouldn't be further questions on top of the idea of multiple dispatch.

It may require some thinking about how to make such things fit in the QA format. But even if we rely on outer approval for what has significance and what is esoteric, it's still open ended on the outer part. It could only make us one level worse for being less consistent, like public opinion and commitee. It doesn't really solve anything. The other places don't actually have another standard, so if we make a standard, we actually seem to have better expertise to the eyes.

I think it's difficult to know what is appropriate until we get a good question on this direction. But there are not only technological subsites on SE. There are also something like WorldBuilding. I'm not familiar with their exact rules, but feel one clear idea is more welcome than listing the options there. We may lie somewhere between WorldBuilding and a traditional technological subsite.

Languages are more permissive than the general what-is-best questions

For the question "Spaces and tabs for indentation, which is better?" in other places, it looks like an endless argument. But in PLDI, I suppose the best answer would be: It doesn't matter. It's better you don't add restrictions on that, and give the choice to the programmer. It might be subjective, and trivial, but is not open-ended.

Some questions might be on the borderline, to attract open-ended bad answers without someone making the conclusion. We may need some samples of bad answers to discuss how to deal with them. But if it is that case, the lack of a good answer might be important information to the asker and anyone interested in the topic, as an indication of "doesn't matter" or "do them all". It doesn't look good if we allow the bad answers without an "I didn't find anything better" disclaimer to be the apparently best received answer, but if it happens, I don't think the problem is from the main question body.