Here are some of my thoughts regarding Question 3 after reading all the insightful comments and answers.

Perhaps one practical guidance in statistical analysis to avoid p-value hacking is to instead look at the scientifically (or, biologically, clinically, etc) significant/meaningful effect size.

Specifically, the research should pre-define the effect size that can be declared useful or meaningful before the data analysis or even before the data collection. For example, if let $\theta$ denote a drug effect, instead of testing the following hypothesis, $$H_0: \theta = 0 \quad \quad vs. \quad \quad H_a: \theta \neq 0,$$ one should always test $$H_0: \theta < \delta \quad \quad vs. \quad \quad H_a: \theta \ge \delta,$$ with $\delta$ being the predefined effect size to claim meaningful significance.

In addition, to avoid of using too large sample size to detect the effect, the sample size required should be taken into account as well. That is, we should put a constrain on the maximum sample size used for the experiment.

To sum up,

1. We need predefine a threshold for the meaningful effect size to declare significance;
2. We need to predefine a threshold for sample size used in the experiment to quantify how detectable the meaningful effect size is;

With above, maybe we can therefore avoid minor "significant" effect claimed by a huge sample size. Any comments about the applicability?

[EDIT]

I have changed the second set of the hypotheses from $$H_0: \theta = 0 \quad \quad vs. \quad \quad H_a: \theta = \delta,$$ to $$H_0: \theta < \delta \quad \quad vs. \quad \quad H_a: \theta \ge \delta,$$ according to the comment from @AndyW (Thanks, @AndyW). Would this sound better?

[END EDIT]

Here are some of my thoughts regarding Question 3 after reading all the insightful comments and answers.

Perhaps one practical guidance in statistical analysis to avoid p-value hacking is to instead look at the scientifically (or, biologically, clinically, etc) significant/meaningful effect size.

Specifically, the research should pre-define the effect size that can be declared useful or meaningful before the data analysis or even before the data collection. For example, if let $\theta$ denote a drug effect, instead of testing the following hypothesis, $$H_0: \theta = 0 \quad \quad vs. \quad \quad H_a: \theta \neq 0,$$ one should always test $$H_0: \theta < \delta \quad \quad vs. \quad \quad H_a: \theta \ge \delta,$$ with $\delta$ being the predefined effect size to claim meaningful significance.

In addition, to avoid of using too large sample size to detect the effect, the sample size required should be taken into account as well. That is, we should put a constrain on the maximum sample size used for the experiment.

To sum up,

1. We need predefine a threshold for the meaningful effect size to declare significance;
2. We need to predefine a threshold for sample size used in the experiment to quantify how detectable the meaningful effect size is;

With above, maybe we can therefore avoid minor "significant" effect claimed by a huge sample size.

[Update 6/9/2015]

Regarding Question 3, here are some suggestions based on the recent paper from nature: "The fickle P value generates irreproducible results" as I mentioned in the Question part.

1. Report effect size estimates and their precision, i.e. 95% confidence interval, since those more informative information answer exactly questions like how big is the difference, or how strong is the relationship or association;
2. Put the effect size estimates and 95% CIs into the context of the specific scientific studies/questions and focus on their relevance of answering those questions and discount the fickle P value;
3. Replace the power analysis with "planning for precision" to determine the sample size required for estimating the effect size to reach a defined degree of precision.

[End update 6/9/2015]

Here are some of my thoughts regarding Question 3 after reading all the insightful comments and answers.

Perhaps one practical guidance in statistical analysis to avoid p-value hacking is to instead look at the scientifically (or, biologically, clinically, etc) significant/meaningful effect size.

Specifically, the research should pre-define the effect size that can be declared useful or meaningful before the data analysis or even before the data collection. For example, if let $\theta$ denote a drug effect, instead of testing the following hypothesis, $$H_0: \theta = 0 \quad \quad vs. \quad \quad H_a: \theta \neq 0,$$ one should always test $$H_0: \theta = 0 \quad \quad vs. \quad \quad H_a: \theta = \delta,$$ with $\delta$ being the predefined effect size to claim meaningful significance.

In addition, to avoid of using too large sample size to detect the effect, the sample size required should be taken into account as well. That is, we should put a constrain on the maximum sample size used for the experiment.

To sum up,

1. We need predefine a threshold for the meaningful effect size to declare significance;
2. We need to predefine a threshold for sample size used in the experiment to quantify how detectable the meaningful effect size is;

With above, maybe we can therefore avoid minor "significant" effect claimed by a huge sample size. Any comments about the applicability?

Here are some of my thoughts regarding Question 3 after reading all the insightful comments and answers.

Perhaps one practical guidance in statistical analysis to avoid p-value hacking is to instead look at the scientifically (or, biologically, clinically, etc) significant/meaningful effect size.

Specifically, the research should pre-define the effect size that can be declared useful or meaningful before the data analysis or even before the data collection. For example, if let $\theta$ denote a drug effect, instead of testing the following hypothesis, $$H_0: \theta = 0 \quad \quad vs. \quad \quad H_a: \theta \neq 0,$$ one should always test $$H_0: \theta < \delta \quad \quad vs. \quad \quad H_a: \theta \ge \delta,$$ with $\delta$ being the predefined effect size to claim meaningful significance.

In addition, to avoid of using too large sample size to detect the effect, the sample size required should be taken into account as well. That is, we should put a constrain on the maximum sample size used for the experiment.

To sum up,

1. We need predefine a threshold for the meaningful effect size to declare significance;
2. We need to predefine a threshold for sample size used in the experiment to quantify how detectable the meaningful effect size is;

With above, maybe we can therefore avoid minor "significant" effect claimed by a huge sample size. Any comments about the applicability?

[EDIT]

I have changed the second set of the hypotheses from $$H_0: \theta = 0 \quad \quad vs. \quad \quad H_a: \theta = \delta,$$ to $$H_0: \theta < \delta \quad \quad vs. \quad \quad H_a: \theta \ge \delta,$$ according to the comment from @AndyW (Thanks, @AndyW). Would this sound better?

[END EDIT]