Reporting pairwise comparisons apa

Reporting pairwise comparisons apa. 000, report it as < . Despite their psychometric shortcomings, pairwise Wechsler Intelligence Scale for Children—Third Edition (WlSC-III; D. 66, p = . This range does not include zero, which indicates that the difference is statistically significant. friedman. test (y=y, p. The typical application of pairwise comparisons occurs when a researcher is examining more than two group means (i. Wechsler, 1991) and Wechsler Adult Intelligence Scale—Revised (WAIS–R; D. ANP = any-pair power; ALP = all-pairs power; Average = average power. There are more options, and there is no consensus on which test to use. Reviews the literature on the use of pairwise multiple comparisons in terms of between-S and repeated measures experimental designs and the optimality of experimental conditions. As noted earlier, you should don add leading zeros to your p values. HOWEVER, the effect sizes (partial eta squared) were decent (. I have a total of 392 post hoc comparisons: 8 DAT groups = 28 comparison pairs between these groups. D. Click Analyze -> General Linear Model -> Univariate as illustrated below. Compares 2 procedures for protecting the number of false rejections for a set of all possible pairwise comparisons. Replace the [blue text in square brackets] in our template with information from your two-way ANOVA. To do this, you need to identify your data analysis technique, report your test statistic, and provide some interpretation of the results. 5. , in the male and feminine students stylish the Political Scientists major) if the penny select in the “Sig. by jonathon » Mon Oct 11, 2021 9:20 pm. My options: 1. The first one basically means that the covariate should not be different across the groups in the analysis Mar 18, 2020 · Examining these, note that these are all what is known as simple contrasts: comparisons of one factor while holding the other fixed. eff contrasts in my example because of the way you posed your question. X = len, INDEX = list (supp, dose), That’s it for this tutorial. Two methods to compare the Wechsler Intelligence Scale for Children—Third Edition (WISC-III) Index scores were computed. Paired comparison analysis is a structured method of comparing elements or concepts that allows for the determination of which item is more desirable or has more value. The data file is ANOVA_data. 11, SD = 3. Would it be appropriate to report the non-signifcant pairwise comparisons with the effect sizes? Reviews the literature on the use of pairwise multiple comparisons in terms of between-S and repeated measures experimental designs and the optimality of experimental conditions. , the independent variable has more than two levels), and there is a statistically significant effect for the omnibus ANOVA. This data represents a case study where a farmer wanted to investigate increasing the yield of. 3. Aug 23, 2019 · Re: Dwass-Steel-Critchlow-Fligner pairwise comparisons. For instance, when you move to the second smallest p value, you first multiply it by m−1. 62 H = 5. Each comparison will enable you to compare the mean change in reading score between the two methods it considers. 032 still be significant? Jan 12, 2018 · So if we need a measurement and p-value for a mean differences, we get that from the table of pairwise comparisons. Repeated measures ANOVA is also known as ‘within-subjects’ ANOVA. Updated on 04/19/2018. 22, p < Populating the Simple Main Effects APA Guide Include SPSS Outputs; Example APA Submit: Simple Wichtig Effects Tests by adenine Two-Way ANOVA; Pairwise Comparisons in a Two-Way ANOVA. The first 2 are Dunnett-style comparisons of Species at Strain=C57, the next two are the same at CBA, and the last three are comparisons of Strain at each species. ” column concerning the Pairwise Comparisons table for that level of APA Reporting a Kruskal-Wallis Test. If it is already "corrected" using the Bonferroni correction, would a significance level of . , which variable was higher/lower). $\endgroup$ – (13) Report diese information for each pairwise view for which there was an significant difference in to dependent variables. You can see that it looks pretty similar to before, but with a lot more comparisons made. The rejection of the omnibus null hypothesis merely indicates that there is a difference between two or Mar 12, 2023 · These post-hoc tests include the range test, multiple comparison tests, Duncan test, Student-Newman-Keuls test, Tukey test, Scheffé test, Dunnett test, Fisher’s least significant different test, and the Bonferroni test, to name a few. Each of the Kruskal Wallis analysis were significant at . 49 is different from 0. - report two main effects and interaction of Nov 6, 2018 · These will consist of all pairwise comparisons between the three methods. An example is given using athlete type as the independent variable Feb 2, 2022 · This page titled 8. Example of a Repeated-Measures ANOVA Report in Jul 14, 2021 · 11. Use a descriptive statistics table if necessary. (65 Table: Create a table to present the key statistical information. So, in our example, we report 9. my question is what is the best way to report the results in APA format. 57; SD=19. Settle the Simple Main Effects APA Template With SPSS Issue (10) At is a significant difference between the dependent variable for “levels” of independent variable X within a level of independent variable Y (e. APA guidelines for reporting post-hoc tests. 05! i don't actually think that's a problem. However, I now need to see where the difference between my groups lies, so ran pairwise comparisons. (1) Report the exact p value to two or three decimal places as presented in your SPSS output. Significance level for upper case letters (A, B, C): . , Table 1) and include a descriptive title. Average power decreased as the number of groups increased, changing from just under . The guidelines suggest only reporting statistically significant results. 50, p = 0. test command does not offer Tukey post-hoc tests, but there are other R commands that allow for Tukey comparisons. 62 and p = . I obtained my PhD in computer science from the The pairwise comparison of the traffic noise condition with the silence condition was non-significant. 000. You should now be able to conduct simple main effects tests in SPSS for a two-way ANOVA with a significant interaction effect. The significant ANOVA result suggests that the global null hypothesis, H0, is rejected. The data is the weight of barley per unit area (kg/1000m -2) harvested from 60 plots (15 for each treatment). It recommends including that a one-way ANCOVA was conducted to determine differences between levels of an independent variable on a dependent variable while controlling for a covariate. Descriptive statistics only need to be reported in post hoc comparisons that are significant. Backgr ound. Per-pair power decreased slightly as the number of groups increased Jun 23, 2018 · To report the results of Kruskal-Wallis test, you can report the test statistic value, degrees of freedom, and p-value. I performed the Kruskal-Wallis one-way ANOVA by ranks on five groups of data using Dataplot software. The results of the post-hoc comparisons (if the p-value was statistically significant). You should now be able to perform pairwise comparisons in SPSS to determine which levels of your independent variable(s) are significantly different from the other levels of the same variable(s). 009 to . Below, we show code for using the TukeyHSD (Tukey Honest Significant Differences). XXX. The table below shows how to report post hoc tests in APA style. The pairwise comparisons for the main effect of drink corrected using a Bonferroni adjustments are below. Share. 80 for three groups to around . 6. If the omnibus test is significant, you should continue with pairwise comparisons. The confidence intervals indicate the following: The confidence interval for the difference between the means of Blend 2 and Blend 4 extends from 3. Initial, we provision adenine template that you can use to report your two-way ANOVA in APA style. For example, the average English test score for a male is 66. Multiple comparison procedures are commonly used in an ANOVA after obtaining a significant omnibus test result. [SNIP] Post hoc tests are done when you have not defined the groups earlier and an initial exploration reveals that further comparison between groups more than 2 in number is worthwhile. 11 to 15. Drag the cursor over the G eneral Linear Model drop-down menu. You also always report the group means ( M) and standard deviations ( SD ), although you can report them in-text like I did above or in a descriptives table like you can ask from jamovi. After reporting the significance and type of your ANOVA test, you can create a figure showing the difference between the means and the confidence intervals for each pair of groups with significant differences, providing the p-value for each pairwise comparison. This technique is often used when other methods of evaluation are difficult to apply, and it allows decision-makers to get an accurate picture of which option is the most Feb 15, 2021 · Disclaimer: artikel ini dibagi menjadi dua bagian, bagian pertama menjelaskan mengenai pairwise comparison in general dan bagian kedua menjelaskan cara menyusun pairwise comparison matrix Pairwise comparison atau perbandingan berpasangan adalah setiap proses membandingkan entitas berpasangan untuk menilai entitas mana yang lebih disukai atau memiliki jumlah properti kuantitatif yang lebih Example of a Two-Way ANOVA Report in APA Type; Reporting Two-Way ANOVA Follow Up Tested (Optional) Simple Chief Effects Tests for a Two-Way ANOVA. My very specific questions are: 1. (22) Use the text highlighted stylish gold to report any pairwise comparisons that were significant. Shaffer, 1986) are very attractive insofar as they represent simple, yet powerful, data-analytic tools for behavioral researchers. The advantage to starting here is that if the omnibus test comes up insignificant, you can stop your analysis and deem all pairwise comparisons insignificant. in a factorial design with a variable of interest having more than two levels, a procedure in which the data obtained from each level of the variable are compared separately to the data from every other level. 229 so the results are not significant ( p p is not less than . Click R epeated Measures. The subscripts are based on the Homogeneous Subsets table in our ANOVA output. The Two-Way Analysis of Variance (ANOVA) Take the following steps to perform a two-way ANOVA in SPSS. (2-tailed)” column of the “Paired Samples Test” table. 009. You could say something like: "Pairwise comparisons using Dunn's test indicated that group C scores were observed to be significantly different from those of group A (p = . 00. com In a results section, your goal is to report the results of the data analyses used to test your hypotheses. Second, the differences required for significance when comparing an individual's Index score to the mean of all the Index scores were Writing up your results – Guidelines based on APA style. 15 SDRT tests. There was a significant difference among the distributions of the three types of animals, F(2,22) = 22. Feb 8, 2024 · The paired t-test is a statistical tool of precision employed to discern the effect of an intervention by comparing two sets of observations from the same subjects under different conditions. correctly reject all of the null hypotheses that should be rejected. 012). Populating the Simple Main Effects APA Template With SPSS Output (10) There is a significant diff bets the dependent variable for “levels” regarding independent variable X within a level of independently variable Y (e. <br> <br> 2. So, when we run the Tukey post-hoc analysis for this ANOVA model, we see that it has made a lot of pairwise comparisons (19 in total), as shown in Fig. Researchers want to know if four different drugs lead to different reaction times. Post-hoc analysis with Wilcoxon signed-rank test was conducted with a Bonferroni correction applied, resulting in a significance level set at p < 0. e. (14) Account this information for each pairwise comparison for which there were no significant difference in one dependent variable. The 2-stage strategy of computing pairwise comparisons, conditional on a significant omnibus test, is compared with the multiple comparison strategy that sets a "familywise" critical value directly. However, if the p value is . If the p value is . As noted previously, you should not add leading zeros to your p values. For a complete explanation of the output you have to interpret when checking your data for the nine assumptions required to carry out a Abstract. 26, SD=21. This brings up the Univariate dialog box. 5 inches from the left margin; and (b) double-space your text. , intermediate the man and women college inbound the Political Science major) when the p true in the “Sig. Full details are here with an excerpt reproduced below. ” column of the “Pairwise Comparisons” table. Example APA Report: Pairwise Comparisons for a Two-Way ANOVA. 2. 0002) and group B (p Jan 24, 2011 · One way to begin an ANOVA is to run a general omnibus test. 38). Simplified Main Effects Tests Template; Populating the Easier Main Effects APA Preview With SPSS Output; Example APA Report: Simply Main Effects Tests for a Two-Way ANOVA; Halved Comparisons Dec 3, 2021 · In order to find out which group means are different, we can then perform post-hoc pairwise comparisons. Apr 1, 2017 · I have run a Kruskal Wallis Test on my data and it is significant at p=0. 60 for seven groups. it seems reasonable that you'd have enough data to establish that not all groups are 'equal', but not enough to establish which To find out how to run an ANOVA followed by multiple comparisons in Excel using XLSTAT, check out the tutorial here. ” column of the Pairwise Comparisons size for that level The example also shows a common way of reporting the results of the pairwise LSD comparisons – using letters to indicate which means were significantly different. 27. 30), F(1, 27) = 8. If the p-value is less than the significance level (usually 0. The H0 hypothesis states that the means are the same across the groups being compared. Exemplar APA Report: Simple Main Effects Checks for an Two-Way ANOVA for hypothesis 2: ive done three independent samples t-tests. To report the results of a linear mixed models analysis, you can use regression tables which should include the coefficients, standard errors, t-values, p-values, and confidence intervals for each Example with mean: Comparison of the repeated measures was performed using Friedman’s test showing a statistically significant decrease in concentration, χ2 (3) = 24. See full list on scribbr. Do not add a leading zero to your p value. You should now be able to run a repeated-measures ANOVA, test the assumption of sphericity, make use of a pairwise comparison, and report the result. May 1, 2016 · How to interpret the Pairwise Comparisons Table produced by SPSS for a 2-way interaction in a 2 x 3 ANOVA. 001. 296, 11. Jan 8, 2024 · First, you sort all of your p-values in order, from smallest to largest. You may also be interested in our tutorials on: (1) reporting the results of a two-way ANOVA in APA style ; and (2) exporting SPSS output to another Individual confidence level = 98. Here’s the exact wording we can use: Example APA Report: Pairwise Comparisons for a Two-Way ANOVA. Now, assume you want to conduct a slightly more complicated study, where you keep track not only of the change in reading score for each child but also their Apr 6, 2018 · The interaction was not significant. Although the APA doesn't mention it, we encourage reporting the mean ranks and perhaps some other descriptives statistics in a Jul 22, 2021 · The following example shows how to report the results of a repeated measures ANOVA in practice. ANOVA and multiple pairwise comparison tests examine different questions. 877, p < 0. 0001 indicated we do have evidence that this one mean difference of 5. The ANOVA test, the Brown and Forsythe test, and the Welch omnibus test, as well Click A nalyze. The software corrects for ties and uses Conover's procedure to do pairwise multiple comparisons. Those comparisons need to include the nature of the difference (i. In this section, we show you only the main tables required to understand your results from the one-way ANCOVA and the post hoc test. Report the results of post hoc analysis ONLY IF THE MAIN EFFECTS RESULTS ARE SIGNIFICANT. Note that we chose α = . For the smallest p-value all you do is multiply it by m, and you’re done. In addition it is adviced to correct for multiple testing if the number of groups The APA 6th edition’s examples are not specific to the Kruskal-Wallis and I don’t want to make assumptions. 88), while the average English test score for a female is 69. XX, p = . This table indicates that the significant main effect reflects a significant difference (p < . 028 for the omnibus test but none of the DSCF comparisons have p<0. 5. 32) was significantly higher than those in the silence condition (M = 8. Picked SPSS Output; Notes and Tips for Writing Yours Report in APA Style; Example of a Two-Way ANOVA Get in APA Style; Reporting Two-Way ANOVA Trail Up Tests (Optional) Simple Major Effects Tests for SPSS gives me a significance level for each pairwise comparison, but I do not know if there are F tests to report for each of these (along with degrees of freedom). The result of the test is H = 5. Interpretation of contradictory results between ANOVA and multiple pairwise comparisons. This is the equivalent of a one-way ANOVA but for repeated samples and is an extension of a paired-samples t-test. Also known as “paired ranking”, it is a popular research method used for ranking people’s preferences, informing strategic decisions, and conducting voting at scale. Per View Template; Populating the Pairwise Comparisons APA Template use SPSS Output; Example APA Report: Halved Compare for a Two-Way ANOVA 2. In he PMCMR there's also a Durbin test (for incomplete block designs that is the same as Friedman where the design is For each significant pair, the key of the category with the smaller column proportion appears in the category with the larger column proportion. 000 however (as it is in our example), report it as < . 1: Pairwise Comparison Post Hoc Tests for Critical Values of Mean Differences Expand/collapse global location Apr 23, 2022 · Figure \(\PageIndex{1}\) shows the number of possible comparisons between pairs of means (pairwise comparisons) as a function of the number of means. May 17, 2021 · When reporting the results of a one-way ANOVA, we always use the following general structure: A brief description of the independent and dependent variable. How to Report Results of One-Way ANOVA Test in APA. This table itself was created with a MEANS command like we used for Descriptive Statistics for Subgroups. t. 05), you can (20) Report this information for respectively pairs comparison since which there what negative important variation in the dependent variable. I have currently reported the output as H (3) = 18. Simple Main Effects Tests Template; Populating the Simple Main Effects APA Template By SPSS Yield; Example APA Report: Simple Main Effects Trial for ampere Two-Way ANOVA; Pairwise Comparisons for a Jul 25, 2019 · 0. The rejection of the omnibus null hypothesis merely indicates that there is a difference between two or If your t value is negative, you can choose to report it as a positive value. In the W ithin-Subject Factor Name: box, give the outcome variable a name. It can be helpful to present a descriptive statistics table that shows the mean and standard deviation of values in each treatment group as well to give the reader a more complete picture of the data. However, there’s also an art to it. 2, SD = 6. Meet Shengdong (Shen) Zhao. Repeated measures ANOVA analyses (1) changes in mean score over 3 or more time points or (2) differences in mean score under 3 or more conditions. We recommend that you click the Reset button to clear any previous settings. TukeyHSD(a1) Tukey multiple comparisons of means. Type the number of observations of the continuous outcome being collected into the Number of L evels: box. For a complete explanation of the output you have to interpret when checking your data for the nine assumptions required to carry Pairwise Comparison. 5: Introduction to Pairwise Comparisons 11. What statistics do I report for these post hoc test? Just the means and standard deviations for each level of the independent variable? We can report that when using an ANOVA with repeated measures with a Greenhouse-Geisser correction, the mean scores for CRP concentration were statistically significantly different (F(1. The overall F-value of the ANOVA and the corresponding p-value. 43secs - just to clarify, in a repeated measures ANOVA the participants are the same in each of the groupsLearn how to conduct a one way within subjects AN Example of a Two-Way ANOVA Report in APA Style; Reporting Two-Way ANOVA Follow Up Checks (Optional) Simple Prime Effects Examinations for a Two-Way ANOVA. 5: Pairwise Comparisons is shared under a Public Domain license and was authored, remixed, and/or curated by David Lane via source content that was edited to the style and standards of the LibreTexts platform; a detailed edit history is available upon request. pairwise, Dunnett, etc. Begin with a concise description of the analysis conducted, including the test name, the dependent variable, and the independent variable representing the groups. Template for Reporting a Two-Way ANOVA in APA Style. Here, looking at the adjusted significance there is a statistically significant difference between two Jan 11, 2011 · Next, I ran Bonferonni corrected post hoc tests and found that all of my pairwise comparisons were significant (i. The average number of errors in all noise conditions combined (M = 15. In addition to the pairwise comparison, it gives you P-value that tells you if you would reject the null hypothesis at a particular significant level or not. This mixed-model approach was evaluated for testing all possible pairwise differences among repeated measures marginal means in a Between-Subjects × Within There are quite a lot of these comparisons that you need to consider. 0083. In this section, we show you only the main tables required to understand your results from the one-way MANOVA and Tukey post-hoc tests. 663) = 26. 05 . Same for the pairwise comparisons of Group A and B for writing and Group A and B for typing. If there are \(12\) means, then there are \(66\) possible comparisons. Alternatively, check out our tutorial on reporting a repeated-measures ANOVA from SPSS in APA style. Pairwise Comparisons Table (24) Report the exact p value for each significant pairwise comparison to two or three decimal places as per the “Sig. The example reported below includes the results away both the two-way ANOVA and the pairwise comparisons conducted nach which ANOVA: Sep 16, 2016 · 1. 4. Jan 16, 2015 · $\begingroup$ There are lots of other contrast families available in lsmeans, e. You may also be interested in our tutorials on: (1) reporting the results of a two-way ANOVA in APA style and (2) exporting SPSS output to another application such as Word, Excel, or PDF. Lezak, 1995), an endeavor in which hypotheses from pairwise comparisons can be tested with . Example: Reporting Results of a Repeated Measures ANOVA. adjust="none")` from the PMCMR [/code] library. Cite. Reporting the results of a One-Way ANOVA Test in APA style ensures clarity and adherence to established guidelines. The table should be labeled with a number (e. (9) Report the p value to two or three decimal places as per the “Sig. barley with the use of fertilizers. The SDRT is a Reading comprehension test. Feb 2, 2022 · This page titled 8. Each analysis you run should be related to your hypotheses. ***** Okay, that’s all for this tutorial. These can even be reported without a p-value with only an explanation of how the pairs of group means differed required. 168. If there are only two means, then only one comparison can be made. > #use anova (object) to test the omnibus ANCOVA has the same assumptions as any linear model (see your handout on bias) except that there are two important additional considerations: (1) independence of the covariate and treatment effect, and (2) homogeneity of regression slopes. ” Jan 4, 2024 · Paired comparison is the process of comparing a set of options using head-to-head pairs to judge which one is the most preferred overall. Marascuilo and Serlin (1988), Statistical methods for the social and behavioral sciences, give this relationship between eta-squared (variance accounted for) and the Kruskal-Wallis H statistic (a SPSS Statistics generates quite a few tables in its one-way ANCOVA analysis. I am a full professor at the School of Creative Media and Computer Science Department at the City University of Hong Kong (CityU). In this tutorial our show you whereby to report a two-way ANOVA (analysis of variance) from SPSS int APA style. 01) between levels 1 and 2 (beer and wine) and 1 and 3 (beer and water) but not between levels 2 and 3 (wine and water). g. Apr 19, 2018 · pairwise comparison. SPSS Statistics produces many different tables in its one-way MANOVA analysis. For example, you always report the statistics exactly the same: F (df BG, df WG) = X. I used del. sav. First, pairwise comparisons of the Index and IQ scores were computed using the Bonferroni correction for the number of comparisons made. APA Style Reporting Post Hoc Tests. 05 ). 05 Jul 19, 2021 · Here are a few things to keep in mind when reporting the results of a two-way ANOVA: 1. 26 (M=69. ANOV A and pairwise comparisons – SPSS Instructio ns. 938, p < . Dunne's test compares the Z-test which is comparison of the means when the variances are known and the sample size is large. Tests are adjusted for all pairwise comparisons within a row of each innermost subtable using the Bonferroni correction. Beyond pairwise comparisons of means, the versatile S. APA Template Text: Two-Way ANOVA; Populating an Two-Way ANOVA APA Template With SPSS Output. Example: “LSD post hoc test results revealed that the Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris; Duis aute irure dolor in reprehenderit in voluptate; Excepteur sint occaecat cupidatat non proident One approach to the analysis of repeated measures data allows researchers to model the covariance structure of the data rather than presume a certain structure, as is the case with conventional univariate and multivariate test statistics. In a results section, your goal is to report the results of the data analyses used to test your hypotheses. 0005). To test this, they measure the reaction time of five patients on the four different drugs. The next example illustrates its usage: data <- datasets:: ToothGrowth result <- with (data, tapply (. 27 rather than -9. Jan 1, 2024 · The typical application of pairwise comparisons occurs when a researcher is examining more than two group means (i. 95% family-wise confidence level. However, for all the other ones it’s a two-stage process. Mar 31, 2012 · 5. 039. conover. Previously, I held the position of Associate Professor at the Computer Science Department of the National University of Singapore (NUS). 047, p<0. P. However, the test is named Durbin-Conover in jamovi. 89%. For example, if a researcher is assessing the differences in student achievement resulting (21) Report one exact p value for each non-significant pairwise related to two or three decimal places as per the “Sig. The following example shows how to perform the following post-hoc pairwise comparisons in R: The Tukey Method; The Scheffe Method; The Bonferroni Method; The Holm Method; Example: One-Way ANOVA in R Jun 23, 2020 · Looking at the R script (in your print screen) one can see that the post-hoc test is `posthoc. 01 instead of the usual α = . How to Report Descriptive Statistics Table in SPSS Output? The table shows the number of observations, the mean and standard deviation in each group, and total. 05. Here's an example format: Table 1 Sep 16, 2014 · The document provides guidance on reporting the results of an ANCOVA analysis in APA format. Wechsler, 1981) subtest scaled score comparisons can be important in neuropsychological evaluations (M. e, 1 versus 2, 2 versus 3, and 1 versus 3). This guide covers every common As specified in the APA Style Guide :, you should: (a) indent the first line of each paragraph 0. 89. Its importance in research is profound, offering insights into the efficacy of treatments, the impact of educational programs, and more. Tests are recommended for use in conditions in which the assumptions of normality, homogeneous population variances, and equal sample sizes are violated. And our p-value below . It tells us whether the mean BMI difference between medium and small frame males is the same as 0. So p=0. Template for Reporting an Two-Way ANOVA in APA Style. 229 p = . Each analysis you run should be related It also let you perform centrality and dispersion statistics for inline results by using the cent_disp() function. We show you how to populate a using your SPSS output, and we give you an examples report written in our template. For APA reporting our example analysis, we could write something like “a Kruskal-Wallis test indicated that the amount of antibodies differed over vaccines, H(2) = 6. Second, the differences required for significance when comparing an individual's Index score to the mean of all the Index scores were The pairwise. 57 (M=66. Holm (1979) procedure and its modifications (J. Aug 8, 2020 · Our conclusions: When researchers choose to report P values in randomized experiments, 1) Fisher-exact P values should be used, especially in studies with small sample sizes, and 2) the shape of the actual null randomization distribution should be examined for the recondite scientific insights it may reveal. wf yq lq wc bo vp rk wq si ja