Statistical Analysis: A Guide to Methods and Tests
Statistical analysis turns raw data into results you can draw conclusions from. The hardest part is not the arithmetic but choosing the right method: every research question has a test that fits it, and every test has assumptions that invalidate its result when they are violated. This guide covers the methods Osloop supports and when to use each.
How do you choose the right statistical method?
The choice comes down to four questions, answered in order:
What type of data do you have?
Quantitative (lengths, scores) or categorical (male/female, yes/no)? Categorical data call for entirely different tests.
How many groups are you comparing?
One group against a benchmark, two groups, or three or more? Each case has its own test.
Are the groups independent?
Different people (independent), or the same people before and after (related)?
Is normality met?
If it is, use parametric methods; otherwise, their non-parametric counterparts.
An example
You want to measure the effect of a training program on one group before and after the training: the data are quantitative, and you compare two measurements of the same people, so the two sets are related. If normality is met, the method is the paired samples t-test; otherwise, the Wilcoxon signed-rank test.
Supported statistical methods
Sixteen methods covering most of what theses and applied research need.
Descriptive Statistics
Summarizing and describing data with the mean, standard deviation and charts.
Frequencies and Percentages
The number of cases in each category and the share of the total sample it represents.
One-Sample t-Test
Comparing a sample mean with a known or hypothesized mean.
Independent Samples t-Test
Comparing the means of two independent groups on a given variable.
Paired Samples t-Test
Comparing the same sample before and after a treatment.
One-Way ANOVA (Analysis of Variance)
Comparing the means of three or more independent groups.
Mann–Whitney U Test
The alternative to the independent samples t-test when normality is not met.
Wilcoxon Signed-Rank Test
The alternative to the paired samples t-test when normality is not met.
Kruskal–Wallis Test
The alternative to ANOVA for three or more groups when normality is not met.
Chi-Square Test and Fisher’s Exact Test
The relationship between two categorical variables.
McNemar’s Test
Change in a binary response for the same sample under two conditions.
Pearson Correlation Coefficient
The strength and direction of the linear relationship between two quantitative variables.
Spearman Correlation Coefficient
The relationship between two ordinal or non-linearly related variables.
Cronbach’s Alpha Reliability Coefficient
The internal consistency reliability of the items in a questionnaire or test.
Normality Test
Do the data follow a normal distribution? This decides which method to use.
Difficulty Index and Discrimination Index
The quality of test items: how difficult they are and how well they discriminate.
Assumptions an analysis cannot ignore
Normality
Check the distribution before choosing the method. With small samples the Shapiro–Wilk test has little power to detect departures, so passing it is not conclusive proof.
Homogeneity of variance
Check it with Levene’s test before an independent samples t-test. If it is violated, switch to Welch’s version rather than the standard one.
Independence of observations
Each observation comes from an independent individual. Stacking the ratings of several raters inflates the number of observations artificially and breaks this assumption.
Sample size and power
With a small sample, “no significant differences” means “no differences were detected”, not “there are no differences” — and the gap between the two is large.
Effect size, not significance alone
Significance tells you whether a difference exists; effect size tells you how big it is. The second is what matters to the reader in practice.
Reliability of the instrument
Cronbach’s alpha measures the consistency of the items; the intraclass correlation coefficient measures agreement between raters. Neither replaces the other.
Let Osloop choose the right test and run it
Upload your data, choose your research question, and get the results and tables ready for interpretation.
Start your analysis now