Once I determine my sample size for my research, I usually spend time thinking over and documenting the potential tests that I may use to determine statistical significance for my research results.
Let me confess.
I never did this in the initial phases of my research. I always used to always “Why bother before we collect data, will decide what tests to use after we collect the data”. A few research projects down the line, I sort of had my light bulb moment (one of the many millions such moments) and figured that it helped bring clarity to my research and analysis if i thought of this before I started data collection. I realized that thinking over these tests before data collection actually helps me to get more clarity on my variables and the way I measure them.
I use this process to link back to all preceding steps of my research protocol development including the hypothesis, question, aims, designs, variables, measures, sample size and cross check that I am collecting what I need and only what I need and in the way that it will be useful for me to answer the research question.
I write down all the tests I might do and the rationale for that. It is possible, and it does happen, that I may have to choose a different test or approach after data collection. That is acceptable to me.
I am sharing a brief document on common statistical tests. Please note that I am not getting into a mathematical approach here or going into great detail. For those of you who like that, the internet gives wonderful resources.
One word, Personally I prefer a simple approach to statistics. Sometimes, we need complex tests or modeling but most of the times a simple approach will suffice to help us translate results to clinical care. Keep it simple.