Based on the diagram, this analysis should make a comparison between two or more groups (i.e. the combinations of the categories of the factors of interest) and the analysis includes a single continuous outcome measure, at least four covariates and at least five categorical factors of interest. All groups are independent and if experimental units are repeatedly measured, a summary measure is used in this analysis.

If this description is not accurate, please check your diagram and verify that all nodes are connected properly, all variables and variable categories are indicated and tagged to the relevant interventions or measurements and the information provided in the properties of each node is accurate; then critique it again.

The number of factors of interest is very high. With five factors, a full factorial design would need at least 32 groups (5 factors with at least 2 categories each: 2⁵) and it seems unlikely that the experiment will be large enough (in terms of numbers of animals) to estimate all the interactions reliably. For example, with five factors, there are ten possible two-way interactions. In practice three-way (and above) interactions are unlikely to occur and you should consider reducing the number of factors of interest. Alternatively you could manually amend the statistical model to reduce the number of higher-order interactions included in the model; in this case the best course of action would be to consult a statistician.

The number of covariates is also very high. Although you may decide to measure all of these covariates during the experiment, the decision to include each covariate in the statistical analysis depends on each covariate explaining some of the unaccounted for variability and meeting the assumptions outlined below. It seems unlikely that all covariates you have selected will each explain a separate amount of the unaccounted for variability – it is likely that at least two covariates will be correlated and hence explain the same source of variability. If this is the case, then only one of the pair of covariates needs to be included in the statistical model. More information on when it is appropriate to include a covariate can be found on the independent variables page of the EDA website.

Statistical analysis methods compatible with this design include an independent factorial ANCOVA (two-way, three-way or four-way ANCOVA, depending on the number of factors of interest) and an independent factorial ANCOVA on the rank transformed outcome measure.

The ANCOVA approach assumes that the data satisfies these assumptions:residuals are normally distributed, homogeneity of variance, independence of the errors and the outcome is measured on a continuous scale (read more about parametric and non-parametric tests), as well as assuming the covariate(s) should be used in the analysis.

A covariate should be used if:

• The covariate is independent of the treatment.
• There is a strong relationship between the covariate and the outcome measure (i.e. either they both increase together or one increases while the other decreases).
• The relationship between the outcome measure and the covariate is similar for all treatments (i.e. there is no significant treatment by covariate interaction).

The above assumptions can be tested by plotting your data, details of what to look for and example graphs can be found on the independent variables page of the EDA website. If there are multiple covariates you are considering including in your analysis, ensure that the assumptions hold for each of them.

In many cases you will not know if including a particular covariate in your analysis is appropriate when planning your experiment. You should measure the covariate during your study, but only include it in your statistical analysis if the assumptions for covariate inclusion are met.

If you have reasons to think the data are not normally distributed, you should consider transforming the data to normalise it (read more about data transformation) and assess if the transformed data satisfies the normality assumptions. Most data can be normalised using transformations such as log or square root and using parametric tests is preferable as they have more statistical power than non-parametric tests, as long as the required assumptions are met.

Alternatively, if data do not satisfy the normality assumptions and the transformation doesn’t work, a non-parametric test could be used. As there is no non-parametric equivalent to the factorial ANCOVA, data could be ranked and an ANCOVA on the rank transformed outcome measure could be run. Note that there are assumptions associated with non-parametric tests also. For example, to perform a rank transformation the data must be able to be ranked, with only a few ties (e.g. identical values that will end up with the same rank), the observations must be independent and the covariate must have a linear relationship with the rank. If data cannot be rank transformed (e.g. it is mainly zeros with only a few non-zero measures), the data can be recoded into binary responses and then analysed using logistic regression. Another analysis option is ordinal logistic regression. Your local statistician can help advise on this. These approaches will lead to a loss in power due to the categorisation of continuous data.

Interpreting a factorial ANCOVA

There is a common mistake when interpreting the results of a factorial ANCOVA. For example, in a situation with two factors of interest and at least one covariate such as an experiment on the effect of exercise on neuronal density, investigated in animals of both sexes, with baseline locomotor activity as a covariate. A claim that the overall effect of exercise is different in males and females can only be supported by the finding that the interaction between the two factors is statistically significant (i.e. the size of the effect is different in males and females). A significant effect of exercise in one sex but not the other is not appropriate to support the claim that there is a difference between sexes.

Analysis software

Software such as InVivoStat can be used to run either of these statistical tests, and apply data transformations. The tests can be found in the following menu:

• Two-way, three-way or four-way ANCOVA: Statistics>Single Measure Parametric Analysis

References and further reading

Nieuwenhuis, S, Forstmann, BU and Wagenmakers, EJ (2011). Erroneous analyses of interactions in neuroscience: a problem of significance. Nat Neurosci 14(9):1105-7. doi: 10.1038/nn.2886

https://www.graphpad.com/guides/prism/latest/statistics/how_to_think_about_results_from_two-way_anova.htm

Based on the diagram, this analysis should make a comparison between two or more groups (i.e. the combinations of the categories of the factors of interest) and the analysis includes a single continuous outcome measure, at least four covariates and at least two categorical factors of interest. All groups are independent and if experimental units are repeatedly measured, a summary measure is used in this analysis.

If this description is not accurate, please check your diagram and verify that all nodes are connected properly, all variables and variable categories are indicated and tagged to the relevant interventions or measurements and the information provided in the properties of each node is accurate; then critique it again.

The number of covariates is very high. Although you may decide to measure all of these covariates during the experiment, the decision to include each covariate in the statistical analysis depends on each covariate explaining some of the unaccounted for variability and meeting the assumptions outlined below. It seems unlikely that all covariates you have selected will each explain a separate amount of the unaccounted for variability – it is likely that at least two covariates will be correlated and hence explain the same source of variability. If this is the case, then only one of the pair of covariates needs to be included in the statistical model. More information on when it is appropriate to include a covariate can be found on the independent variables page of the EDA website.

Statistical analysis methods compatible with this design include an independent factorial ANCOVA (two-way, three-way or four-way ANCOVA, depending on the number of factors of interest) and an independent factorial ANCOVA on the rank transformed outcome measure.

The ANCOVA approach assumes that the data satisfies these assumptions: residuals are normally distributed, homogeneity of variance, independence of the errors and the outcome is measured on a continuous scale (read more about parametric and non-parametric tests), as well as assuming the covariate(s) should be used in the analysis.

A covariate should be used if:

• The covariate is independent of the treatment.
• There is a strong relationship between the covariate and the outcome measure (i.e. either they both increase together or one increases while the other decreases).
• The relationship between the outcome measure and the covariate is similar for all treatments (i.e. there is no significant treatment by covariate interaction).

The above assumptions can be tested by plotting your data, details of what to look for and example graphs can be found on the independent variables page of the EDA website. If there are multiple covariates you are considering including in your analysis, ensure that the assumptions hold for each of them.

In many cases you will not know if including a particular covariate in your analysis is appropriate when planning your experiment. You should measure the covariate during your study, but only include it in your statistical analysis if the assumptions for covariate inclusion are met.

If you have reasons to think the data are not normally distributed, you should consider transforming the data to normalise it (read more about data transformation) and assess if the transformed data satisfies the normality assumptions. Most data can be normalised using transformations such as log or square root and using parametric tests is preferable as they have more statistical power than non-parametric tests, as long as the required assumptions are met.

Alternatively, if data do not satisfy the normality assumptions and the transformation doesn’t work, a non-parametric test could be used. As there is no non-parametric equivalent to the factorial ANCOVA, data could be ranked and an ANCOVA on the rank transformed outcome measure could be run. Note that there are assumptions associated with non-parametric tests also. For example, to perform a rank transformation the data must be able to be ranked, with only a few ties (e.g. identical values that will end up with the same rank), the observations must be independent and the covariate must have a linear relationship with the rank. If data cannot be rank transformed (e.g. it is mainly zeros with only a few non-zero measures), the data can be recoded into binary responses and then analysed using logistic regression. Another analysis option is ordinal logistic regression. Your local statistician can help advise on this. These approaches will lead to a loss in power due to the categorisation of continuous data.

Interpreting a factorial ANCOVA

There is a common mistake when interpreting the results of a factorial ANCOVA. For example, in a situation with two factors of interest and at least one covariate such as an experiment on the effect of exercise on neuronal density, investigated in animals of both sexes, with baseline locomotor activity as a covariate. A claim that the overall effect of exercise is different in males and females can only be supported by the finding that the interaction between the two factors is statistically significant (i.e. the size of the effect is different in males and females). A significant effect of exercise in one sex but not the other is not appropriate to support the claim that there is a difference between sexes.

Analysis software

Software such as InVivoStat can be used to run two-way, three-way or four-way ANCOVAs, and apply data transformations. The tests can be found in the following menu:

• Statistics>Single Measure Parametric Analysis

References and further reading

Nieuwenhuis, S, Forstmann, BU and Wagenmakers, EJ (2011). Erroneous analyses of interactions in neuroscience: a problem of significance. Nat Neurosci 14(9):1105-7. doi: 10.1038/nn.2886

https://www.graphpad.com/guides/prism/latest/statistics/how_to_think_about_results_from_two-way_anova.htm

Based on the diagram, this analysis should make a comparison between measurements at different time points within a single group, or a comparison between at least two related groups. For example, multiple treatments may be tested (in the same order) within each animal or a single treatment is administered to all animals and the animals are then assessed at multiple time points. This analysis includes a single continuous outcome measure, at least four covariates, five or more blocking factors and a single categorical factor of interest: the repeated factor.

If this description is not accurate, please check your diagram and verify that all nodes are connected properly, all variable categories are indicated and tagged to the relevant interventions or measurements and the information provided in the properties of each node is accurate; then critique it again.

The number of blocking factors is very high. The decision to include blocking factors in the randomisation and the analysis depends on previous experience and/or literature evidence that nuisance variables (e.g. practical constraints on the experiment) may influence the outcome and potentially increase the variability. Blocking involves dividing the experiment into a series of mini-experiments; each mini-experiment should ideally contain all the treatments but have fewer animals per treatment group than the full design. If all treatments are present in each of the mini-experiments, then five blocking factors imply dividing the experiment into a minimum of 32 mini-experiments (5 factors with at least 2 categories each: 2⁵) and it seems unlikely to run an experiment with such a large number of animals. The best course of action would be to consult a statistician to estimate the effect of these blocking factors, and screen them to only include those that are influential.

The number of covariates is also very high. Although you may decide to measure all of these covariates during the experiment, the decision to include each covariate in the statistical analysis depends on each covariate explaining some of the unaccounted for variability and meeting the assumptions outlined below. It seems unlikely that all covariates you have selected will each explain a separate amount of the unaccounted for variability – it is likely that at least two covariates will be correlated and hence explain the same source of variability. If this is the case, then only one of the pair of covariates needs to be included in the statistical model. More information on when it is appropriate to include a covariate can be found on the independent variables page of the EDA website.

Statistical analysis methods compatible with this design include a one-way repeated measures mixed model with blocking factor(s). This test assumes that the data satisfies these assumptions: residuals are normally distributed, homogeneity of variance and the outcome is measured on a continuous scale (read more about parametric and non-parametric tests).

Alternatively, a repeated measures one-way ANCOVA with blocking factor(s) could be used but note that the mixed model approach includes two benefits over the ANCOVA approach:

1. If a data point is missing for an animal the rest of the data from that animal can still be used.
2. The mixed model approach can be applied when the assumption of sphericity does not hold (condition where the variances of the estimates of the differences between all possible pairs of groups are equal).

Note that the data cannot be analysed using a standard one-way ANCOVA — a repeated measures approach should be applied because the levels of the repeated factor are not randomised within each animal and the independence of the errors assumption will probably not hold. It is also important to consider if the covariate should be used in your analysis.

Only include a covariate in your analysis if:

• The covariate is independent of the treatment.
• There is a strong relationship between the covariate and the outcome measure (i.e. either they both increase together or one increases while the other decreases).
• The relationship between the outcome measure and the covariate is similar for all treatments (i.e. there is no significant treatment by covariate interaction).

The above assumptions can be tested by plotting your data, details of what to look for and example graphs can be found on the independent variables page of the EDA website. If there are multiple covariates you are considering including in your analysis, ensure that the assumptions hold for each of them.

In many cases you will not know if including a particular covariate in your analysis is appropriate when planning your experiment. You should measure the covariate during your study, but only include it in your statistical analysis if the assumptions for covariate inclusion are met.

If you have reasons to think the data are normally distributed, and/or the variability of the responses is related to their numeric size, you should first consider transforming the data to normalise it (read more about data transformation) and assess if the transformed data fits the parametric assumptions. Most data can be normalised using transformations such as log or square root and using parametric tests is preferable as they have more statistical power than non-parametric tests, as long as the assumptions of normality are met.

When normality assumptions do not hold, even after a mathematical data transformation, a rank transformation can be applied to the data and parametric tests (in this case a repeated measures one-way ANCOVA with blocking factor(s)) can be performed on the ranked data. Note that there are assumptions associated with non-parametric tests also. For example, to perform a rank transformation the data must be able to be ranked, with only a few ties (e.g. identical values that will end up with the same rank), the observations must be independent and the covariate must have a linear relationship with the rank. Another option is categorical data analysis (e.g. categorising data as ordinal or binary outcomes and using generalized estimating equations or a generalized linear mixed effects model for the analysis). Your local statistician can help advise on this. These approaches will lead to a loss in power due to the categorisation of continuous data.

Please note that an experiment with a single variable of interest, a covariate and a blocking factor might not make the most efficient use of the data. Consider using a factorial design.

Analysis software

Software such as InVivoStat can be used to run a one-way repeated measure mixed model with blocking factors, and apply data transformations. The test can be found in the following menu:

Statistics>Additional Analyses>Paired t-test/within subject Analysis

Based on the diagram, this analysis should make a comparison between measurements at different time points within a single group, or a comparison between at least two related groups. For example, multiple treatments may be tested (in the same order) within each animal or a single treatment is administered to all animals and the animals are then assessed at multiple time points. This analysis includes a single continuous outcome measure, at least four covariates, at least one blocking factor and a single categorical factor of interest: the repeated factor.

If this description is not accurate, please check your diagram and verify that all nodes are connected properly, all variable categories are indicated and tagged to the relevant interventions or measurements and the information provided in the properties of each node is accurate; then critique it again.

The number of covariates is very high. Although you may decide to measure all of these covariates during the experiment, the decision to include each covariate in the statistical analysis depends on each covariate explaining some of the unaccounted for variability and meeting the assumptions outlined below. It seems unlikely that all covariates you have selected will each explain a separate amount of the unaccounted for variability – it is likely that at least two covariates will be correlated and hence explain the same source of variability. If this is the case, then only one of the pair of covariates needs to be included in the statistical model. More information on when it is appropriate to include a covariate can be found on the independent variables page of the EDA website.

Statistical analysis methods compatible with this design include a one-way repeated measures mixed model with blocking factor(s). This test assumes that the data satisfies these assumptions: residuals are normally distributed, homogeneity of variance and the outcome is measured on a continuous scale (read more about parametric and non-parametric tests).

Alternatively, a repeated measures one-way ANCOVA with blocking factor(s) could be used but note that the mixed model approach includes two benefits over the ANCOVA approach:

1. If a data point is missing for an animal the rest of the data from that animal can still be used.
2. The mixed model approach can be applied when the assumption of sphericity does not hold (condition where the variances of the estimates of the differences between all possible pairs of groups are equal).

Note that the data cannot be analysed using a standard one-way ANCOVA — a repeated measures approach should be applied because the levels of the repeated factor are not randomised within each animal and the independence of the errors assumption will probably not hold. It is also important to consider if the covariate should be used in your analysis.

Only include a covariate in your analysis if:

• The covariate is independent of the treatment.
• There is a strong relationship between the covariate and the outcome measure (i.e. either they both increase together or one increases while the other decreases).
• The relationship between the outcome measure and the covariate is similar for all treatments (i.e. there is no significant treatment by covariate interaction).

The above assumptions can be tested by plotting your data, details of what to look for and example graphs can be found on the independent variables page of the EDA website. If there are multiple covariates you are considering including in your analysis, ensure that the assumptions hold for each of them.

In many cases you will not know if including a particular covariate in your analysis is appropriate when planning your experiment. You should measure the covariate during your study, but only include it in your statistical analysis if the assumptions for covariate inclusion are met.

If you have reasons to think the data are normally distributed, and/or the variability of the responses is related to their numeric size, you should first consider transforming the data to normalise it (read more about data transformation) and assess if the transformed data fits the parametric assumptions. Most data can be normalised using transformations such as log or square root and using parametric tests is preferable as they have more statistical power than non-parametric tests, as long as the assumptions of normality are met.

When normality assumptions do not hold, even after a mathematical data transformation, a rank transformation can be applied to the data and parametric tests (in this case a repeated measures one-way ANCOVA with blocking factor(s)) can be performed on the ranked data. Note that there are assumptions associated with non-parametric tests also. For example, to perform a rank transformation the data must be able to be ranked, with only a few ties (e.g. identical values that will end up with the same rank), the observations must be independent and the covariate must have a linear relationship with the rank. Another option is categorical data analysis (e.g. categorising data as ordinal or binary outcomes and using generalized estimating equations or a generalized linear mixed effects model for the analysis). Your local statistician can help advise on this. These approaches will lead to a loss in power due to the categorisation of continuous data.

Please note that an experiment with a single variable of interest, a covariate and a blocking factor might not make the most efficient use of the data. Consider using a factorial design.

Analysis software

Software such as InVivoStat can be used to run a one-way repeated measure mixed model with blocking factors, and apply data transformations. The test can be found in the following menu:

Statistics>Additional Analyses>Paired t-test/within subject Analysis

Based on the diagram, this analysis should make a comparison between measurements at two or more different time points within a single group, or a comparison between three or more related groups. For example, multiple treatments may be tested (in the same order) within each animal or a single treatment is administered to all animals and the animals are then assessed at multiple time points. This analysis includes a single continuous outcome measure, at least four covariates and a single categorical factor of interest: the repeated factor.

If this description is not accurate, please check your diagram and verify that all nodes are connected properly, all variable categories are indicated and tagged to the relevant interventions or measurements and the information provided in the properties of each node is accurate; then critique it again.

The number of covariates is very high. Although you may decide to measure all of these covariates during the experiment, the decision to include each covariate in the statistical analysis depends on each covariate explaining some of the unaccounted for variability and meeting the assumptions outlined below. It seems unlikely that all covariates you have selected will each explain a separate amount of the unaccounted for variability – it is likely that at least two covariates will be correlated and hence explain the same source of variability. If this is the case, then only one of the pair of covariates needs to be included in the statistical model. More information on when it is appropriate to include a covariate can be found on the independent variables page of the EDA website.

Statistical analysis methods compatible with this design include a one-way repeated measures mixed model. This test assumes that the data satisfies these assumptions: residuals are normally distributed, homogeneity of variance, and the outcome is measured on a continuous scale (read more about parametric and non-parametric tests).

Alternatively, a repeated measures one-way ANCOVA could be used but note that the mixed model approach includes two benefits over the ANCOVA approach:

1. If a data point is missing for an animal the rest of the data from that animal can still be used.
2. The mixed model approach can be applied when the assumption of sphericity does not hold (condition where the variances of the estimates of the differences between all possible pairs of groups are equal).

Note that the data cannot be analysed using a standard one-way ANCOVA — a repeated measures approach should be applied because the levels of the repeated factor are not randomised within each animal and the independence of the errors assumption will probably not hold. It is also important to consider if the covariate should be used in your analysis.

Only include a covariate in your analysis if:

• The covariate is independent of the treatment.
• There is a strong relationship between the covariate and the outcome measure (i.e. either they both increase together or one increases while the other decreases).
• The relationship between the outcome measure and the covariate is similar for all treatments (i.e. there is no significant treatment by covariate interaction).

The above assumptions can be tested by plotting your data, details of what to look for and example graphs can be found on the independent variables page of the EDA website. If there are multiple covariates you are considering including in your analysis, ensure that the assumptions hold for each of them.

In many cases you will not know if including a particular covariate in your analysis is appropriate when planning your experiment. You should measure the covariate during your study, but only include it in your statistical analysis if the assumptions for covariate inclusion are met.

If you have reasons to think the data are not normally distributed, and/or the variability of the responses is related to their numeric size, you should first consider transforming the data to normalise it (read more about data transformation) and assess if the transformed data fits the normality assumptions. Most data can be normalised using transformations such as log or square root and using parametric tests is preferable as they have more statistical power than non-parametric tests, as long as the required assumptions are met.

When normality assumptions do not hold, even after a mathematical data transformation, a rank transformation can be applied to the data and parametric tests (in this case a repeated measures one-way ANCOVA) can be performed on the ranked data. Note that there are assumptions associated with non-parametric tests also. For example, to perform a rank transformation the data must be able to be ranked, with only a few ties (e.g. identical values that will end up with the same rank), the observations must be independent and the covariate must have a linear relationship with the rank. Another option is categorical data analysis (e.g. categorising data as ordinal or binary outcomes and using generalized estimating equations or a generalized linear mixed effects model for the analysis). Your local statistician can help advise on this. These approaches will lead to a loss in power due to the categorisation of continuous data.

Please note that an experiment with a single variable of interest and a covariate might not make the most efficient use of the data. Consider using a factorial design or taking other sources of variability into account by including blocking factors in the design.

Analysis software

Software such as InVivoStat can be used to run a one-way repeated measures mixed model, and apply data transformations. The test can be found in the following menu:

Statistics>Additional Analyses>Paired t-test/within subject Analysis

Based on the diagram, this analysis should make a comparison between two or more groups. The analysis includes a single continuous outcome measure and a single, categorical factor of interest with two or more categories. It also includes five or more blocking factors and at least four covariates. All groups are independent and if animals are repeatedly measured, a summary measure is used in this analysis.

If this description is not accurate, please check your diagram and verify that all nodes are connected properly, all variables and variable categories are indicated and tagged to the relevant interventions or measurements and the information provided in the properties of each node is accurate; then critique it again.

The number of blocking factors is very high. The decision to include blocking factors in the randomisation and the analysis depends on previous experience and/or literature evidence that nuisance variables (e.g. practical constraints on the experiment) may influence the outcome and potentially increase the variability. Blocking involves dividing the experiment into a series of mini-experiments; each mini-experiment should ideally contain all the treatments but have fewer animals per treatment group than the full design. If all treatments are present in each of the mini-experiments, then five blocking factors imply dividing the experiment into a minimum of 32 mini-experiments (5 factors with at least 2 categories each: 2⁵) and it seems unlikely to run an experiment with such a large number of animals. The best course of action would be to consult a statistician to estimate the effect of these blocking factors, and screen them to only include those that are influential.

The number of covariates is also very high. Although you may decide to measure all of these covariates during the experiment, the decision to include each covariate in the statistical analysis depends on each covariate explaining some of the unaccounted for variability and meeting the assumptions outlined below. It seems unlikely that all covariates you have selected will each explain a separate amount of the unaccounted for variability – it is likely that at least two covariates will be correlated and hence explain the same source of variability. If this is the case, then only one of the pair of covariates needs to be included in the statistical model. More information on when it is appropriate to include a covariate can be found on the independent variables page of the EDA website.

Statistical analysis methods compatible with this design include the one-way ANCOVA with blocking factor(s).

The ANCOVA approach assumes that the data satisfies these assumptions: residuals are normally distributed, homogeneity of variance, independence of the errors, and the outcome is measured on a continuous scale (read more about parametric and non-parametric tests), as well as assuming the covariate(s) should be used in the analysis. 

A covariate should be used if:

• The covariate is independent of the treatment. 
• There is a strong relationship between the covariate and the outcome measure (i.e. either they both increase together or one increases while the other decreases).
• The relationship between the outcome measure and the covariate is similar for all treatments (i.e. there is no significant treatment by covariate interaction).

The above assumptions can be tested by plotting your data, details of what to look for and example graphs can be found on the independent variables page of the EDA website. If there are multiple covariates you are considering including in your analysis, ensure that the assumptions hold for each of them.

In many cases you will not know if including a particular covariate in your analysis is appropriate when planning your experiment. You should measure the covariate during your study, but only include it in your statistical analysis if the assumptions for covariate inclusion are met.

If you have reasons to think the data are not normally distributed, and/or the variability of the responses is related to their numeric size, you should first consider transforming the data to normalise it (read more about data transformation) and assess if the transformed data fits the parametric assumptions. Most data can be normalised using transformations such as log or square root and using parametric tests is preferable as they have more statistical power than non-parametric tests, as long as the parametric assumptions are met.

When parametric assumptions do not hold, even after a mathematical data transformation, a rank transformation can be applied to the data and parametric tests (in this case a one-way ANCOVA with blocking factors) can be performed on the ranked data. Note that there are assumptions associated with non-parametric tests also. For example, to perform a rank transformation the data must be able to be ranked, with only a few ties (e.g. identical values that will end up with the same rank), the observations must be independent and the covariate must have a linear relationship with the rank. If data cannot be rank transformed (e.g. it is mainly zeros with only a few non-zero measures), the data can be recoded into binary responses and then analysed using logistic regression. Another analysis option is ordinal logistic regression. Your local statistician can help advise on this. These approaches will lead to a loss in power due to the categorisation of continuous data.

Note that a one-way ANCOVA can answer the question: is there an overall difference between the groups; they do not provide information on which individual groups differ. To identify where any differences lie, it may be necessary to carry out a post-hoc test. Note that in this case the sample size might have to increase and the experiment should be powered to detect differences in the pairwise comparisons. Note the variability estimate used in the power calculation should be adjusted to account for the effect of any covariates if you have an idea of what this could be based on previous experiments. If you do not have an estimate of the effect that the covariates will have on unexplained variability to use in your power calculation, be aware that your experiment may end up with higher power once your analysis has taken account of the variability contributed by the covariate(s).

Please note that an experiment with a single variable of interest a covariate and a blocking factor might not make the most efficient use of the data. Consider using a factorial design.

Analysis software

Software such as InVivoStat can be used to run a one-way ANCOVA with blocking factors and apply data transformations. The test can be found in the following menu:

• Statistics>Single Measure Parametric Analysis

Dose-response experiments

In dose-response experiments, drug dose is often treated as a categorical factor of interest and different doses compared to one another. There is however an alternative approach which consists in treating drug dose as a continuous factor of interest and analysing the data using for example a non-linear regression. This is arguably more relevant as the statistical analysis strategy reflects the underlying biology better. For example, just because a drug has a statistically significant effect at 10 mg/kg, does not mean that it has no effect at 9 mg/kg. A non-linear regression analysis estimates the dose-response relationship and can provide, for example, an estimate of the dose which causes 50% of the maximal effect (ED₅₀). Treating drug dose as a continuous factor of interest generally needs an increased number of different doses (generally at least five or six doses), but it allows the number of animals per dose group to be reduced, as you no longer need sufficient sample size to compare each dose back to the control group. As little as three animals per dose might be sufficient. For more information see Bate and Clark (2014).

Should you want to modify your design and treat your categorical factor of interest as a continuous factor, update the node properties of your independent variable of interest and update your diagram, and critique it again.

Based on the diagram, this analysis should make a comparison between two or more groups. The analysis includes a single continuous outcome measure and a single, categorical factor of interest with two or more categories. It also includes at least one blocking factor and at least four covariates. All groups are independent and if animals are repeatedly measured, a summary measure is used in this analysis.

If this description is not accurate, please check your diagram and verify that all nodes are connected properly, all variables and variable categories are indicated and tagged to the relevant interventions or measurements and the information provided in the properties of each node is accurate; then critique it again.

The number of covariates is very high. Although you may decide to measure all of these covariates during the experiment, the decision to include each covariate in the statistical analysis depends on each covariate explaining some of the unaccounted for variability and meeting the assumptions outlined below. It seems unlikely that all covariates you have selected will each explain a separate amount of the unaccounted for variability – it is likely that at least two covariates will be correlated and hence explain the same source of variability. If this is the case, then only one of the pair of covariates needs to be included in the statistical model. More information on when it is appropriate to include a covariate can be found on the independent variables page of the EDA website.

Statistical analysis methods compatible with this design include the one-way ANCOVA with blocking factor(s).

The ANCOVA approach assumes that the data satisfies these assumptions: residuals are normally distributed, homogeneity of variance, independence of the errors, and the outcome is measured on a continuous scale (read more about parametric and non-parametric tests), as well as assuming the covariate(s) should be used in the analysis. 

A covariate should be used if:

• The covariate is independent of the treatment. 
• There is a strong relationship between the covariate and the outcome measure (i.e. either they both increase together or one increases while the other decreases).
• The relationship between the outcome measure and the covariate is similar for all treatments (i.e. there is no significant treatment by covariate interaction).

The above assumptions can be tested by plotting your data, details of what to look for and example graphs can be found on the independent variables page of the EDA website. If there are multiple covariates you are considering including in your analysis, ensure that the assumptions hold for each of them.

In many cases you will not know if including a particular covariate in your analysis is appropriate when planning your experiment. You should measure the covariate during your study, but only include it in your statistical analysis if the assumptions for covariate inclusion are met.

If you have reasons to think the data are not normally distributed, and/or the variability of the responses is related to their numeric size, you should first consider transforming the data to normalise it (read more about data transformation) and assess if the transformed data fits the parametric assumptions. Most data can be normalised using transformations such as log or square root and using parametric tests is preferable as they have more statistical power than non-parametric tests, as long as the parametric assumptions are met.

When parametric assumptions do not hold, even after a mathematical data transformation, a rank transformation can be applied to the data and parametric tests (in this case a one-way ANCOVA with blocking factors) can be performed on the ranked data. Note that there are assumptions associated with non-parametric tests also. For example, to perform a rank transformation the data must be able to be ranked, with only a few ties (e.g. identical values that will end up with the same rank), the observations must be independent and the covariate must have a linear relationship with the rank. If data cannot be rank transformed (e.g. it is mainly zeros with only a few non-zero measures), the data can be recoded into binary responses and then analysed using logistic regression. Another analysis option is ordinal logistic regression. Your local statistician can help advise on this. These approaches will lead to a loss in power due to the categorisation of continuous data.

Note that a one-way ANCOVA can answer the question: is there an overall difference between the groups; they do not provide information on which individual groups differ. To identify where any differences lie, it may be necessary to carry out a post-hoc test. Note that in this case the sample size might have to increase and the experiment should be powered to detect differences in the pairwise comparisons. Note the variability estimate used in the power calculation should be adjusted to account for the effect of any covariates if you have an idea of what this could be based on previous experiments. If you do not have an estimate of the effect that the covariates will have on unexplained variability to use in your power calculation, be aware that your experiment may end up with higher power once your analysis has taken account of the variability contributed by the covariate(s).

Please note that an experiment with a single variable of interest a covariate and a blocking factor might not make the most efficient use of the data. Consider using a factorial design.

Analysis software

Software such as InVivoStat can be used to run a one-way ANCOVA with blocking factors and apply data transformations. The test can be found in the following menu:

• Statistics>Single Measure Parametric Analysis

Dose-response experiments

In dose-response experiments, drug dose is often treated as a categorical factor of interest and different doses compared to one another. There is however an alternative approach which consists in treating drug dose as a continuous factor of interest and analysing the data using for example a non-linear regression. This is arguably more relevant as the statistical analysis strategy reflects the underlying biology better. For example, just because a drug has a statistically significant effect at 10 mg/kg, does not mean that it has no effect at 9 mg/kg. A non-linear regression analysis estimates the dose-response relationship and can provide, for example, an estimate of the dose which causes 50% of the maximal effect (ED₅₀). Treating drug dose as a continuous factor of interest generally needs an increased number of different doses (generally at least five or six doses), but it allows the number of animals per dose group to be reduced, as you no longer need sufficient sample size to compare each dose back to the control group. As little as three animals per dose might be sufficient. For more information see Bate and Clark (2014).

Should you want to modify your design and treat your categorical factor of interest as a continuous factor, update the node properties of your independent variable of interest and update your diagram, and critique it again.

Based on the diagram, this analysis should make a comparison between two or more groups. The analysis includes a single continuous outcome measure a single, categorical factor of interest, with at least two categories and at least four covariates. All groups are independent and if experimental units are repeatedly measured, a summary measure is used in this analysis.

If this description is not accurate, please check your diagram and verify that all nodes are connected properly, all variables and variable categories are indicated and tagged to the relevant interventions or measurements and the information provided in the properties of each node is accurate; then critique it again.

The number of covariates is very high. Although you may decide to measure all of these covariates during the experiment, the decision to include each covariate in the statistical analysis depends on each covariate explaining some of the unaccounted for variability and meeting the assumptions outlined below. It seems unlikely that all covariates you have selected will each explain a separate amount of the unaccounted for variability – it is likely that at least two covariates will be correlated and hence explain the same source of variability. If this is the case, then only one of the pair of covariates needs to be included in the statistical model. More information on when it is appropriate to include a covariate can be found on the independent variables page of the EDA website.

Statistical analysis methods compatible with this design include a one-way ANCOVA and an ANCOVA on the rank transformed outcome measure.

The ANCOVA approach assumes that the data satisfies these assumptions: residuals are normally distributed, homogeneity of variance, independence of the errors, and the outcome is measured on a continuous scale (read more about parametric and non-parametric tests), as well as assuming the covariate(s) should be used in the analysis.

A covariate should be used if:

• The covariate is independent of the treatment.
• There is a strong relationship between the covariate and the outcome measure (i.e. either they both increase together or one increases while the other decreases).
• The relationship between the outcome measure and the covariate is similar for all treatments (i.e. there is no significant treatment by covariate interaction).

The above assumptions can be tested by plotting your data, details of what to look for and example graphs can be found on the independent variables page of the EDA website. If there are multiple covariates you are considering including in your analysis, ensure that the assumptions hold for each of them.

In many cases you will not know if including a particular covariate in your analysis is appropriate when planning your experiment. You should measure the covariate during your study, but only include it in your statistical analysis if the assumptions for covariate inclusion are met.

If you have reasons to think the data are not normally distributed, and/or the variability of the responses is related to their numeric size, you should first consider transforming the data to normalise it (read more about data transformation) and assess if the transformed data fits the normality assumptions. Most data can be normalised using transformations such as log or square root, and using parametric tests is preferable as they have more statistical power than non-parametric tests, as long as the required assumptions are met.

When normality assumptions do not hold, even after a mathematical data transformation, a rank transformation can be applied to the data and parametric tests (in this case a one-way ANCOVA) can be performed on the ranked data. Note that there are assumptions associated with non-parametric tests also. For example, to perform a rank transformation the data must be able to be ranked, with only a few ties (e.g. identical values that will end up with the same rank), the observations must be independent and the covariate must have a linear relationship with the rank. If data cannot be rank transformed (e.g. it is mainly zeros with only a few non-zero measures), the data can be recoded into binary responses and then analysed using logistic regression. Another analysis option is ordinal logistic regression. Your local statistician can help advise on this. These approaches will lead to a loss in power due to the categorisation of continuous data.

Note that a one-way ANCOVA or an ANCOVA on the rank transformed response can answer the question: is there an overall difference between the groups; they do not provide information on which individual groups differ. To identify where any differences lie, it may be necessary to carry out a post-hoc test. Note that in this case the sample size might have to increase and the experiment should be powered to detect differences in the pairwise comparisons. Note the variability estimate used in the power calculation should be adjusted to account for the effect of any covariates if you have an idea of what this could be based on previous experiments. If you do not have an estimate of the effect that the covariates will have on unexplained variability to include in your power calculation, be aware that your experiment may end up with higher power once your analysis has taken account of the variability contributes by the covariate(s).

Please note that an experiment with a single variable of interest and a covariate might not make the most efficient use of the data. Consider using a factorial design or taking other sources of variability into account by including blocking factors in the design.

Analysis software

Software such as InVivoStat can be used to run a one-way ANCOVA, and apply data transformations (including a rank transformation). The test can be found in the following menu:

• Statistics>Single Measure Parametric Analysis

Dose-response experiments

In dose-response experiments, drug dose is often treated as a categorical factor of interest and different doses compared to one another. There is however an alternative approach which consists in treating drug dose as a continuous factor of interest and analysing the data using for example a non-linear regression. This is arguably more relevant as the statistical analysis strategy reflects the underlying biology better. For example, just because a drug has a statistically significant effect at 10 mg/kg, does not mean that it has no effect at 9 mg/kg. A non-linear regression analysis estimates the dose-response relationship and can provide, for example, an estimate of the dose which causes 50% of the maximal effect (ED₅₀). Treating drug dose as a continuous factor of interest generally needs an increased number of different doses (generally at least five or six doses), but it allows the number of animals per dose group to be reduced, as you no longer need sufficient sample size to compare each dose back to the control group. As little as three animals per dose might be sufficient. For more information see Bate and Clark (2014).

Should you want to modify your design and treat your categorical factor of interest as a continuous factor, update the node properties of your independent variable of interest and update your diagram, and critique it again.

Based on the diagram, this experiment includes a single continuous outcome measure and at least five factors of interest which are all categorical, one is also a repeated factor. The levels of the repeated factor are not randomised, unlike the levels of the other factors of interest and this has implications on the choice of statistical analysis. The analysis also includes at least one covariate and at least five blocking factors.

If this description is not accurate, please check your diagram and verify that all nodes are connected properly, all variable categories are indicated and tagged to the relevant interventions or measurements and the information provided in the properties of each node is accurate; then critique it again.

The number of factors of interest is very high. With five factors, a full factorial design would need at least 32 groups (5 factors with at least 2 categories each: 2⁵) and it seems unlikely that the experiment will be large enough (in terms of numbers of animals) to estimate all the interactions reliably. For example, with five factors, there are ten possible two-way interactions. In practice three-way (and above) interactions are unlikely to occur and you should consider reducing the number of factors of interest. Alternatively you could manually amend the statistical model to reduce the number of higher-order interactions included in the model; in this case the best course of action would be to consult a statistician.

The number of blocking factors is also very high. The decision to include blocking factors in the randomisation and the analysis depends on previous experience and/or literature evidence that nuisance variables (e.g. practical constraints on the experiment) may influence the outcome and potentially increase the variability. Blocking involves dividing the experiment into a series of mini-experiments; each mini-experiment should ideally contain all the treatments but have fewer animals per treatment group than the full design. If all treatments are present in each of the mini-experiments, then five blocking factors imply dividing the experiment into a minimum of 32 mini-experiments (5 factors with at least 2 categories each: 2⁵) and it seems unlikely to run an experiment with such a large number of animals. The best course of action would be to consult a statistician to estimate the effect of these blocking factors, and screen them to only include those that are influential.

Statistical analysis methods compatible with this design include two-way, three-way or four-way repeated measures mixed model with blocking factor(s), depending on the total number of factors of interest. This test assumes that the data satisfies these assumptions: residuals are normally distributed, homogeneity of variance and the outcome is measured on a continuous scale (read more about parametric and non-parametric tests).

Alternatively, a repeated measures factorial ANCOVA with blocking factor(s) could be used but note that the mixed model approach includes two benefits over the ANCOVA approach:

1. If a data point is missing for an animal the rest of the data from that animal can still be used.
2. The mixed model approach can be applied when the assumption of sphericity does not hold (condition where the variances of the estimates of the differences between all possible pairs of groups are equal).

Note that the data cannot be analysed using a standard factorial ANCOVA — a repeated measure approach should be applied because the levels of the repeated factor are not randomised within each animal and the independence of the errors assumption will probably not hold. It is also important to consider if the covariate should be used in your analysis.

A covariate should be used if:

• The covariate is independent of the treatment.
• There is a strong relationship between the covariate and the outcome measure (i.e. either they both increase together or one increases while the other decreases).
• The relationship between the outcome measure and the covariate is similar for all treatments (i.e. there is no significant treatment by covariate interaction).

The above assumptions can be tested by plotting your data, details of what to look for and example graphs can be found on the independent variables page of the EDA website. If there are multiple covariates you are considering including in your analysis, ensure that the assumptions hold for each of them.

In many cases you will not know if including a particular covariate in your analysis is appropriate when planning your experiment. You should measure the covariate during your study, but only include it in your statistical analysis if the assumptions for covariate inclusion are met.

If you have reasons to think the data are not normally distributed, and/or the variability of the responses is related to their numeric size, you should first consider transforming the data to normalise it (read more about data transformation) and assess if the transformed data satisfies the parametric assumptions. Most data can be normalised using transformations such as log or square root and using parametric tests is preferable as they have more statistical power than non-parametric tests, as long as the parametric assumptions are met.

When normality assumptions do not hold, even after a mathematical data transformation, a rank transformation can be applied to the data and parametric tests (in this case a repeated measures factorial ANCOVA with blocking factor(s)) can be performed on the ranked data. Note that there are assumptions associated with non-parametric tests also. For example, to perform a rank transformation the data must be able to be ranked, with only a few ties (e.g. identical values that will end up with the same rank), the observations must be independent and the covariate must have a linear relationship with the rank. Another option is categorical data analysis (e.g. categorising data as ordinal or binary outcomes and using generalized estimating equations or a generalized linear mixed effects model for the analysis). Your local statistician can help advise on this. These approaches will lead to a loss in power due to the categorisation of continuous data.

Interpreting a factorial ANOVA

There is a common mistake when interpreting the results of a factorial ANCOVA. For example, a study of the effect of exercise on performance of a behavioural task is investigated in animals of both sexes with baseline locomotor activity as a covariate. A claim that the overall effect of exercise is different in males and females can only be supported by the finding that the interaction between the two factors is statistically significant (i.e. the size of the effect is different in males and females). A significant effect of exercise in one sex but not the other is not appropriate to support the claim that there is a difference between sexes.

Analysis software

Software such as InVivoStat can be used to run a factorial repeated measures mixed model with blocking factor(s), and apply data transformations. The test can be found in the following menus:

Statistics>Repeated Measures Parametric Analysis

References and further reading

Nieuwenhuis, S, Forstmann, BU and Wagenmakers, EJ (2011). Erroneous analyses of interactions in neuroscience: a problem of significance. Nat Neurosci 14(9):1105-7. doi: 10.1038/nn.2886

https://www.graphpad.com/guides/prism/latest/statistics/how_to_think_about_results_from_two-way_anova.htm

Based on the diagram, this experiment includes a single continuous outcome measure and at least two factors of interest which are all categorical, one is also a repeated factor. The levels of the repeated factor are not randomised, unlike the levels of the other factors of interest and this has implications on the choice of statistical analysis. The analysis also includes at least one covariate and at least five blocking factors.

If this description is not accurate, please check your diagram and verify that all nodes are connected properly, all variable categories are indicated and tagged to the relevant interventions or measurements and the information provided in the properties of each node is accurate; then critique it again.

The number of blocking factors is very high. The decision to include blocking factors in the randomisation and the analysis depends on previous experience and/or literature evidence that nuisance variables (e.g. practical constraints on the experiment) may influence the outcome and potentially increase the variability. Blocking involves dividing the experiment into a series of mini-experiments; each mini-experiment should ideally contain all the treatments but have fewer animals per treatment group than the full design. If all treatments are present in each of the mini-experiments, then five blocking factors imply dividing the experiment into a minimum of 32 mini-experiments (5 factors with at least 2 categories each: 2⁵) and it seems unlikely to run an experiment with such a large number of animals. The best course of action would be to consult a statistician to estimate the effect of these blocking factors, and screen them to only include those that are influential.

Statistical analysis methods compatible with this design include two-way, three-way or four-way repeated measures mixed model with blocking factor(s), depending on the total number of factors of interest. This test assumes that the data satisfies these assumptions: residuals are normally distributed, homogeneity of variance and the outcome is measured on a continuous scale (read more about parametric and non-parametric tests).

Alternatively, a repeated measures factorial ANCOVA with blocking factor(s) could be used but note that the mixed model approach includes two benefits over the ANCOVA approach:

1. If a data point is missing for an animal the rest of the data from that animal can still be used.
2. The mixed model approach can be applied when the assumption of sphericity does not hold (condition where the variances of the estimates of the differences between all possible pairs of groups are equal).

Note that the data cannot be analysed using a standard factorial ANCOVA — a repeated measure approach should be applied because the levels of the repeated factor are not randomised within each animal and the independence of the errors assumption will probably not hold. It is also important to consider if the covariate should be used in your analysis.

A covariate should be used if:

• The covariate is independent of the treatment.
• There is a strong relationship between the covariate and the outcome measure (i.e. either they both increase together or one increases while the other decreases).
• The relationship between the outcome measure and the covariate is similar for all treatments (i.e. there is no significant treatment by covariate interaction).

The above assumptions can be tested by plotting your data, details of what to look for and example graphs can be found on the independent variables page of the EDA website. If there are multiple covariates you are considering including in your analysis, ensure that the assumptions hold for each of them.

In many cases you will not know if including a particular covariate in your analysis is appropriate when planning your experiment. You should measure the covariate during your study, but only include it in your statistical analysis if the assumptions for covariate inclusion are met.

If you have reasons to think the data are not normally distributed, and/or the variability of the responses is related to their numeric size, you should first consider transforming the data to normalise it (read more about data transformation) and assess if the transformed data satisfies the parametric assumptions. Most data can be normalised using transformations such as log or square root and using parametric tests is preferable as they have more statistical power than non-parametric tests, as long as the parametric assumptions are met.

When normality assumptions do not hold, even after a mathematical data transformation, a rank transformation can be applied to the data and parametric tests (in this case a repeated measures factorial ANCOVA with blocking factor(s)) can be performed on the ranked data. Note that there are assumptions associated with non-parametric tests also. For example, to perform a rank transformation the data must be able to be ranked, with only a few ties (e.g. identical values that will end up with the same rank), the observations must be independent and the covariate must have a linear relationship with the rank. Another option is categorical data analysis (e.g. categorising data as ordinal or binary outcomes and using generalized estimating equations or a generalized linear mixed effects model for the analysis). Your local statistician can help advise on this. These approaches will lead to a loss in power due to the categorisation of continuous data.

Interpreting a factorial ANOVA

There is a common mistake when interpreting the results of a factorial ANCOVA. For example, a study of the effect of exercise on performance of a behavioural task is investigated in animals of both sexes with baseline locomotor activity as a covariate. A claim that the overall effect of exercise is different in males and females can only be supported by the finding that the interaction between the two factors is statistically significant (i.e. the size of the effect is different in males and females). A significant effect of exercise in one sex but not the other is not appropriate to support the claim that there is a difference between sexes.

Analysis software

Software such as InVivoStat can be used to run a factorial repeated measures mixed model with blocking factor(s), and apply data transformations. The test can be found in the following menus:

Statistics>Repeated Measures Parametric Analysis

References and further reading

Nieuwenhuis, S, Forstmann, BU and Wagenmakers, EJ (2011). Erroneous analyses of interactions in neuroscience: a problem of significance. Nat Neurosci 14(9):1105-7. doi: 10.1038/nn.2886

https://www.graphpad.com/guides/prism/latest/statistics/how_to_think_about_results_from_two-way_anova.htm