A free resource from the NC3Rs used by over 18,000 researchers worldwide to help design robust experiments more likely to yield reliable and reproducible results.
The EDA helps you build a diagram representing your experimental plan, which can be critiqued by the system to provide bespoke feedback.
The EDA also:
The EDA website provides information about experimental design concepts, and how to apply these in your experiments.
THE EXPERIMENTAL DESIGN ASSISTANT
Helping researchers worldwide design robust and reliable experiments
This variable category node has been used in two contradictory ways in your diagram. The diagram currently suggests that this variable category is both distributed evenly between the groups shown on the diagram and that this variable category is allocated to specific groups.
This variable category node is connected to an independent variable of interest node which is connected to the allocation node using ‘categories allocated separately’. This means all variable categories connected to this independent variable of interest node will be distributed evenly between the groups directly following the allocation (e.g. if the variable category nodes are ‘male’ and ‘female’ all groups will have both sexes and have the same number of each sex).
You have also put the variable category nodes on either group or intervention nodes in the diagram. This means that you want this variable category to only be in the group(s) (or following intervention nodes) labelled with this variable category node.
See below for different options for your diagram depending on your experiment.
If you want the categories of the independent variable of interest to be distributed evenly between the groups shown on your diagram (e.g. you want each group to have the same proportion of male and female animals in them), leave the independent variable of interest node connected to both the allocation node and the analysis nodes. The variable category nodes from this independent variable of interest should only be connected to the independent variable of interest, delete any that are tagged to group or intervention nodes (see images below).
This approach is only appropriate for animal characteristics (e.g. sex, strain or age).
The above experiment should be represented like this:
If you want each category of the independent variable of interest to be allocated to a different group on your diagram, delete the link connecting the independent variable of interest node to the allocation node. Leave the independent variable of interest node connected to the analysis node and make sure the variable category nodes are on intervention nodes (not on group nodes). You may need to add more group and intervention nodes to accurately represent your experiment (see images below).
This is appropriate for interventions that can be allocated to animals such as a drug treatment, or a surgical intervention.
The experiment above should be represented like this:
This issue may cause other prompts to be triggered inappropriately, please disregard other warnings. Once you have modified the diagram, please critique it again.
In the properties of this nuisance variable, how variable is accounted for is set to ‘used as a covariate’ but it is connected to an analysis node with the link ‘is blocking factor for’. Using this link indicates that the nuisance variable will be taken account of in the analysis as a blocking factor. It is possible to use a nuisance variable as a blocking factor in the random allocation and as a covariate in the analysis. In order for the system to treat this nuisance variable as a covariate in the analysis it should be connected to the analysis node with ‘is covariate for’ as this information is used to generate a recommendation regarding appropriate methods of analysis.
Including a covariate in the analysis allows the variability associated with a continuous nuisance variable to be accounted for. This may be done if experimental units differ due to the influence of a continuous numerical value that is not readily controllable. Examples of independent variables that can be used as covariates include a pre-treatment measure of the response of interest, baseline body weight or age of the animal or baseline locomotor activity.
The nuisance variable can be included in the analysis either as a blocking factor or as a covariate. There are a few reasons for choosing one over the other:
More information about covariates can be found in the Independent variables section of the EDA website.
To indicate that a nuisance variable is used as a covariate in the statistical analysis, it should be connected to the analysis node, with a link ‘is_covariate_for’. Note that some variables can be considered as either continuous or categorical. For example, body weight can be considered categorical (categories: 'low weight' and 'high weight') or continuous, thus body weight could be treated as categorical and included as a blocking factor in the randomisation, and then treated as continuous and included as a covariate in the analysis. If the nuisance variable is used as a blocking factor in the randomisation and as a covariate in the analysis, it should be connected to the allocation node with a link ‘is_blocking_factor_for’ and the analysis with a link 'is_covariate_for'. In the properties of a nuisance variable, the fields 'continuous or categorical' and 'how is variable accounted for' relate to the analysis. For example, if a nuisance variable ‘body weight’ is included as a blocking factor in the allocation (i.e. animals in the 'low weight' and 'high weight' categories are randomised into treatment groups separately) and as a covariate in the analysis, then in the properties of this nuisance variable, ‘continuous or categorical’ should be set to ‘continuous or discrete’.
The link to the analysis node can be altered by clicking on the link, this will make a red spanner icon appear. Click on the spanner icon to display the dropdown menu allowing you to select the 'is_covariate_for' link.
The variability associated with a categorical nuisance variable can be accounted for by including it as a blocking factor in the statistical analysis.
Including a blocking factor in the randomisation is a way to ensure that treatment comparisons are not influenced by known nuisance effects. Including this blocking factor in the statistical analysis allows the variability induced by that nuisance variable to be taken into account. For example, if all the results are higher on a Monday than a Tuesday, and each treatment is allocated to animals on both days, then the within-group random variability will be inflated by any overall day to day differences. Including Day as a blocking factor in the analysis allows this nuisance source of variability to be taken into account and hence prevent it inflating the underlying estimate of the variability of the data. This increases the precision of the effect of treatment, thus increasing the ability to detect a real effect with fewer experimental units.
If the nuisance variable is going to be treated as a blocking factor in the analysis ‘how variable is accounted for’ should be updated to ‘blocked’ as shown in the image below.
If the nuisance variable is going to be treated as a blocking factor in the analysis it should be treated as categorical in the analysis also. This can be indicated in the properties of its node as shown in the image below.
In some instances a baseline measure, that is used to generate the levels of the blocking factor, can also be used to transform the data. For example, baseline bodyweight can be used as a blocking factor in the randomisation (where the blocking factor categories are low, medium and high) and animals are assigned to blocks based on their actual baseline bodyweight. The actual baseline bodyweight can also be used to transform the outcome measure - expressed as percentage of the baseline bodyweight. In this case it is not crucial to include the baseline body weight as a blocking factor or covariate in the analysis. Instead a data transformation node should be added to the diagram as shown in the image below.
This issue may cause other prompts to be triggered inappropriately, please disregard other warnings. Once you have modified the diagram, please critique it again.
In the properties of this nuisance variable, how variable is accounted for is set to ‘used as a covariate’ but it is connected to an analysis node with the link ‘causes variation to’. Using this link indicates that it is not possible to control for this nuisance variable (i.e. it is uncontrolled). If you want the system to treat this nuisance variable as a covariate in the analysis it should be connected to the analysis node with ‘is covariate for’ as this information is used to generate a recommendation regarding appropriate methods of analysis.
Including a covariate in the analysis allows the variability associated with a continuous nuisance variable to be accounted for. This may be done if experimental units differ due to the influence of a continuous numerical value that is not readily controllable. Examples of independent variables that can be used as covariates include a pre-treatment measure of the response of interest, baseline body weight or age of the animal or baseline locomotor activity.
The nuisance variable can be included in the analysis either as a blocking factor or as a covariate. There are a few reasons for choosing one over the other:
More information about covariates can be found in the Independent variables section of the EDA website.
To indicate that a nuisance variable is used as a covariate in the statistical analysis, it should be connected to the analysis node, with a link ‘is_covariate_for’. Note that some variables can be considered as either continuous or categorical. For example, body weight can be considered categorical (categories: 'low weight' and 'high weight') or continuous, thus body weight could be treated as categorical and included as a blocking factor in the randomisation, and then treated as continuous and included as a covariate in the analysis. If the nuisance variable is used as a blocking factor in the randomisation and as a covariate in the analysis, it should be connected to the allocation node with a link ‘is_blocking_factor_for’ and the analysis with a link 'is_covariate_for'. In the properties of a nuisance variable, the fields 'continuous or categorical' and 'how is variable accounted for' relate to the analysis. For example, if a nuisance variable ‘body weight’ is included as a blocking factor in the allocation (i.e. animals in the 'low weight' and 'high weight' categories are randomised into treatment groups separately) and as a covariate in the analysis, then in the properties of this nuisance variable, ‘continuous or categorical’ should be set to ‘continuous or discrete’.
The analysis suggestion generated by the EDA will only take into account the nuisance variables that are connected to the analysis node. The link to the analysis node can be added by selecting the analysis node icon from the nuisance variable's next node menu and drag and drop it onto the existing analysis node on the diagram. Select the red ‘unlinked’ link created and click on the spanner icon to display the dropdown menu allowing you to select the appropriate link as shown as the image below.
In some instances a baseline measure, that is used to generate the levels of the blocking factor, can also be used to transform the data. For example, baseline bodyweight can be used as a blocking factor in the randomisation (where the blocking factor categories are low, medium and high) and animals are assigned to blocks based on their actual baseline bodyweight. The actual baseline bodyweight can also be used to transform the outcome measure - expressed as percentage of the baseline bodyweight. In this case it is not crucial to include the baseline body weight as a blocking factor or covariate in the analysis. Instead a data transformation node should be added to the diagram as shown in the image below.
Once you have modified the diagram, please critique it again.
In the properties of this nuisance variable, how variable is accounted for is set to ‘used as a covariate’ but this nuisance variable is not connected to an analysis node.
If you intend to use this nuisance variable as a blocking factor, ignore this message, open the properties of this node and change the 'how is variable accounted for' field to 'blocked' and critique again.
Including a covariate in the analysis allows the variability associated with a continuous nuisance variable to be accounted for. This may be done if experimental units differ due to the influence of a continuous numerical value that is not readily controllable. Examples of independent variables that can be used as covariates include a pre-treatment measure of the response of interest, baseline body weight or age of the animal or baseline locomotor activity.
The nuisance variable can be included in the analysis either as a blocking factor or as a covariate. There are a few reasons for choosing one over the other:
More information about covariates can be found in the Independent variables section of the EDA website.
To indicate that a nuisance variable is used as a covariate in the statistical analysis, it should be connected to the analysis node, with a link ‘is_covariate_for’. Note that some variables can be considered as either continuous or categorical. For example, body weight can be considered categorical (categories: 'low weight' and 'high weight') or continuous, thus body weight could be treated as categorical and included as a blocking factor in the randomisation, and then treated as continuous and included as a covariate in the analysis. If the nuisance variable is used as a blocking factor in the randomisation and as a covariate in the analysis, it should be connected to the allocation node with a link ‘is_blocking_factor_for’ and the analysis with a link 'is_covariate_for'. In the properties of a nuisance variable, the fields 'continuous or categorical' and 'how is variable accounted for' relate to the analysis. For example, if a nuisance variable ‘body weight’ is included as a blocking factor in the allocation (i.e. animals in the 'low weight' and 'high weight' categories are randomised into treatment groups separately) and as a covariate in the analysis, then in the properties of this nuisance variable, ‘continuous or categorical’ should be set to ‘continuous or discrete’.
The analysis suggestion generated by the EDA will only take into account the nuisance variables that are connected to the analysis node. The link to the analysis node can be added by selecting the analysis node icon from the nuisance variable's next node menu and drag and drop it onto the existing analysis node on the diagram. Select the red ‘unlinked’ link created and click on the spanner icon to display the dropdown menu allowing you to select the appropriate link as shown as the image below.
In some instances a baseline measure that is used to generate the levels of the blocking factor, can also be used to transform the data. For example, baseline bodyweight can be used as a blocking factor in the randomisation (where the blocking factor categories are low, medium and high) and animals are assigned to blocks based on their actual baseline bodyweight. The actual baseline bodyweight can also be used to transform the outcome measure - expressed as percentage of the baseline bodyweight. In this case it is not crucial to include the baseline body weight as a blocking factor or covariate in the analysis. Instead a data transformation node should be added to the diagram as shown in the image below.
Once you have modified the diagram, please critique it again.
You have connected your independent variable of interest to your allocation node and to an analysis node. This suggests that you want the categories of this independent variable of interest distributed evenly between the groups shown on your diagram. This is only appropriate for variables defining an animal characteristic (e.g. strain or sex). By including it as a factor in the analysis, you also want to investigate whether the effect of the intervention depends on that animal characteristic.
If this independent variable is not an animal characteristic, disconnect it from the allocation node, and ensure that all experimental groups resulting from the allocation are represented explicitly on your diagram. You should then disregard the rest of this prompt and critique your diagram again to get updated feedback.
For the purposes of your sample size calculation, you need to consider that there are more groups in your experiment than those shown on your diagram. For example, if the independent variable of interest is sex, with two categories: male and female, the actual number of groups in the experiment is twice the number of groups on the diagram because each group node on the diagram represents a group of males and a group of females.
The example below is an experiment testing the effect of two doses of a drug. There are three groups (vehicle, low dose and high dose) and the independent variable of interest that is being randomised to these groups separately is sex and has two categories (male and female. This means that you essentially have six groups in the experiment (vehicle males, vehicle females, low dose males, low dose females, high dose males, high dose females) but only three are represented on the diagram for simplicity.
When calculating the sample size for the example shown above, the resulting n per group is for each of the six groups (e.g. the three female groups and the three male groups in your experiment). The power calculator in the EDA does not automatically take this into account. Therefore on your diagram, in the properties of the group node, the ‘planned number of experimental units’ field the number should be doubled as the groups on your diagrams represent male and female groups combined.
In the image of the power calculator below the n per group calculated is 5 (outlined in red in the image below).
In the group node properties the ‘planned number of experimental units’ (outlined in red in the image below) should be double the n per group calculated as each group node represents the male and female groups combined. You can use the ‘additional information’ field to clarify that the group is half male and half female.
The same principle applies to other animal characteristics, for example if you are testing three different strains, each group on the diagram represents three different single-strain groups and the n per group will need to be multiplied by three. If you are testing an animal characteristic with x different categories (e.g. each category being a different genotype), the n per group from your power calculation should be multiplied by x (the number of different genotypes) when entered into the group node properties.
If the sample size for your experiment means the experiment has to be performed in batches it is important that each batch has all experimental groups in it. In this case you would treat the batch as a blocking factor in both the randomisation and the analysis to take account of any variability introduced by the different times the batches were run.
This approach will enable you to investigate whether the effect of the experimental intervention depends on this animal characteristic and to test the effect of the intervention in, for example, males and females separately, and compare the responses between sexes.
However, if the purpose of your experiment is to determine the overall effect of an intervention, and the animal characteristic is not of direct interest, it might be appropriate to use the animal characteristic as a blocking factor (i.e. a nuisance variable), which would enable you to reduce the overall number of animals in your experiment.
To do this, delete the independent variable of interest node and add a nuisance variable node, connect the nuisance variable node to the variable category nodes (i.e. the variable category nodes that were connected to the independent variable of interest node, e.g. male and female). The nuisance variable node should be connected to both the allocation and the analysis nodes with the link ‘is blocking factor for’ as shown in the image below.
If you modify the diagram, please critique it again.
An independent variable of interest node represents a parameter specifically manipulated to test your hypothesis. You have connected your independent variable of interest to your allocation node but not to an analysis node. Connecting your independent variable of interest node to the allocation node suggests that you want the categories of this independent variable of interest distributed evenly between the groups shown on your diagram. This is only appropriate for variables defining an animal characteristic (e.g. strain or sex). In order to use this variable to test your hypothesis (e.g. to investigate whether the effect of the intervention depends on that animal characteristic) it must be a factor of interest in your analysis, this means it should be connected to an analysis node.
If this node represents something that you want to randomise separately (e.g. sex or strain) but is not a part of your hypothesis, you should represent it as a blocking factor, using a nuisance variable node.
If this independent variable is not an animal characteristic but is something you are testing as part of your hypothesis, disconnect it from the allocation node, and ensure that all experimental groups resulting from the allocation are represented explicitly on your diagram. You should then disregard the rest of this prompt and critique your diagram again to get updated feedback.
If you do wish to use this animal characteristic as an independent variable of interest in your analysis, then you must connect it to your analysis node as well as the allocation node. By including your independent variable of interest as a factor in the analysis, you are indicating that you also want to investigate whether the effect of the intervention depends on that animal characteristic.
If this node represents an animal characteristic that you want to randomise separately (e.g. sex or strain) but you are not interested in the effects of this on your outcome measure, you should represent it as a blocking factor. To do this, delete the independent variable of interest node and add a nuisance variable node and connect it to the variable category nodes that the independent variable of interest was connected to. Then connect the nuisance variable node to both the allocation and the analysis nodes using the link ‘is blocking factor for’.
By including this as a blocking factor in your allocation you are ensuring that this factor is evenly distributed between your groups. By including the blocking factor in your analysis you are taking account of the variability introduced by this factor, increasing the chances of detecting the effect of the intervention you are interested in (your independent variable of interest).
If you modify the diagram, please critique it again.
