Selection limits

A selection limit is a term from animal breeding and quantitative genetics that refers to a cessation of progress even when continued directional selection is being applied to a trait, such as body size. In other words, a breeder or scientist is using selective breeding (artificial selection) and choosing individuals as breeders within a population based on some phenotypic trait or traits. If this is done, then the average value of the population typically evolves across generations in the direction being favored by selection (i.e., for higher or lower values of the trait), but then at some point the population stops evolving. The trait under selection is then said to have reached a limit or plateau at that value.

The existence of limits in artificial selection experiments was discussed in the scientific literature in the 1940s or earlier.^[1] The most obvious possible cause of reaching a limit (or plateau) when a population is under continued directional selection is that all of the additive-genetic variation (see additive genetic effects) related to that trait gets "used up" or fixed.^[2] For example, if a trait, such as body mass, is under selection to increase, then, over time (i.e., across generations), the alleles (genetic variants) at all loci (most simply, positions on chromosomes) that tend to make individuals larger than average will increase in frequency, while those that tend to make an individual smaller than average will decrease in frequency. Eventually, in principle, the favored alleles at all relevant loci will become the only ones remaining at those loci. In reality, mutation, random genetic drift (especially in small populations), and gene flow from immigrants may stop some loci from becoming fixed for the "good" alleles.

However, other factors may interfere with the realization of genetic gains before loss of genetic variation causes a selection limit. As noted by Lerner and Dempster,^[1] these factors are generally one of two types: 1) negative relations with Darwinian fitness; 2) non-additive gene action and/or genotype-environment interaction (although others are possible^{[3] [4] [2] [5]}).

A negative relation with Darwinian fitness is a situation in which an allele that is "good" for the trait under directional selection is "bad" with respect to lifetime reproductive success. For example, an allele that tends to confer larger body size might also lead to infertility, thus reducing the ability of individuals with that allele to produce offspring, limiting further response to selection, and sometimes even leading to extinction of the selected line.^[6]

Non-additive gene action refers to such situations as heterozygote advantage, where heterozygous individuals have higher (or lower) values for a trait (such as body size) than do either of the two homozygotes. In such a case, selection will tend to maintain more than one allele in the population, and a selection limit may be reached while additive-genetic variation (narrow-sense heritability) remains for the trait under directional selection.

Genotype-environment interaction occurs when the phenotype produced by a particular set of alleles (at one or more loci) confers relatively higher or lower values of a trait depending on the environmental circumstances in which an individual is born or raised, or under which the trait is measured. For instance, somewhat different genes (a term that can refer to alleles or loci) tend to give the highest value of a trait depending on the season. If this occurs, then directional selection will act to favor some genes in winter and others in summer, for example. Again, the result may be that a selection plateau is attained while the population retains some additive-genetic variance for the trait under directional selection.^{[citation needed]}

Some traits have a natural physical limit beyond which a trait cannot possibly go.^[2] For example, replicated selection for the building of small thermoregulatory nests in mice reached a limit at or near zero (i.e., none of the provided cotton was being used to make nests).^[7] Similarly, lines of maize selected for low oil or protein content in the kernels reached limits near to zero percent.^[4]

Aside from absolute physical limits, and whatever their cause, limits or plateaus have often been observed in artificial selection experiments with animals, including: bristle number in fruit flies (Drosophila);^[8] avoidance behavior in laboratory rats;^[9] and large body size,^[4] large litter size,^[10] large nest size,^[7] and high voluntary wheel-running behavior in laboratory house mice.^[11]

Experimental approaches to probe the causes of selection are of two general types, quantitative genetic and functional. The former asks general questions about the genetic architecture of the trait when a limit has been attained (e.g., has narrow-sense heritability gone to zero?), whereas the latter attempts to determine what aspect of physiological or other function might have reached a limit or constraint. Experimental studies may involve attempts to break an apparent selection limit. As an example of a genetic approach, two replicate lines of mice at a limit for large nest size were crossed and selection was continued on this new population, resulting in further increase in nest size.^[12] From a functional perspective, in lines of mice at a selection limit for high wheel running, administration of an erythropoietin analog increased the maximal rate of oxygen consumption during forced exercise, but did not increase wheel running, a result suggesting that motivation for exercise may be limiting the behavior, rather than inherent ability to run on wheels.^[13]

• Bill Hill (geneticist)
• I. Michael Lerner
• Kenneth Mather
• Douglas Scott Falconer
• Experimental evolution
• Plant breeding