Appendix E. Detailed Methodology

In this appendix, we provide details about our methodology that go further than the brief overview presented in Chapter 3. We first detail how our database was created (e.g., how studies were identified, screened, and coded). We then describe how coded data were processed (e.g., how personality measures and ability tests were assigned to constructs). Lastly, we specify meta-analytic features (e.g., artifact distributions) of our research that are important to the psychometric meta-analytic methods used in this work (Schmidt & Hunter, 2014).

Database Creation

In creating a meta-analytic database, three areas are of critical importance: how primary studies are identified, which inclusion and exclusion criteria are used to screen studies, and how studies are coded. These areas are delineated below.

Study Identification

To ensure our results were representative of people across regions, ages, races, and backgrounds; transcendent of specific measurement instruments; and as accurate and precise as possible, we systematically employed nine diverse source identification strategies based on the methods devised in the dissertation of the first author (Stanek, 2014). Figure 3 in Chapter 3 summarizes the counts of studies identified, sifted, and eventually included in/excluded from the meta-analyses.

The identification of studies for potential inclusion in these meta-analyses began with studies included in relevant, previous meta-analyses and reviews (Aamodt, 2004; Ackerman & Heggestad, 1997; Chen et al., 2001; Cohn & Westenberg, 2004; Credé et al., 2010; DeRue et al., 2011; Hembree, 1988; Lange, 2013; Meriac et al., 2008; O’Boyle et al., 2013; Payne et al., 2007; Robbins et al., 2004; Signorella & Jamison, 1986; Steel, 2007; von Stumm et al., 2011; von Stumm & Ackerman, 2013; Wolf & Ackerman, 2005). From these materials we identified 1,945 studies, which constituted the starting point for our research.

Second, these initial studies were supplemented with database searches in PsycINFO, Web of Science, and Google Scholar. Keyword searches were conducted with the following cognitive ability and personality terms: intelligence, cognitive ability, general mental ability, mental ability, aptitude, g, g loaded, intellect, WAIS, Wechsler, Raven, Bennett, Bennett Mechanical Aptitude Test, Differential Aptitude Test, and DAT; personality, extraversion, extroversion, neuroticism, emotional stability, conscientiousness, agreeableness, openness, openness to experience, Big 5, Big Five, disposition, Myers Briggs, NEO PI-R, California Psychological Inventory, and CPI. Recognizing that strong research is conducted in many countries and languages, studies did not need to be written in English to be included in our meta-analyses. Trained research assistants and volunteers proficient in all required languages screened the identified articles according to the inclusion criteria and then assisted with processing and coding. To cover the countries with the most scientific psychology output in the world and the languages with the most speakers, Google Scholar was queried in English, German, Chinese, and Spanish (“Scimago Journal & Country Rank for Psychology 1996-2018,” 2018; Simons & Fennig, 2018). These procedures identified over 3 million studies that were then initially screened by 31 trained research assistants and volunteers over a five-year period.

The third study identification strategy queried WorldCat, the Educational Resources Information Center (ERIC), and the Network of Libraries, as well as search portals/databases specific to Africa, Australia, Brazil, Britain, Canada, Chile, China, Europe, Finland, France, Germany, Hong Kong, India, Korea, Latin America, Mexico, Netherlands, New Zealand, Portugal, Russia, Scandinavia, Singapore, Scotland (University of Edinburgh), South Africa, South America, Spain, Taiwan, and Wales. Beyond these continent-, country-, and even university-specific search portals, we also examined “open” databases and online repositories including Open Thesis, Journal of Articles in Support of the Null Hypothesis, Journal of Negative Results, Journal of Null Results, Social Science Research Network, The Data, Data Dryad, and Open Science Framework.

The fourth study identification strategy has been largely overlooked in previous meta-analyses: examination of governmental organizations’, especially military, research and databases. We inspected the online database for all public research conducted by the United States military, the Defense Technical Information Center, as well as the online database for Defence Research and Development Canada. Specific agencies’ databases were also searched (e.g., United States Federal Aviation Association). To incorporate research from other nations’ militaries, we examined all presentations that had been given at the International Military Testing Association and made available online (2004–2011). Many of these authors were also contacted via e-mail, phone, and/or in-person to request access to materials, clarify details, and investigate the existence of other relevant data. Fourteen studies were identified through this strategy.

The fifth strategy went beyond database searches to manually examine each article in high-yield journals for all issues across 20 years. Given their relevance, the following journals were selected: Intelligence, Journal of Personality and Social Psychology, Journal of Applied Psychology, European Journal of Personality, Personality and Individual Differences, Journal of Personality, Personality and Social Psychology Bulletin, Psychological Bulletin, Psychological Assessment, Journal of Experimental Psychology, Journal of Individual Differences, Personnel Psychology, Journal of Research in Personality, Annual Review of Psychology, and Medical Education.

The sixth strategy further extended the search to conference materials. Digitally archived programs of several high-relevance conferences were examined for any materials (e.g., posters, symposia, presentations, debates) that appeared to contain or cite relevant data. In most cases, authors had to be contacted for the relevant data/statistics since personality-cognitive ability relations were not the focus of their studies. All obtainable materials were examined.1 Conference programs examined were from the following: International Society for the Study of Individual Differences (1993–2013, excluding 2007, which was not available online), International Society for Intelligence Research (2000–2011), Society for Industrial and Organizational Psychology (1998–2013), European Conference on Personality (2008, 2010, 2012), World Conference on Personality (2013), and Association for Research in Personality (2000–2009, 2011, 2013). Through this strategy, 1,214 potentially usable materials were identified. Upon request, some of these organizations also sent their members invitations on our behalf to contribute to this meta-analysis (i.e., International Society for Intelligence Research, International Personality Assessment Council, International Military Testing Association). A couple organizations (i.e., International Society for the Study of Individual Differences, Association for Research in Personality) posted the invitation to contribute on their websites, but response to these indirect solicitations was very weak. Of the studies identified in this vein, less than 10% (i.e., 48) were obtained using solicitations and appeals.

The seventh study identification strategy involved contacting publishers of relevant assessments and other organizations involved in applied psychological measurement. In striving to be comprehensive, we contacted all members of the Association of Test Publishers (n = 179) individually with bespoke messages. Some members provided raw data, others provided summary statistics, and still others provided manuals or technical reports that contained useful information (n = 11). In instances when no response was received from test publishers, we searched through any test manuals/reports/notes we could otherwise obtain (more than 1,000). The personality and cognitive ability test manuals archive of Professor Deniz Ones was an invaluable resource in this endeavor. This archive incorporates manuals from the former University of Minnesota Psychology Library as well as from the personal libraries of Professors Thomas Bouchard, John P. Campbell, David Campbell, Marvin Dunnette, Stephan Dilchert, Jo-Ida-Hansen, and Deniz S. Ones. Based on examination of this archive, requests were sent to 42 additional test publishers of personality and cognitive ability measures. Relatedly, e-mail, phone, and postal mail requests were also sent to consulting firms, independent consultants, contacts in corporations, and practitioners at other institutions who the authors thought had access to datasets containing both personality and cognitive ability measures. Test manuals were pored over page-by-page, since relevant correlations and data were often reported for multiple samples within a single manual. We noticed that test manuals are especially likely to report data for the same sample in multiple publications (e.g., editions of the manual), though the sample sizes and even effect sizes sometimes changed, usually indicating updating of original samples by the addition of (a) new study participants and/or (b) additional measures. In such cases, we took care to include non-overlapping data in the present meta-analyses so as not to violate the independence of samples within a given analysis.

An eighth strategy to locate relevant data was to sift through the protocols of professionally run, large-scale studies (185 studies), since these projects tend to recognize the importance of individual difference domains such as personality and cognitive ability. Such studies were identified by examining reference lists of other materials found for this research, manually sifting through datasets in repositories compiled by others (e.g., Henry A. Murray Research Archive, Inter-university Consortium for Political and Social Research), and inspecting published directories of such studies (e.g., Hur & Craig, 2013; van Dongen et al., 2012). Most of these raw datasets have not been included in prior meta-analytic work, and their inclusion was paramount, given their quality, quantity, and relevance to our research questions. Twenty-five such studies provided personality trait and cognitive ability raw data. These studies often included multiple samples, waves, and/or datasets, which contained thousands of variables that required examining tens of thousands of pages of scanned codebooks and other relevant documents to be used.

Finally, the reference lists of all obtained materials were examined for citations of materials including relevant data/statistics. Relevant materials were retrieved from publication portals or by contacting study authors/project managers.

Inclusion and Exclusion Criteria

To be included in the current meta-analyses, studies had to provide at least one correlation coefficient or convertible effect size between a personality trait and a cognitive ability as well as an associated sample size. Secondary summaries that aggregated primary effects were excluded. Many studies collected relevant personality and ability measurements in usable samples but did not report effect sizes relating personality constructs to cognitive ability constructs. For 673 such studies, we spent a year attempting to individually contact the authors and obtain the raw data or usable statistics. This enabled 100 more materials to be included.

Study Design Characteristics

Bivariate relations had to be reported in the contributing material or calculable based on individual-level data. Studies reporting personality-cognitive ability relations at group (e.g., team), organization (e.g., school, company), or country levels of analysis were excluded. Within-individual personality-cognitive ability relations were also excluded. We also excluded studies with experimental manipulations that may have impacted personality or cognitive ability scores, as well as contrasted/extreme group studies where participants were selected based on having extreme personality and/or cognitive ability scores. After being converted to Pearson correlations and correcting for dichotomization, phi coefficients from non-extreme group contrasted studies were included (Hunter & Schmidt, 1990).

Samples

Clinical or pathological samples and samples with participants younger than 12 years old were not included. These criteria were implemented to report only on relatively stable (Briley & Tucker-Drob, 2013) and normative trait variation not affected by psychoactive medications or psychiatric settings.

When processing raw data and the information was provided, we excluded data from individuals who received more than minimal prompting/assistance on cognitive ability tests, who had their testing sessions interrupted (e.g., by a phone ringing), or about whom the original researchers had noted a significant impairment (e.g., “unable to read the test due to a visual impairment; glasses were not sufficiently helpful”).

Statistical Criteria

In contrast to some previous personality-cognitive ability meta-analyses (e.g., von Stumm & Ackerman, 2013), we included studies that presented results from a non-select set of all relevant correlations/effect sizes. Including studies reporting only statistically significant results (e.g., Lukey & Baruss, 2005) would have inflated personality-ability relations (Ones et al., 1993). Studies that only reported partial or semipartial correlations (e.g., path coefficients) controlling for other variables were excluded if bivariate Pearson correlation coefficients could not be calculated retroactively or obtained from the authors. This is important because partial and semipartial correlations are not directly comparable to zero order correlations.

Personality Trait Variables

To be included, personality variables had to meet multiple criteria. Measures of attitudes, values, self-efficacy, state affect, mood, response bias, and validity scales were not included in the current analyses. Only personality scales that measured personality using self-reports were included (i.e., we did not include others’ ratings or objectively recorded behavioral indicators of personality [cf. Wiernik et al., 2020]). Self-report and other-report measurements of personality each have their own strengths and weaknesses and therefore are complementary approaches to measuring personality (Connelly & Ones, 2010). However, combining self- and other-reports in the same meta-analysis would have created heterogeneity and muddled results. Without conducting two separate sets of meta-analyses, one focusing on self-report personality-cognitive ability relations and the other on other-report personality-cognitive ability relations, there was no conceptually and psychometrically clean way of including other-report measures in this research. We also focused on studies that measured personality with scales (i.e., studies that reported relations with single personality items were avoided as we wished to examine latent personality traits, and, without repeated measurements, single item measures of personality tend to be too unreliable and typically construct deficient to assess broad personality factors or sub-factors (Epstein, 1979; Ones et al., 2016)). We encourage future researchers to undertake item-level meta-analyses (Mõttus et al., 2020), if robust data become available.

For multiple reasons, ipsative personality measures were also excluded. Ipsative measures force respondents to make choices between traits (i.e., “Are you higher on extraversion or openness?”). First, ipsative measures are not designed to yield normative scores. In fact, ipsativity hampers between-individual normative comparisons and biases the rank ordering of individuals. Such issues of fully ipsative measures are well-documented in the literature (Dunlap & Cornwell, 1994; Hicks, 1970; Tenopyr, 1988). Second, some evidence indicates that ipsative measures are more cognitively demanding for respondents and therefore more highly correlated with cognitive ability than normative measures of personality (e.g., Vasilopoulos et al., 2006). Normative measures of personality require respondents to make a single decision: how much they agree or disagree with an item. Forced choice measures require multiple decisions: indicating how much they agree or disagree with an item and ranking the item relative to the rest of the alternatives. Quasi-ipsative measures were included (Salgado et al., 2015).

Cognitive Ability Variables

Only studies that utilized objective measures of cognitive abilities were included. We excluded subjective ratings of cognitive ability (e.g., self-estimates of ability). Measures of the consequences or outcomes of cognitive ability were also excluded (e.g., school grades/grade point average, performance on situational judgement tests and assessment center exercises). We also excluded cognitive style, metacognition, attributional complexity, and executive functioning measures (e.g., Stroop, trail-making, tower of London, Wisconsin card sort). Finally, we excluded measures of emotional intelligence, practical intelligence, cultural intelligence, rational intelligence, etc. since they have been too heterogeneously defined and measured. They are also not yet fully understood and incorporated into models of human intelligence (e.g., CHC [Schneider & McGrew, 2012] and Unified CHC [Stanek & Ones, 2018]). Literatures are emerging around each of these constructs (Cabrera & Nguyen, 2001; Dilchert & Ones, 2004; Joseph & Newman, 2010; McDaniel et al., 2007; Van Rooy et al., 2006; Van Rooy & Viswesvaran, 2004), but including them in the present effort would have injected undesirable heterogeneity in the meta-analytic estimates.

Final Set of Included Studies

The above search strategies and inclusion/exclusion criteria yielded 1,325 studies contributing independent data to the meta-analyses in this research. We were as thorough as possible in our approach to minimize the impact of publication bias; to investigate beyond Western, educated, industrialized, rich, and democratic societies; and to avoid the biases inherent in the paradigm of any one field/research perspective (Henrich et al., 2010; Simmons et al., 2011). Even after more than five years of searching, the current study is unlikely to be fully exhaustive. That is, there were likely studies in existence that we did not locate, but it is hoped that the search methods employed yielded a representative majority of the relevant, available, quantitative information on the relations between personality traits and cognitive ability constructs.2 Comparison of the number of studies included in this set of meta-analyses with prior meta-analyses of personality-cognitive ability relations indicates that we created the largest and most comprehensive database to date on this topic, and one of the largest meta-analytic databases on any topic (cf. only larger meta-analysis [Polderman et al., 2015]). The meta-analytic database is publicly available in Appendix F. A full list of studies contributing effect sizes to the present meta-analyses is presented in Appendix K.

Coding of Studies and Data Entry

A supervised team of trained researchers including the authors, graduate students, undergraduate students, and university graduates coded the studies. More than 90% of the entries were double-checked by the lead author or via blind double-entry by multiple research assistants. These double-entry and checking procedures identified a small number of typographical errors, which were corrected before the analyses were run.

Each contributing study’s year, authors, publication type, source name, sample size, cognitive ability measure used, personality measure used, reliabilities for the personality and cognitive ability measures observed in the sample, type of reliability calculated, and correlation between the cognitive ability and personality variables were coded. To the extent they were reported, sample demographics were coded in order to help characterize the meta-analytic database.

We also conducted a final, independent check on our database using the database from the most cited previous meta-analyses examining personality and cognitive ability (Ackerman & Heggestad, 1997). Dr. Heggestad graciously provided the database from that investigation to allow for comparisons of data entry consistency and accuracy. The database of Ackerman and Heggestad included 119 materials, while the current meta-analyses examined 1,325 materials. Several studies included by Ackerman and Heggestad were excluded from the current analyses because they did not meet our inclusion criteria (e.g., included children younger than 12, utilized ipsative measures). Comparisons of the overlapping studies’ sample sizes and effect sizes coded revealed 98% agreement. Disagreements were due to ambiguities in the original primary study reporting (e.g., sample size reported as a range). This approach of independently coding then checking against previous meta-analyses’ database is a rare but important method for quality assurance. We advocate for this approach to become a routine method for assessing coding quality, especially as meta-analytic reporting standards become more widespread and updates are conducted.

Data Preparation

Prior to running meta-analyses, the data had to be further processed and prepared. While Pearson correlations were reported in many cases, in other instances they had to be computed from raw data or converted from other types of effect sizes. Additionally, each of the myriad personality and cognitive ability measures needed to be mapped to the construct they assessed. After assigning measures to constructs, we checked for independence of effect sizes for each meta-analysis to be conducted. These data preparation steps are described in detail below.

Construct Categorization: Personality

A consistent personality framework had to be implemented to organize the personality measures and constructs for analysis. Stanek and Ones (2018) studied a variety of personality taxonomies (e.g., Ashton et al., 2004; Cattell, 1946; Costa & McCrae, 1992; Davies, 2012; DeYoung et al., 2007; Edmonds, 1929; H. J. Eysenck, 1959; H. J. Eysenck & Himmelweit, 1947; M. Eysenck, 1992; S. B. G. Eysenck, 1965; Galton, 1884; Goldberg, 1992; Hofstee et al., 1992; Hough & Ones, 2002; John et al., 1991; Norman, 1963; Stark et al., 2014; Tellegen, 1982; Wundt, 1897), as well as emerging factor analytic and nomological network evidence (Soto & John, 2017), to synthesize the Pan-Hierarchical Five Factor Model and create an empirically rooted compendium of personality measures. Although it can be tempting to ascribe constructs to measures based on the scale name or a primary study author’s description, such methods suffer from significant flaws, including the jingle-jangle fallacy, disparate usages of common terms (e.g., extraversion), and common misunderstandings by researchers of what constructs are actually captured by the measures they employ.

Across all levels of the personality hierarchy, there were 79 personality constructs included in the present meta-analyses. Definitions of each construct and a compendium of scales assessing each may be found in Stanek and Ones (2018) and Appendices C and D.

The Pan-Hierarchical Five Factor Model (Stanek & Ones, 2018) was used in the present meta-analytic investigations (see Chapter 2, Figure 2 for a depiction). In our classifications for this study, each personality measure was identified as an indicator of a single personality factor, meta-trait, aspect, facet, or compound trait according to the extensive Stanek and Ones (2018) compendium listing measures of each personality construct. Particularly challenging scale classification decisions benefited from the input of Dr. Colin DeYoung. Overall, 2,861 personality scales contributed data to the present analyses, and the use of transparent, open access, well-validated compendia for both the personality domain and the cognitive ability domain supports the replicability and future extension of the analyses presented.

Construct Categorization: Cognitive Ability

Cognitive ability is a domain of interrelated abilities that are hierarchically organized, with general mental ability at the apex (see Figure 1 in Chapter 2 for depiction). The general mental ability factor (g) stems from the positive manifold among specific indicators of cognitive ability (Carroll, 1993; Spearman, 1904). General mental ability can be measured via the g factor extracted from a sufficiently broad battery of cognitive tests or via the overall score of a cognitive ability test assessing different primary abilities. The present research endeavored to examine the relations between constructs of personality and cognitive ability (e.g., general mental ability, extraversion), not specific scales (e.g., Wonderlic, Big Five Aspects Scales). To achieve this, we used a consistent cognitive ability framework to organize constructs and map measures: the Stanek and Ones’ (2018) Unified CHC taxonomy and compendium of associated measures, which is an update of the Cattell-Horn-Carroll (CHC) taxonomy of cognitive abilities (McGrew, 1997, 2005, 2009).

Ninety-seven cognitive abilities were included. These include 70 specific abilities (e.g., number facility and quantitative reasoning), which are grouped into 10 dimensions of primary abilities (e.g., processing speed and fluid abilities), which in turn can be organized into four groups: domain independent general capacities (i.e., fluid abilities, memory), sensory-motor domain specific abilities (e.g., visual processing, auditory processing), speed capacities (e.g., processing speed, reaction and decision speed), and invested abilities (i.e., acquired knowledge). There were also compound ability measures, which include two or three primary abilities.

Another goal of the current research was to examine personality-cognitive ability relations in fine detail. Therefore, each contributing ability scale was assigned to a single ability by the authors according to the Stanek and Ones (2018) compendium. Particularly challenging scale classifications benefited from the input of Dr. Kevin McGrew, the current intellectual custodian of the CHC taxonomy. Overall, 2,388 cognitive ability scales contributed to the present analyses. 97 cognitive ability constructs across all levels of the ability hierarchy were included.3 Definitions of each construct and a compendium of scales assessing each can be found in Stanek and Ones (2018) and Appendices A and B

Effect Size Independence

To avoid issues of non-independence, each sample was permitted to contribute only one effect size to each personality-cognitive ability construct meta-analysis. All meta-analyses presented contain only effect sizes obtained from independent samples. In instances where one sample would have contributed two effect sizes to the same meta-analysis, a composite was formed to ensure the sample was only counted once but all available information was incorporated (Schmidt & Hunter, 2014).

Correlations for the same sample and measures were reported in multiple sources in several cases (e.g., Furnham et al., 2007, 2008). Most sources failed to properly indicate sample dependence/reuse, so our team had to manually examine materials for clues of non-independence. To do so, we examined sample descriptions, demographics of the sample (if reported), sources’ authors, and publication/data-collection year. The majority of these cases were not straightforward investigations, in part because authors tended to drop cases and combine measures differentially across publications (e.g., Aitken Harris, 1999; Ashton et al., 2000). When needed, we contacted authors directly in attempts to verify.

Meta-Analytic Approach

Psychometric meta-analysis (Schmidt & Hunter, 2014) methods were used to combine effect sizes across studies and improve statistical power, reduce error variance of the estimated effect sizes, and estimate the degree to which results might generalize to populations and situations beyond those investigated in individual primary studies. The technique has its origins in applied psychology (Schmidt & Hunter, 1977), but has now become ubiquitous in every domain of science. Psychometric meta-analysis is a random effects approach to cumulating findings across studies that, in addition to minimizing the influence of sampling error, also accounts for differences across studies in reliability of measurement as well as other applicable statistical artifacts.

In meta-analysis, the mean observed correlation r̄ indicates the average, sample-size-weighted observed correlation across studies’ effect sizes. Like most other meta-analytic approaches, psychometric meta-analysis reduces the impact of sampling error by pooling data across many studies and thus increasing the sample size associated with effect sizes produced. However, most other meta-analytic approaches do not account for the systematic effects that downwardly bias effect sizes (Schmidt & Hunter, 2014). For example, no measure is perfectly reliable, and this is particularly true for psychological measures. Measurement error in both personality and cognitive ability measures (e.g., due to misreading an item) systematically lowers the magnitudes of observed correlations, masking correlations that exist between constructs from the two domains (Wiernik & Ones, 2017). Psychometric meta-analysis corrects for this source of systematic bias, thereby producing accurate estimates of the relations between personality and cognitive ability constructs. r̄ is corrected for unreliability as well as other applicable statistical artifacts (e.g., dichotomization) to attain the estimated mean corrected correlation p̂ (i.e., estimated true-score correlation). In our meta-analyses, correlations were corrected for sampling error and unreliability in both the cognitive ability and personality measures. Such corrections result in estimates of the construct-level relations free of measurement error, which are described further in the next section. For the interested reader, Supplementary Tables 3–99 and 197–275 report the observed as well as the psychometrically-corrected relations (see Appendices G and I).

The variability of observed correlations across effect sizes is indicated by SDr. Observed standard deviations reflect the fact that unreliability and other statistical artifacts inflate variation in effects observed across studies. Thus, corrections are necessary to control for the differential reliabilities of various scales contributing to each meta-analysis in order to estimate true variability. The estimated standard deviation of corrected correlations (SDp̂) indicates the degree of true (i.e., non-artifactual) variability associated with the mean, corrected correlation. It indexes true heterogeneity. Other meta-analysts sometimes refer to this as τ (e.g., Hedges & Vevea, 1998).4 The present meta-analyses also produced precise estimates of the degree of true variability in the meta-analytic distributions. Beyond just a dichotomous pronouncement of heterogeneity being present/null, these SDp̂ values are important indicators of the degree of variability beyond that due to sampling error and measure unreliability in the relations examined.

We also computed 80% credibility values for each meta-analytic result. The credibility value range indicates the range in which most individual true-score correlations would be expected to fall (e.g., when new studies are conducted; Schmidt & Hunter, 2014). Reporting credibility value ranges reveals whether the examined relations are expected to generalize (Ones et al., 2017; Wiernik et al., 2017). Overall, the results from these meta-analyses constitute a large-scale quantification of the generalizability of personality-cognitive ability relations. To estimate these various statistics, we used the psychmeta package in R (Dahlke & Wiernik, 2018).

Corrections for Unreliability in Cognitive Ability and Personality Measures: Artifact Distributions

As noted above, observed correlations are systematically downwardly biased, and differences in reliabilities across studies inflate observed variation in effect sizes (Schmidt & Hunter, 2014). To correct for this attenuation due to measurement unreliability, robust artifact distributions were compiled drawing on several sources: the individual effect sizes in the current meta-analytic database, previous meta-analyses that compiled reliability distributions for various personality constructs (Davies, 2012), technical manuals of the respective measures, and articles containing representative samples. The means, standard deviations, and number of reliability estimates for each construct’s artifact distribution are reported in Table S1 (for cognitive ability constructs) and Table S2 (for personality constructs). For cognitive ability constructs, these distributions were constructed exclusively from internal consistency reliability estimates and short-term (less than two months) test-retest reliability values.5

Although less common, some psychometric meta-analyses also correct for the attenuating effect of range restriction. This is appropriate in cases where range restriction, direct or indirect, is known to affect variation in either of the variables of study. In the present meta-analyses, information about the levels of direct or indirect range restriction was not available and could not be computed based on information provided in most contributing studies. To the degree that range restriction existed, it would result in conservative estimates of personality-cognitive ability relations and larger associated variabilities. In any case, we do not expect the degree of underestimation of true-score correlations and overestimation of variability to be large. Based on the contributing studies’ sample descriptions, direct range restriction was only even possible in 8% of samples (e.g., occupational samples). Previous research has shown that gravitation to jobs reduces variability by about 10% for cognitive ability (Sackett & Ostgaard, 1994) and about 4% for personality (Ones & Viswesvaran, 2003). While there is convincing empirical evidence that differential gravitation into occupations affects mean cognitive ability and personality levels, variabilities are far less affected, especially for personality traits. Therefore, the impact of not correcting for direct or indirect range restriction on cognitive ability and personality variables can be expected to be a rather small attenuating influence on the results. If full information were available in the primary studies and appropriate corrections could be undertaken, the result would be slightly larger personality-cognitive ability relations than those we report, since range restriction depresses correlation coefficients. Finally, because most previous meta-analytic investigations of personality-cognitive ability relations did not correct for range restriction, leaving our results uncorrected for range restriction makes them directly comparable to previous research.

Potential Impact of Publication Bias

An often-cited potential threat to the validity of meta-analytic conclusions is publication bias. The meta-analyses presented in this volume are unlikely to have been affected by publication bias for three reasons. First, almost two-thirds (63%) of the effect sizes contributing to these analyses are from unpublished sources (e.g., data archives, unpublished conference manuscripts, dissertations). By including these materials, not only did we safeguard against publication bias, but we also brought to light and made accessible findings from otherwise missed primary studies, research samples, and databases. These freshly computed metrics are now surfaced from archives, mapped to constructs, and available to the scientific community as part of the publicly available dataset associated with the current meta-analyses.

Second, few of the studies included in this research were conducted for the express purpose of examining personality and cognitive ability relations. Rather, these two sets of variables were often included in broader examinations of other phenomena such as physical health, educational attainment, job success, development and aging, interpersonal relations, and decision-making. That is, relations between personality and cognitive ability were often incidentally reported in the correlation matrices, reducing the threat of underreporting of null relations (see supplement of Wilmot & Ones, 2019). Thus, in the case of the published studies (i.e., one-third of the database), we have no reason to suspect that only significant or sizable personality-intelligence relations are reported since they were rarely the focus of the investigations.

Third, the threat of publication bias was also mitigated by the depth of our search strategies: multiple electronic databases were scrutinized, relevant studies from reference lists of examined studies were culled, and sources were not excluded because they were in different languages. In this vein, the contribution of non-USA-based materials and data from nationally representative, large-scale studies6 was another means of protection against publication bias.

Impact of Outlier Samples

For some meta-analyses, results varied when an extremely large study (Project Talent) was excluded. In some of these instances, Project Talent contributed data for approximately twice as many participants as other primary studies, which influenced some of the reported relations. Project Talent was a nation-wide study of 1960s American high schoolers. However, if it is not representative of the broader population of studies in terms of factors that impact the relations between personality and cognitive ability, then the results of otherwise small-N meta-analyses including them may be biased.

We had no a priori reason to think that the Project Talent data were deviant. However, we ran all meta-analyses excluding Project Talent to examine its influence. Some results did not change substantially (e.g., relations with cautiousness), but others did. The general trend was toward smaller relations. In general, the pattern of conclusions did not change. Relations with two conscientiousness constructs, however, were significantly altered by the exclusion of Project Talent data: order and industriousness. With Project Talent data included, these traits were more highly correlated with cognitive abilities, especially with acquired verbal abilities. Excluding Project Talent data also muted some of the effects reported for some extraversion facets, including the activity facet. Relations with openness deflated the least. As an additional check on our data, a non-author PhD-level psychologist at a different research university independently computed and verified the effect sizes included from Project Talent. Other researchers have also found sizable uncorrected relations for personality and intelligence relations in the Project Talent data (Reeve et al., 2006). All the technical results without Project Talent data can be found in Supplementary Tables 100–196 and 276–354 in Appendices H and J.

To further explore the potential impact of Project Talent we examined various features of the data. Scales used as part of Project Talent are well-established (Flanagan et al., 1964; Pozzebon et al., 2013). Although some of the cognitive ability measures in Project Talent displayed very poor reliability, that fact would have attenuated the observed correlations when the data were included, rather than made them unusually strong (e.g., in the case of relations with industriousness). We also examined whether distributional properties of the measures (e.g., ceiling and floor effects) could produce aberrant results. To the extent that they exist, floor and ceiling effects also attenuate correlations. However, we found no evidence of these effects. Instead, we observed distributions that would be expected from nationally representative samples. On the other hand, range enhancement could result in stronger relations (Schmidt & Hunter, 2014, p. 37). We were not able to ascertain the degree of any potential range enhancement in any specific variable. We note, however, that variances in study variables are typically not restricted or enhanced in nationally representative samples such as the one studied in Project Talent. The national origin, age, and generation of the sample are also unlikely to be the driving force behind stronger results, as other studies of individuals from this country, age group, and era showed more typical personality-cognitive ability relations. One possible explanation is that youths, especially those high on industriousness, participating in this once-in-an-era study may have felt particularly motivated to do their best on the cognitive ability assessments. In contrast, those lower in industriousness may have felt that the results of the assessments had no impact on their future and were therefore not worth maximal effort. If both effects occurred simultaneously, range enhancement might be a possible explanatory mechanism. Comparisons of motivated versus less motivated samples' personality-cognitive ability relations could shed light on the question. We also encourage a new Project Talent-like study so that research on individual differences and their life consequences are not limited to one such study, half a century old.

References

Aamodt, M. G. (2004). Special issue on using MMPI-2 scale configurations in law enforcement selection: Introduction and meta-analysis. Applied HRM Research, 9(2), 41–52.

Ackerman, P. L., & Heggestad, E. D. (1997). Intelligence, personality, and interests: Evidence for overlapping traits. Psychological Bulletin, 121, 219–245.

Aitken Harris, J. (1999). Personality and Measured Intelligence [Unpublished doctoral dissertation]. University of Western Ontario.

Ashton, M. C., Lee, K., Perugini, M., Szarota, P., De Vries, R. E., Di Blas, L., Boies, K., & De Raad, B. (2004). A six-factor structure of personality-descriptive adjectives: Solutions from psycholexical studies in seven languages. Journal of Personality and Social Psychology, 86(2), 356.

Ashton, M. C., Lee, K., Vernon, P. A., & Jang, K. L. (2000). Fluid intelligence, crystallized intelligence, and the openness/intellect factor. Journal of Research in Personality, 34(2), 198–207.

Briley, D. A., & Tucker-Drob, E. M. (2013). Explaining the Increasing Heritability of Cognitive Ability Across Development A Meta-Analysis of Longitudinal Twin and Adoption Studies. Psychological Science, 24(9), 1704–1713.

Cabrera, M. A. M., & Nguyen, N. T. (2001). Situational judgment tests: A review of practice and constructs assessed. International Journal of Selection and Assessment, 9(1–2), 103–113.

Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. Cambridge Univ Pr.

Cattell, R. B. (1946). Description and measurement of personality. (1st ed.). World Book Co.

Chen, G., Casper, W., & Cortina, J. (2001). The roles of self-efficacy and task complexity in the relationships among cognitive ability, conscientiousness, and work-related performance: A meta-analytic examination. Human Performance, 14(3), 209–230.

Cohn, L. D., & Westenberg, P. M. (2004). Intelligence and maturity: Meta-analytic evidence for the incremental and discriminant validity of Loevinger’s measure of ego development. Journal of Personality and Social Psychology, 86(5), 760.

Connelly, B. S., & Ones, D. S. (2010). An other perspective on personality: Meta-analytic integration of observers’ accuracy and predictive validity. Psychological Bulletin, 136(6), 1092.

Costa, P. T., & McCrae, R. R. (1992). NEO PI-R professional manual. Psychological Assessment Resources.

Credé, M., Roch, S. G., & Kieszczynka, U. M. (2010). Class Attendance in College A Meta-Analytic Review of the Relationship of Class Attendance With Grades and Student Characteristics. Review of Educational Research, 80(2), 272–295.

Dahlke, J. A., & Wiernik, B. M. (2018). psychmeta: An R package for psychometric meta-analysis. Applied Psychological Measurement. https://doi.org/0146621618795933

Davies, S. E. (2012). Lower and higher order facets and factors of the interpersonal traits among the big Five: Specifying, measuring, and understanding extraversion and agreeableness [Unpublished doctoral dissertation]. University of Minnesota.

DeRue, D. S., Nahrgang, J. D., Wellman, N. E. D., & Humphrey, S. E. (2011). Trait and behavioral theories of leadership: An integration and meta-analytic test of their relative validity. Personnel Psychology, 64(1), 7–52.

DeYoung, C. G., Quilty, L. C., & Peterson, J. B. (2007). Between facets and domains: 10 aspects of the Big Five. Journal of Personality and Social Psychology, 93(5), 880.

Dilchert, S. (2008). Measurement and Prediction of Creativity at Work [Ph.D., University of Minnesota]. http://search.proquest.com.ezp2.lib.umn.edu/dissertations/docview/993006072/abstract/25467E957C524AFAPQ/1?accountid=14586

Dilchert, S., & Ones, D. S. (2004, April). Meta-analysis of practical intelligence: Contender to the throne of g? Society for Industrial and Organizational Psychology, Chicago, Illinois, United States of America.

Dunlap, W. P., & Cornwell, J. M. (1994). Factor analysis of ipsative measures. Multivariate Behavioral Research, 29(1), 115–126.

Edmonds, J. M. (1929). The Characters of Theophrastus. William Heinemann Ltd.

Epstein, S. (1979). The stability of behavior: I. On predicting most of the people much of the time. Journal of Personality and Social Psychology, 37(7), 1097.

Eysenck, H. J. (1959). Personality and problem solving. Psychological Reports, 5(3), 592–592.

Eysenck, H. J., & Himmelweit, H. T. (1947). Dimensions of personality: A record of research carried out in collaboration with HT Himmelweit [and others]. K. Paul, Trench, Trubner.

Eysenck, M. (1992). Anxiety: The cognitive perspective. Psychology Press. http://books.google.com/books?hl=en&lr=&id=uUu1aFkq_h0C&oi=fnd&pg=PR11&dq=eysenck&ots=ijZ7UV7t4F&sig=ZJ0QRZ8sQtnKi1pVtGtxP5-AEEA

Eysenck, S. B. G. (1965). Manual of the junior Eysenck personality inventory. University of London Press.

Flanagan, J. C., Davis, F. B., Dailey, J. T., Shaycoft, M. F., Orr, D. B., Goldberg, I., & Neyman, C. A. (1964). The American high-school student (Technical Report Vol. 4). University of Pittsburgh, ProjectTALENT Office.

Furnham, A., Christopher, A., Garwood, J., & Martin, N. G. (2008). Ability, demography, learning style, and personality trait correlates of student preference for assessment method. Educational Psychology, 28(1), 15–27.

Furnham, A., Christopher, A. N., Garwood, J., & Martin, G. N. (2007). Approaches to learning and the acquisition of general knowledge. Personality and Individual Differences, 43(6), 1563–1571.

Galton, F. (1884). Measurement of character. Fortnightly Review, 36(212), 179–185.

Goldberg, L. R. (1992). The development of markers for the Big-Five factor structure. Psychological Assessment, 4(1), 26.

Hedges, L. V., & Nowell, A. (1995). Sex differences in mental test scores, variability, and numbers of high-scoring individuals. Science, 269(5220), 41–45.

Hedges, L. V., & Vevea, J. L. (1998). Fixed-and random-effects models in meta-analysis. Psychological Methods, 3(4), 486.

Hembree, R. (1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58(1), 47.

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83.

Hicks, L. E. (1970). Some properties of ipsative, normative, and forced-choice normative measures. Psychological Bulletin, 74(3), 167.

Hofstee, W. K., De Raad, B., & Goldberg, L. R. (1992). Integration of the big five and circumplex approaches to trait structure. Journal of Personality and Social Psychology, 63(1), 146.

Hough, L. M., & Ones, D. S. (2002). The structure, measurement, validity, and use of personality variables in industrial, work, and organizational psychology. In Handbook of industrial, work, and organizational psychology (Vol. 1, pp. 233–277). Sage.

Hunter, J. E., & Schmidt, F. L. (1990). Dichotomization of continuous variables: The implications for meta-analysis. Journal of Applied Psychology, 75(3), 334.

Hur, Y.-M., & Craig, J. M. (2013). Twin registries worldwide: An important resource for scientific research. Twin Research and Human Genetics, 16(01), 1–12.

John, O. P., Donahue, E. M., & Kentle, R. L. (1991). The Big Five Inventory—Versions 4a and 54. Institute of Personality and Social Research.

Joseph, D. L., & Newman, D. A. (2010). Emotional intelligence: An integrative meta-analysis and cascading model. Journal of Applied Psychology, 95(1), 54.

Lange, S. (2013). Metaanalytic investigation between g, Gf, Gc with the Big Five with regard to publication bias [Doctoral dissertation]. University of Bonn.

Lukey, N., & Baruss, I. (2005). Intelligence correlates of transcendent beliefs: A preliminary study. Imagination, Cognition and Personality, 24(3), 259–270.

McDaniel, M. A., Hartman, N. S., Whetzel, D. L., & Grubb, W. (2007). Situational Judgment Tests, Response Instructions, and Validity: A meta-analysis. Personnel Psychology, 60(1), 63–91.

McGrew, K. S. (1997). Analysis of the major intelligence batteries according to a proposed comprehensive Gf-Gc framework. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 151–179). Guilford Press. http://psycnet.apa.org/psycinfo/1997-97010-009

McGrew, K. S. (2005). The Cattell-Horn-Carroll Theory of Cognitive Abilities: Past, Present, and Future. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary Intellectual Assessment (pp. 136–181). Guilford Press.

McGrew, K. S. (2009). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research. Intelligence, 37(1), 1–10.

Meriac, J. P., Hoffman, B. J., Woehr, D. J., & Fleisher, M. S. (2008). Further evidence for the validity of assessment center dimensions: A meta-analysis of the incremental criterion-related validity of dimension ratings. Journal of Applied Psychology, 93(5), 1042.

Mõttus, R., Wood, D., Condon, D. M., Back, M. D., Baumert, A., Costantini, G., Epskamp, S., Greiff, S., Johnson, W., Lukaszewski, A., Murray, A., Revelle, W., Wright, A. G. C., Yarkoni, T., Ziegler, M., & Zimmerman, J. (2020). Descriptive, predictive and explanatory personality research: Different goals, different approaches, but a shared need to move beyond the Big Few traits. European Journal of Personality, 34(6), 1175–1201.

Norman, W. T. (1963). Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings. Journal of Abnormal and Social Psychology, 66(6), 574.

O’Boyle, E. H., Forsyth, D., Banks, G. C., & Story, P. A. (2013). A meta-analytic review of the Dark Triad–intelligence connection. Journal of Research in Personality, 47(6), 789–794.

Ones, D. S., & Viswesvaran, C. (2003). Job-specific applicant pools and national norms for personality scales: Implications for range-restriction corrections in validation research. Journal of Applied Psychology, 88(3), 570.

Ones, D. S., Viswesvaran, C., & Schmidt, F. L. (1993). Comprehensive meta-analysis of integrity test validities: Findings and implications for personnel selection and theories of job performance. Journal of Applied Psychology, 78(4), 679.

Ones, D. S., Viswesvaran, C., & Schmidt, F. L. (2017). Realizing the full potential of psychometric meta-analysis for a cumulative science and practice of human resource management. Human Resource Management Review, 27(1), 201–215. https://doi.org/10.1016/j.hrmr.2016.09.011

Ones, D. S., Wiernik, B. M., Wilmot, M. P., & Kostal, J. W. (2016). Conceptual and methodological complexity of narrow trait measures in personality-outcome research: Better knowledge by partitioning variance from multiple latent traits and measurement artifacts. European Journal of Personality, 30(4), 319–321.

Payne, S. C., Youngcourt, S. S., & Beaubien, J. M. (2007). A meta-analytic examination of the goal orientation nomological net. Journal of Applied Psychology, 92(1), 128–150.

Polderman, T. J. C., Benyamin, B., De Leeuw, C. A., Sullivan, P. F., Van Bochoven, A., Visscher, P. M., & Posthuma, D. (2015). Meta-analysis of the heritability of human traits based on fifty years of twin studies. Nature Genetics, 47(7), 702.

Pozzebon, J., Damian, R. I., Hill, P., Lin, Y., Lapham, S., & Roberts, B. W. (2013). Establishing the validity and reliability of the Project Talent Personality Inventory. Frontiers in Psychology, 4, 968.

Reeve, C. L., Meyer, R. D., & Bonaccio, S. (2006). Intelligence–personality associations reconsidered: The importance of distinguishing between general and narrow dimensions of intelligence. Intelligence, 34(4), 387–402.

Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin, 130(2), 261.

Sackett, P. R., & Ostgaard, D. J. (1994). Job-specific applicant pools and national norms for cognitive ability tests: Implications for range restriction corrections in validation research. Journal of Applied Psychology, 79(5), 680.

Salgado, J. F., Anderson, N., & Tauriz, G. (2015). The validity of ipsative and quasi-ipsative forced-choice personality inventories for different occupational groups: A comprehensive meta-analysis. Journal of Occupational and Organizational Psychology, 88(4), 797–834.

Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62(5), 529.

Schmidt, F. L., & Hunter, J. E. (2014). Methods of Meta-Analysis: Correcting Error and Bias in Research Findings (3rd ed.). Sage Publications.

Schneider, W. J., & McGrew, K. S. (2012). The Cattell-Horn-Carroll model of intelligence. In D. P. Flanagan & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (3rd ed., pp. 99–144). Guilford Press.

Scimago journal & country rank for psychology 1996-2018. (2018, December 31). [Website]. Scimago Rankings. https://www.scimagojr.com/countryrank.php?area=3200

Signorella, M. L., & Jamison, W. (1986). Masculinity, femininity, androgyny, and cognitive performance: A meta-analysis. Psychological Bulletin, 100(2), 207.

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology. Psychological Science, 22(11), 1359–1366.

Simons, G. F., & Fennig, C. D. (Eds.). (2018). Ethnologue: Languages of the world (21st ed.). SIL International. http://www.ethnologue.com

Soto, C. J., & John, O. P. (2017). The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Personality and Social Psychology, 113(1), 117.

Spearman, C. (1904). “General Intelligence,” Objectively Determined and Measured. The American Journal of Psychology, 201–292.

Stanek, K. C. (2014). Meta-Analyses of Personality and Cognitive Ability [Doctora dissertation, University of Minnesota-Twin Cities]. https://hdl.handle.net/11299/201107

Stanek, K. C., & Ones, D. S. (2018). Taxonomies and compendia of cognitive ability and personality constructs and measures relevant to industrial, work and organizational psychology. In D. S. Ones, C. Anderson, C. Viswesvaran, & H. K. Sinangil (Eds.), The SAGE handbook of industrial, work & organizational psychology: Personnel psychology and employee performance (pp. 366–407). Sage.

Stark, S., Chernyshenko, O. S., Drasgow, F., Nye, C. D., White, L. A., Heffner, T., & Farmer, W. L. (2014). From ABLE to TAPAS: A new generation of personality tests to support military selection and classification decisions. Military Psychology, 26(3), 153–164.

Steel, P. (2007). The nature of procrastination: A meta-analytic and theoretical review of quintessential self-regulatory failure. Psychological Bulletin, 133(1), 65.

Tellegen, A. (1982). Brief manual for the multidimensional personality questionnaire. Unpublished Manuscript, University of Minnesota, Minneapolis, 1031–1010.

Tenopyr, M. L. (1988). Artifactual reliability of forced-choice scales. Journal of Applied Psychology, 73(4), 749.

van Dongen, J., Slagboom, P. E., Draisma, H. H., Martin, N. G., & Boomsma, D. I. (2012). The continuing value of twin studies in the omics era. Nature Reviews Genetics, 13(9), 640–653.

Van Rooy, D. L., Dilchert, S., Viswesvaran, C., & Ones, D. S. (2006). Multiplying intelligences: Are general, emotional, and practical intelligences equal. In A critique of emotional intelligence (pp. 235–262).

Van Rooy, D. L., & Viswesvaran, C. (2004). Emotional intelligence: A meta-analytic investigation of predictive validity and nomological net. Journal of Vocational Behavior, 65(1), 71–95.

Vasilopoulos, N., Cucina, J., Dyomina, N., Morewitz, C., & Reilly, R. (2006). Forced-choice personality tests: A measure of personality and cognitive ability? Human Performance, 19(3), 175–199.

Viswesvaran, C., & Ones, D. S. (2000). Measurement error in “Big Five Factors” personality assessment: Reliability generalization across studies and measures. Educational and Psychological Measurement, 60(2), 224–235.

von Stumm, S., & Ackerman, P. L. (2013). Investment and intellect: A review and meta-analysis. Psychological Bulletin, 139(4), 841–869. https://doi.org/10.1037/a0030746

von Stumm, S., Hell, B., & Chamorro-Premuzic, T. (2011). The hungry mind intellectual curiosity is the third pillar of academic performance. Perspectives on Psychological Science, 6(6), 574–588.

Wiernik, B. M., Kostal, J. W., Wilmot, M. P., Dilchert, S., & Ones, D. S. (2017). Empirical benchmarks for interpreting effect size variability in meta-analysis. Industrial and Organizational Psychology, 10(3), 472–479.

Wiernik, B. M., & Ones, D. S. (2017). Correcting measurement error to build scientific knowledge. Science ELetters. http://science.sciencemag.org/content/355/6325/584/tab-e-letters

Wiernik, B. M., Ones, D. S., Marlin, B. M., Giordano, C., Dilchert, S., Mercado, B. K., Stanek, K. C., Birkland, A., Wang, Y., Ellis, B., Yazar, Y., Kostal, J. W., Kumar, S., Hnat, T., Ertin, E., Sano, A., Ganesan, D. K., Choudhoury, T., & al’Absi, M. (2020). Using Mobile Sensors to Study Personality Dynamics. European Journal of Psychological Assessment.

Wilmot, M. P., & Ones, D. S. (2019). A century of research on conscientiousness at work. Proceedings of the National Academy of Sciences, 116(46), 23004–23010.

Wolf, M. B., & Ackerman, P. L. (2005). Extraversion and intelligence: A meta-analytic investigation. Personality and Individual Differences, 39(3), 531–542.

Wundt, W. (1897). Outlines of psychology (CH Judd, trans.) (C. H. Judd, Trans.; Vol. 1). Wilhelm Engelmann.