I. Introduction to Item Response Theory

A. Definition and historical background

I. Definition of Item Response Theory (IRT)

• IRT is a mathematical and statistical approach to the analysis of data from educational and psychological tests.
• It provides a framework for understanding how individuals respond to items on a test and how their responses can be used to measure various abilities, attributes, or traits.

II. Historical Background of IRT

• IRT has its roots in the early 20th century when educational psychologists began to develop new methods for evaluating test scores.
• The earliest versions of IRT were developed in the 1930s and 1940s, but it wasn’t until the 1960s that IRT became widely adopted in the field of psychological assessment.
• IRT has since become one of the most widely used methods for test analysis and has been applied to a wide range of applications, including ability testing, personality assessment, and medical diagnosis.

B. Key concepts and assumptions

I. Key Concepts in IRT

• Item Difficulty: The degree to which an item is difficult for individuals to answer correctly.
• Item Discrimination: The degree to which an item is able to differentiate between individuals with different levels of ability.
• Item Response Function (IRF): The probability of a correct response to an item as a function of the individual’s ability.
• Ability or Latent Trait: A characteristic or dimension that is being measured by the test.

II. Assumptions of IRT

• Local Independence: The assumption that the response to one item is independent of the response to other items, given the individual’s ability.
• Unidimensionality: The assumption that the test measures a single underlying ability or dimension.
• Model Linearity: The assumption that the IRF is a linear function of the individual’s ability.
• Monotonicity: The assumption that the probability of a correct response increases as the individual’s ability increases.
• Scalability: The assumption that the test scores can be compared across different groups and across different levels of ability.

C. Types of item response models

I. Types of Item Response Models

• Unidimensional IRT Models: Models that assume that the test measures a single underlying ability or dimension. Examples include:
  • Rasch Model
  • One-parameter logistic model (1PL)
  • Two-parameter logistic model (2PL)
  • Three-parameter logistic model (3PL)
• Multidimensional IRT Models: Models that allow for the measurement of multiple underlying abilities or dimensions. Examples include:
  • Generalized Partial Credit Model (GPCM)
  • Graded Response Model (GRM)
  • Nominal Response Model (NRM)
• Hybrid IRT Models: Models that incorporate elements of both unidimensional and multidimensional models.

II. Characteristics of Different IRT Models

• Unidimensional Models:
  • Simple to estimate and interpret
  • Can provide insight into the difficulty and discrimination of items
  • Assume that the test measures a single underlying ability
• Multidimensional Models:
  • More flexible in terms of the types of data they can handle
  • Can provide a more nuanced understanding of individual differences
  • Assume that the test measures multiple underlying abilities or dimensions
• Hybrid Models:
  • Offer the best of both worlds, combining the simplicity of unidimensional models with the flexibility of multidimensional models.
  • Can provide a more comprehensive understanding of test data than either unidimensional or multidimensional models alone.

II. Unidimensional IRT Models

A. Rasch Model

I. Introduction to the Rasch Model

• The Rasch Model is a type of unidimensional IRT model.
• It was first developed by Danish mathematician Georg Rasch in the 1960s.

II. Key Assumptions of the Rasch Model

• The Rasch Model assumes that the test measures a single underlying ability.
• It assumes that the item response function (IRF) is a logistic function that is the same for all individuals.
• The Rasch Model also assumes that the difficulty of each item is a fixed parameter, and that the ability of each individual is a random variable.

III. Characteristics of the Rasch Model

• The Rasch Model is simple to estimate and interpret.
• It provides a clear understanding of item difficulty and item discrimination.
• It can be used to measure ability across different groups of individuals.
• The Rasch Model is limited in that it assumes that the test measures a single underlying ability, and that the item response function is a logistic function.

IV. Applications of the Rasch Model

• The Rasch Model is widely used in educational and psychological assessment to evaluate test scores and to develop tests.
• It is used to measure ability and attribute levels in a variety of domains, including academic ability, health outcomes, and personality traits.
• The Rasch Model is also used to construct computerized adaptive tests (CATs) and to analyze data from large-scale assessment programs.

B. One-parameter logistic model (1PL)

I. Introduction to the One-parameter Logistic Model (1PL)

• The One-parameter Logistic Model (1PL) is a type of unidimensional IRT model.
• It is a simpler version of the Two-parameter Logistic Model (2PL).

II. Key Assumptions of the 1PL

• The 1PL assumes that the test measures a single underlying ability.
• It assumes that the item response function (IRF) is a logistic function with a single parameter that reflects the difficulty of the item.
• The 1PL assumes that the discrimination of the item is constant across all levels of ability.

III. Characteristics of the 1PL

• The 1PL is simple to estimate and interpret.
• It provides a clear understanding of item difficulty.
• It is limited in that it does not account for differences in item discrimination across different levels of ability.
• The 1PL is most appropriate for tests where item difficulty is the main focus of the analysis.

IV. Applications of the 1PL

• The 1PL is widely used in educational and psychological assessment to evaluate test scores and to develop tests.
• It is used to measure ability and attribute levels in a variety of domains, including academic ability, health outcomes, and personality traits.
• The 1PL is also used to analyze data from large-scale assessment programs.
• It is a useful model for cases where the focus is on item difficulty, and where the discrimination of the items is assumed to be constant across different levels of ability.

C. Two-parameter logistic model (2PL)

I. Introduction to the Two-parameter Logistic Model (2PL)

• The Two-parameter Logistic Model (2PL) is a type of unidimensional IRT model.
• It is a more complex version of the One-parameter Logistic Model (1PL).

II. Key Assumptions of the 2PL

• The 2PL assumes that the test measures a single underlying ability.
• It assumes that the item response function (IRF) is a logistic function with two parameters: the difficulty and the discrimination of the item.
• The 2PL assumes that the discrimination of the item varies with the level of ability.

III. Characteristics of the 2PL

• The 2PL provides a more nuanced understanding of item difficulty and item discrimination than the 1PL.
• It can be used to examine how the discrimination of items changes with different levels of ability.
• The 2PL is more complex to estimate and interpret than the 1PL.
• It is appropriate for tests where both item difficulty and item discrimination are of interest.

IV. Applications of the 2PL

• The 2PL is widely used in educational and psychological assessment to evaluate test scores and to develop tests.
• It is used to measure ability and attribute levels in a variety of domains, including academic ability, health outcomes, and personality traits.
• The 2PL is also used to analyze data from large-scale assessment programs.
• It is a useful model for cases where both item difficulty and item discrimination are of interest, and where the discrimination of the items is expected to vary with different levels of ability.

D. Three-parameter logistic model (3PL)

I. Introduction to the Three-parameter Logistic Model (3PL)

• The Three-parameter Logistic Model (3PL) is a type of unidimensional IRT model.
• It is a more complex version of the Two-parameter Logistic Model (2PL).

II. Key Assumptions of the 3PL

• The 3PL assumes that the test measures a single underlying ability.
• It assumes that the item response function (IRF) is a logistic function with three parameters: the difficulty, discrimination, and the guessing parameter of the item.
• The 3PL assumes that the discrimination of the item varies with the level of ability, and that the guessing parameter reflects the chance of a correct response due to guessing.

III. Characteristics of the 3PL

• The 3PL provides a comprehensive understanding of item difficulty, item discrimination, and the influence of guessing on test scores.
• It is a useful model for tests where guessing is an important consideration.
• The 3PL is more complex to estimate and interpret than the 1PL or 2PL.
• It is appropriate for tests where both item difficulty and item discrimination are of interest, and where the discrimination of the items is expected to vary with different levels of ability, and where guessing may impact test scores.

IV. Applications of the 3PL

• The 3PL is widely used in educational and psychological assessment to evaluate test scores and to develop tests.
• It is used to measure ability and attribute levels in a variety of domains, including academic ability, health outcomes, and personality traits.
• The 3PL is also used to analyze data from large-scale assessment programs.
• It is a useful model for cases where both item difficulty, item discrimination, and the influence of guessing are of interest, and where the discrimination of the items is expected to vary with different levels of ability, and where guessing may impact test scores.

III. Multidimensional IRT Models

A. Generalized Partial Credit Model (GPCM)

I. Introduction to the Generalized Partial Credit Model (GPCM)

• The Generalized Partial Credit Model (GPCM) is a type of IRT model that can be used to analyze ordinal data, such as multi-choice test scores or rating scales.
• It is an extension of the Partial Credit Model (PCM) that allows for a more flexible modeling of the item response function (IRF).

II. Key Assumptions of the GPCM

• The GPCM assumes that the test measures a single underlying ability.
• It assumes that the item response function (IRF) is a smooth monotonic function with a flexible shape, allowing for more complex and nuanced representations of the item response data.
• The GPCM assumes that the score received by the examinee on the item reflects their underlying ability level.

III. Characteristics of the GPCM

• The GPCM provides a flexible and comprehensive approach to modeling ordinal data, allowing for a more nuanced understanding of the relationship between ability and item responses.
• It is a useful model for tests where the item response function is expected to be more complex than can be captured by other IRT models.
• The GPCM is more complex to estimate and interpret than other IRT models, and requires a greater number of parameters to be estimated.

IV. Applications of the GPCM

• The GPCM is widely used in educational and psychological assessment to evaluate test scores and to develop tests.
• It is used to measure ability and attribute levels in a variety of domains, including academic ability, health outcomes, and personality traits.
• The GPCM is also used to analyze data from large-scale assessment programs.
• It is a useful model for cases where the item response function is expected to be more complex than can be captured by other IRT models, and where a more nuanced understanding of the relationship between ability and item responses is of interest.

B. Graded Response Model (GRM)

I. Introduction to the Graded Response Model (GRM)

• The Graded Response Model (GRM) is a type of IRT model that can be used to analyze data from tests that consist of items with multiple response categories.
• It is specifically designed to handle data from tests with items that have more than two response options.

II. Key Assumptions of the GRM

• The GRM assumes that the test measures a single underlying ability.
• It assumes that the item response function (IRF) is a step function that corresponds to the different response categories of the item.
• The GRM assumes that the score received by the examinee on the item reflects their underlying ability level.

III. Characteristics of the GRM

• The GRM provides a comprehensive approach to modeling data from tests with items that have more than two response categories.
• It is a useful model for tests where the response categories are ordinal, and where the response patterns of the examinees can be analyzed in terms of their underlying ability level.
• The GRM is more complex to estimate and interpret than other IRT models, and requires a greater number of parameters to be estimated.

IV. Applications of the GRM

• The GRM is widely used in educational and psychological assessment to evaluate test scores and to develop tests.
• It is used to measure ability and attribute levels in a variety of domains, including academic ability, health outcomes, and personality traits.
• The GRM is also used to analyze data from large-scale assessment programs.
• It is a useful model for cases where the items have more than two response categories, and where the response patterns of the examinees can be analyzed in terms of their underlying ability level.

C. Nominal Response Model (NRM)

I. Introduction to the Nominal Response Model (NRM)

• The Nominal Response Model (NRM) is a type of IRT model that can be used to analyze data from tests that consist of items with nominal response categories.
• It is specifically designed to handle data from tests with items that have only two response categories (e.g., true/false, correct/incorrect).

II. Key Assumptions of the NRM

• The NRM assumes that the test measures a single underlying ability.
• It assumes that the item response function (IRF) is a logistic function that corresponds to the two response categories of the item.
• The NRM assumes that the score received by the examinee on the item reflects their underlying ability level.

III. Characteristics of the NRM

• The NRM provides a simple and straightforward approach to modeling data from tests with items that have only two response categories.
• It is a useful model for tests where the response categories are binary, and where the response patterns of the examinees can be analyzed in terms of their underlying ability level.
• The NRM is one of the most widely used IRT models due to its simplicity and ease of interpretation.

IV. Applications of the NRM

• The NRM is widely used in educational and psychological assessment to evaluate test scores and to develop tests.
• It is used to measure ability and attribute levels in a variety of domains, including academic ability, health outcomes, and personality traits.
• The NRM is also used to analyze data from large-scale assessment programs.
• It is a useful model for cases where the items have only two response categories, and where the response patterns of the examinees can be analyzed in terms of their underlying ability level.

IV. Model Estimation and Validation

Model Estimation and Validation refers to the process of using statistical methods to determine the parameters of a statistical model that best fit the data being analyzed. The goal of model estimation is to find the values of the model’s parameters that provide the best fit to the data, based on a specified criterion. Model validation involves evaluating the fit of the estimated model to the data, to ensure that the model is appropriate for the data and that the parameter estimates are reasonable. Model validation is an important step in the model building process, as it provides evidence of the reliability and validity of the model’s predictions.

A. Maximum Likelihood Estimation (MLE)

I. Introduction to Maximum Likelihood Estimation (MLE)

• Maximum Likelihood Estimation (MLE) is a statistical method for estimating the parameters of a statistical model that best fit the data.
• The goal of MLE is to find the parameter values that maximize the likelihood of the observed data, given the model.

II. Key Characteristics of MLE

• MLE is a commonly used method for estimating parameters in IRT models, including the 1PL, 2PL, 3PL, and NRM.
• MLE is based on the idea of finding the parameter values that make the observed data as probable as possible, given the model.
• MLE provides a flexible and efficient way to estimate parameters, as it can handle complex models and large datasets.

III. Steps in MLE

• Define the statistical model and specify the likelihood function.
• Choose an initial starting value for the parameters.
• Iteratively update the parameters to maximize the likelihood function.
• Repeat the above steps until the maximum likelihood estimate is found.

IV. Advantages of MLE

• MLE provides an efficient way to estimate parameters and handle complex models.
• MLE provides a well-founded statistical framework for parameter estimation.
• MLE is widely used in a variety of fields, including psychology, biology, and economics.

V. Limitations of MLE

• MLE requires a large sample size in order to provide accurate estimates.
• MLE can be computationally intensive, especially for complex models and large datasets.
• MLE assumes that the model and the likelihood function are correctly specified, which may not always be the case.

B. Model fit indices and goodness-of-fit tests

I. Introduction to Model Fit Indices and Goodness-of-Fit Tests

• Model fit indices and goodness-of-fit tests are used to evaluate the fit of an IRT model to the data.
• These measures provide information about how well the model represents the data and how well the estimated parameters fit the data.

II. Key Characteristics of Model Fit Indices and Goodness-of-Fit Tests

• Model fit indices and goodness-of-fit tests are used to evaluate the fit of IRT models, including the 1PL, 2PL, 3PL, GPCM, and NRM.
• Different fit indices and tests may be more appropriate for different models and datasets.
• Model fit indices and goodness-of-fit tests should be used in conjunction with other methods, such as residual analysis, to assess the fit of the model.

III. Types of Model Fit Indices

• Chi-Square Test of Fit
• Root Mean Squared Error of Approximation (RMSEA)
• Comparative Fit Index (CFI)
• Tucker-Lewis Index (TLI)
• Akaike Information Criterion (AIC)
• Bayesian Information Criterion (BIC)

IV. Steps in Evaluating Model Fit

• Choose appropriate model fit indices and goodness-of-fit tests for the IRT model being used.
• Calculate the fit indices and goodness-of-fit statistics for the estimated model.
• Compare the fit indices and statistics to established standards or cutoff values.
• Use the results of the fit indices and goodness-of-fit tests to determine whether the model fits the data adequately.

V. Advantages of Model Fit Indices and Goodness-of-Fit Tests

• Model fit indices and goodness-of-fit tests provide a quantitative assessment of the fit of the model to the data.
• These measures can be used to compare the fit of different models and to select the best model for the data.
• Model fit indices and goodness-of-fit tests can provide valuable information for model improvement and refinement.

VI. Limitations of Model Fit Indices and Goodness-of-Fit Tests

• Model fit indices and goodness-of-fit tests may not always provide a complete picture of the fit of the model to the data.
• These measures can be sensitive to sample size and distributional assumptions, which can affect the results.
• Model fit indices and goodness-of-fit tests should be used in conjunction with other methods, such as residual analysis, to assess the fit of the model.

C. Model selection and comparison

I. Introduction to Model Selection and Comparison

• Model selection and comparison is an important step in the application of item response theory (IRT).
• This process involves selecting the most appropriate IRT model for a given dataset and comparing the fit of different models to determine the best fit.

II. Key Characteristics of Model Selection and Comparison

• Model selection and comparison may involve comparing the fit of different models, including the 1PL, 2PL, 3PL, GPCM, GRM, and NRM.
• Different models may be more appropriate for different datasets and research questions.
• Model selection and comparison should be guided by established standards and methods, such as model fit indices and goodness-of-fit tests.

III. Steps in Model Selection and Comparison

• Select a set of candidate IRT models for the data, based on the research question and the characteristics of the data.
• Estimate the parameters of each candidate model and calculate model fit indices and goodness-of-fit tests.
• Compare the fit of the models, using established standards and cutoffs for model fit indices and goodness-of-fit tests.
• Select the best-fitting model, based on the results of the model fit indices and goodness-of-fit tests.

IV. Advantages of Model Selection and Comparison

• Model selection and comparison provides a systematic way to select the best-fitting IRT model for a given dataset.
• This process helps to ensure that the most appropriate model is selected and that the results are accurate and reliable.
• Model selection and comparison can provide valuable information for model improvement and refinement.

V. Limitations of Model Selection and Comparison

• Model selection and comparison may not always lead to a clear winner, as different models may fit the data similarly well.
• The results of model selection and comparison may be sensitive to the choice of models, the estimation method, and the sample size.
• Model selection and comparison should be used in conjunction with other methods, such as residual analysis and cross-validation, to assess the fit of the model.

V. Applications of IRT in Psychological Assessment

I. Introduction to Applications of IRT in Psychological Assessment

• Item response theory (IRT) has a wide range of applications in psychological assessment, from the development of educational and occupational tests to the measurement of mental health and personality traits.

II. Key Applications of IRT in Psychological Assessment

• Item banking: IRT can be used to develop large banks of assessment items that can be used to create customized tests for a wide range of applications.
• Test development: IRT can be used to develop new tests and improve existing tests, by selecting items that are highly discriminatory and validly measuring the construct of interest.
• Computerized adaptive testing (CAT): IRT can be used to develop computerized adaptive tests that dynamically adjust the difficulty of the items based on the response of the test-taker.
• Item scoring and scoring accuracy: IRT can be used to improve item scoring and scoring accuracy, by taking into account the response patterns of individual test-takers.

III. Advantages of Using IRT in Psychological Assessment

• IRT can provide a more precise and accurate measurement of psychological constructs than other methods, such as classical test theory.
• IRT can reduce the impact of measurement error, by taking into account the individual response patterns of test-takers.
• IRT can provide valuable information for test improvement and refinement, by identifying the strengths and weaknesses of individual items.
• IRT can increase the efficiency of psychological assessment, by reducing the number of items required to obtain an accurate measurement.

IV. Limitations of Using IRT in Psychological Assessment

• IRT requires a large sample size and a high level of expertise to implement, which may limit its practicality for some applications.
• IRT may not be suitable for all types of psychological constructs, as some constructs may not be easily measured by IRT.
• IRT may not be appropriate for some cultural or ethnic groups, as it may not take into account the unique response patterns of these groups.
• The results of IRT may be sensitive to the choice of model, the estimation method, and the sample size, and may require additional methods, such as residual analysis and cross-validation, to validate the results.

A. Ability and attribute measurement

I. Introduction to Ability and Attribute Measurement

• Ability and attribute measurement refers to the use of IRT to measure psychological traits or abilities, such as cognitive abilities, personality traits, and emotional states.

II. Ability Measurement using IRT

• Ability measurement using IRT is a method for measuring an individual’s ability on a particular construct, such as intelligence, mathematical ability, or reading ability.
• The measurement of ability using IRT requires a set of items that are designed to measure the construct of interest, and the responses to these items are used to estimate an individual’s ability on that construct.

III. Attribute Measurement using IRT

• Attribute measurement using IRT is a method for measuring an individual’s level of a particular attribute, such as personality traits, emotional states, or attitudes.
• The measurement of attributes using IRT requires a set of items that are designed to measure the attribute of interest, and the responses to these items are used to estimate an individual’s level of that attribute.

IV. Advantages of Ability and Attribute Measurement using IRT

• Ability and attribute measurement using IRT can provide a more precise and accurate measurement of psychological constructs than other methods, such as classical test theory.
• IRT can provide valuable information for test improvement and refinement, by identifying the strengths and weaknesses of individual items.
• IRT can increase the efficiency of psychological assessment, by reducing the number of items required to obtain an accurate measurement.

V. Limitations of Ability and Attribute Measurement using IRT

• Ability and attribute measurement using IRT may not be suitable for all types of psychological constructs, as some constructs may not be easily measured by IRT.
• IRT may not be appropriate for some cultural or ethnic groups, as it may not take into account the unique response patterns of these groups.
• The results of IRT may be sensitive to the choice of model, the estimation method, and the sample size, and may require additional methods, such as residual analysis and cross-validation, to validate the results.

B. Test construction and item bank development

I. Introduction to Test Construction and Item Bank Development

• Test construction and item bank development refer to the process of designing and developing a psychological assessment tool using IRT.

II. Steps in Test Construction and Item Bank Development

• Define the construct to be measured and establish the goals of the assessment tool.
• Choose the appropriate IRT model for the construct to be measured.
• Write a set of items that will measure the construct.
• Pilot test the items to assess their psychometric properties and refine the item bank as needed.
• Estimate the IRT model parameters for each item.
• Evaluate the goodness-of-fit of the IRT model to the data.
• Select the final set of items to be included in the item bank.

III. Considerations in Test Construction and Item Bank Development

• The quality of the items is a critical factor in the accuracy of the IRT measurement.
• The number of items in the item bank should be sufficient to provide an accurate measurement of the construct.
• The difficulty level of the items should be appropriate for the target population.
• The content of the items should be relevant and appropriate for the target population.
• The language and format of the items should be accessible to the target population.

IV. Advantages of Test Construction and Item Bank Development using IRT

• Test construction and item bank development using IRT can result in a more efficient and effective assessment tool, as the item bank can be used for multiple assessments and can be updated as needed.
• IRT can provide valuable information for test improvement and refinement, by identifying the strengths and weaknesses of individual items.
• Test construction and item bank development using IRT can provide a more precise and accurate measurement of psychological constructs.

V. Limitations of Test Construction and Item Bank Development using IRT

• The process of test construction and item bank development using IRT may require significant time and resources, and may require expertise in IRT and psychometrics.
• The results of IRT may be sensitive to the choice of model, the estimation method, and the sample size, and may require additional methods, such as residual analysis and cross-validation, to validate the results.

C. Item calibration and differential item functioning (DIF) analysis

I. Introduction to Item Calibration and Differential Item Functioning (DIF) Analysis

• Item calibration refers to the estimation of item parameters in IRT models, which describe the difficulty and discrimination of items.
• Differential item functioning (DIF) refers to the phenomenon in which items behave differently for different subgroups of examinees based on characteristics such as gender, ethnicity, or language.

II. Item Calibration

• Item calibration involves estimating the item parameters of an IRT model, such as the difficulty, discrimination, or guessing parameters, based on the item response data.
• The goal of item calibration is to obtain a set of parameters that accurately describe the properties of the items.

III. Differential Item Functioning (DIF) Analysis

• DIF analysis involves comparing the performance of different subgroups of examinees on a set of items to determine if some items are functioning differently for different subgroups.
• The goal of DIF analysis is to identify items that may be biased towards or against specific subgroups of examinees, and to make appropriate modifications to the items or the assessment process to minimize the impact of DIF.

IV. Methods for Item Calibration and DIF Analysis

• Item calibration and DIF analysis can be performed using various statistical techniques, such as chi-square goodness-of-fit tests, logistic regression models, and Bayesian methods.
• The choice of method depends on the specific goals and requirements of the assessment, and the characteristics of the data.

V. Advantages of Item Calibration and DIF Analysis

• Item calibration and DIF analysis can improve the accuracy and fairness of psychological assessments by identifying and correcting items that may be biased towards or against specific subgroups of examinees.
• Item calibration and DIF analysis can provide valuable information for test improvement and refinement, by identifying the strengths and weaknesses of individual items.
• Item calibration and DIF analysis can increase the reliability and validity of psychological assessments by reducing measurement error and improving the representativeness of the assessment results.

VI. Limitations of Item Calibration and DIF Analysis

• Item calibration and DIF analysis may require significant time and resources, and may require expertise in IRT, psychometrics, and data analysis.
• The results of item calibration and DIF analysis may be sensitive to the choice of model, the estimation method, and the sample size, and may require additional methods, such as residual analysis and cross-validation, to validate the results.
• The presence of DIF may not necessarily indicate bias in an item, and may reflect true differences in the abilities or characteristics of the subgroups being compared.

VI. Advanced Topics in IRT

A. Computerized adaptive testing (CAT)

• Introduction: Definition and Overview
  • CAT is a type of assessment method where the difficulty level of test items is adjusted in real-time based on the responses of the examinee.
  • The goal of CAT is to estimate the examinee’s ability level as accurately as possible with a minimum number of items.
• Advantages:
  • Increased efficiency and accuracy of measurement
  • Ability to provide immediate feedback
  • Ability to administer tests to large populations
  • Reduced time and cost
• Basic components:
  • Item pool: A large collection of test items with known difficulty levels
  • Item selection algorithm: A procedure that selects items based on the examinee’s responses to previous items
  • Ability estimation algorithm: A procedure that calculates the examinee’s ability level based on the responses to the selected items.
• Item selection algorithms:
  • Maximum Information (MaxInf)
  • Bayesian
  • Fisher Information
• Ability estimation algorithms:
  • Maximum Likelihood (ML)
  • Bayesian
  • Maximum a Posteriori (MAP)
• Implementations and examples:
  • CAT for standardized tests (e.g. GRE, SAT)
  • CAT for diagnostic and therapeutic assessments (e.g. mental health assessments)
  • CAT for certification and licensing exams (e.g. medical licensure exams)
• Limitations and Challenges:
  • Need for high-quality item pools
  • The need for accurate ability estimation algorithms
  • Computational demands
  • Challenges in ensuring fairness and bias-free assessments

B. IRT models for categorical and ordinal data

• Introduction:
  • Item response theory (IRT) can be applied to both continuous and categorical (or ordinal) data.
  • Categorical and ordinal data refer to response formats where individuals choose from a set of discrete options, such as multiple-choice or Likert-scale items.
• Key differences between IRT models for continuous and categorical data:
  • Continuous data models assume that the latent trait of interest is a continuous variable.
  • Categorical data models assume that the latent trait of interest is a categorical or ordinal variable.
• Common IRT models for categorical and ordinal data:
  • Nominal Response Model (NRM)
  • Graded Response Model (GRM)
  • Generalized Partial Credit Model (GPCM)
• Nominal Response Model (NRM):
  • Assumes that the response categories are independent of each other.
  • Does not account for the ordinal relationship between response categories.
• Graded Response Model (GRM):
  • Accounts for the ordinal relationship between response categories.
  • Models the probability of a response as a monotonically increasing function of the latent trait.
• Generalized Partial Credit Model (GPCM):
  • Accounts for the ordinal relationship between response categories.
  • Models the response as a combination of underlying partial credit scores.
  • Provides a more flexible approach to modeling responses than the GRM.
• Applications:
  • Assessment of attitudes and opinions (e.g. Likert-scale items)
  • Assessment of educational outcomes (e.g. multiple-choice tests)
  • Assessment of psychological and behavioral traits (e.g. personality assessments)
• Challenges:
  • Model specification and estimation
  • The need for high-quality item pools
  • The need for robust methods of item calibration and differential item functioning (DIF) analysis.

C. Bayesian IRT models

• Introduction:
  • Bayesian IRT models are a class of item response theory (IRT) models that incorporate Bayesian statistical methods.
  • Bayesian IRT models provide a flexible and computationally efficient approach to modeling item responses.
• Key features of Bayesian IRT models:
  • Incorporate prior knowledge about the parameters of interest.
  • Provide posterior distributions for the parameters, allowing for estimation and inference under uncertainty.
  • Offer a variety of model selection and comparison methods.
• Common Bayesian IRT models:
  • One-parameter logistic model (1PL)
  • Two-parameter logistic model (2PL)
  • Three-parameter logistic model (3PL)
• Advantages of Bayesian IRT models:
  • Ability to incorporate prior knowledge and information about the parameters.
  • Improved handling of missing data and sparse data.
  • Increased flexibility in model selection and comparison.
  • Ability to incorporate hierarchical models for multiple groups or multilevel data structures.
• Challenges:
  • Requirement for good prior knowledge or selection of appropriate prior distributions.
  • Need for computational resources to estimate and compare models.
  • Potential for slow convergence or estimation problems.
• Applications:
  • Assessment of educational outcomes (e.g. multiple-choice tests)
  • Assessment of psychological and behavioral traits (e.g. personality assessments)
  • Assessment of attitudes and opinions (e.g. Likert-scale items)
• Best practices:
  • Careful selection of prior distributions to avoid biasing results.
  • Comparison of different model specifications to ensure adequate fit.
  • Consideration of multiple sources of information and data to inform prior distributions.