Interview Questions for Interpretation and Use of Assessment Results

Norm-referenced and criterion-referenced assessments differ fundamentally in how they interpret scores. Norm-referenced assessments compare an individual’s performance to that of a larger group (the norm group), typically using standardized tests. The goal is to determine how the individual ranks relative to others. Think of a percentile score – it tells you what percentage of the norm group scored below a particular individual. For example, a percentile score of 80 means the individual scored better than 80% of the norm group.

Criterion-referenced assessments, on the other hand, focus on measuring performance against a predetermined standard or criterion. These assessments aim to determine what an individual knows or can do, regardless of how others perform. A driver’s license test is a perfect example; you’re not graded against other test-takers, but rather on whether you meet the minimum criteria for safe driving. The score reflects mastery of specific skills, not relative standing.

In short: Norm-referenced tests tell you where someone ranks compared to others, while criterion-referenced tests tell you what someone knows or can do.

Identifying and addressing bias in assessment instruments is crucial for ensuring fairness and accuracy. This involves a multi-step process:

• Careful Item Analysis: Each item on the assessment should be scrutinized for potentially biased language, imagery, or content. For instance, using culturally specific examples or terminology could disadvantage certain groups. We need to ensure items are relevant and equally accessible to everyone.
• Reviewing the Norm Group (if applicable): For norm-referenced tests, the composition of the norm group is critical. A representative sample reflecting the diversity of the population being assessed is essential to prevent bias. If the norm group doesn’t represent the test takers, the results will be skewed.
• Differential Item Functioning (DIF) Analysis: Statistical techniques like DIF analysis help identify items that function differently for various subgroups (e.g., males vs. females, different ethnic groups). Items showing DIF might need to be revised or removed.
• Bias Review Panel: Involving experts with diverse backgrounds and perspectives in the review process is highly beneficial. A panel can identify subtle biases that might be missed by a single reviewer. For example, a panel may provide feedback on a test’s culturally appropriate images and language.
• Ongoing Monitoring and Evaluation: Bias is not a one-time fix. Regularly reviewing and updating assessments ensures they remain fair and unbiased over time. Data analysis of test results can highlight potential biases that may emerge after the test is used.

Addressing bias involves rewriting or removing problematic items, revising instructions, and ensuring fair testing conditions. The goal is to create an assessment that provides an accurate and unbiased measure of the construct being assessed for all test-takers.

Throughout my career, I’ve extensively utilized various assessment types, including:

• Cognitive Assessments: These measure intellectual abilities, such as intelligence and reasoning skills (e.g., Wechsler Adult Intelligence Scale – WAIS, Raven’s Progressive Matrices). I’ve used these to assess cognitive strengths and weaknesses in clinical settings and for educational placement.
• Personality Assessments: These explore personality traits, styles, and preferences (e.g., Myers-Briggs Type Indicator – MBTI, Big Five Inventory). I’ve utilized these for team building, career counseling, and employee selection, always emphasizing the limitations of such assessments as indicators of behavior.
• Aptitude Assessments: These gauge potential abilities or talents in specific areas (e.g., mechanical aptitude tests, verbal reasoning tests). I have used these in selection processes for roles requiring specific skills, like engineering or technical writing. My interpretations always included the consideration of other assessment data to provide a comprehensive perspective.

My experience spans various settings, including educational institutions, corporate environments, and clinical practices. I always prioritize using assessments that are appropriate for the specific context and purpose, ensuring that the results are interpreted in a comprehensive and nuanced manner, considering the limitations of each assessment.

Confidentiality and security of assessment data are paramount. My approach is multi-faceted:

• Secure Storage: Assessment data is stored in encrypted databases, with access limited to authorized personnel only. Physical security measures are in place to protect paper records (if any).
• Data Anonymization: Whenever possible, I anonymize data to remove personally identifiable information. This ensures individual privacy while still allowing for meaningful analysis.
• Compliance with Regulations: I strictly adhere to relevant data privacy regulations (e.g., HIPAA, GDPR) ensuring the protection of individuals’ rights and sensitive data.
• Informed Consent: Before administering any assessment, I obtain informed consent, clearly explaining the purpose, procedures, and implications of the assessment to participants. This includes outlining how their data will be used and stored.
• Training and Procedures: All individuals with access to assessment data receive appropriate training on data security protocols and ethical guidelines.

My commitment to data security extends beyond technical measures. It involves a strong ethical framework that prioritizes the privacy and well-being of all participants.

Relying on a single assessment type to make significant decisions about candidates is a risky strategy. Assessments are tools, not the sole source of information. Several limitations exist:

• Limited Scope: Each assessment type measures a specific aspect of an individual. For instance, a cognitive ability test doesn’t reveal personality traits or motivation. Using only one type can lead to an incomplete understanding of a candidate’s overall suitability.
• Methodological Biases: Different assessment types are prone to different kinds of biases. Relying on one method ignores the potential for bias inherent in that particular method. The use of multiple methods can often help to reveal those biases.
• Test-Taker Factors: Individual factors like test anxiety, cultural background, or physical health can affect performance. A single assessment might not capture the true potential of a candidate due to these factors.
• Lack of Contextual Information: Assessments alone don’t always provide the complete picture. Other crucial information like experience, references, and work samples are needed for a holistic evaluation.

A comprehensive approach involves using multiple assessments, interviews, and background checks, creating a much more robust and fair evaluation process.

Standard scores, percentiles, and other statistical measures are crucial for interpreting assessment results. Let’s look at them:

• Standard Scores: These scores represent how far an individual’s performance deviates from the mean of a norm group, expressed in standard deviation units (e.g., z-scores, T-scores). A higher standard score indicates a higher performance relative to the group.
• Percentiles: A percentile score indicates the percentage of individuals in the norm group who scored below a particular individual. A percentile of 75 means the individual scored better than 75% of the group.
• Other Measures: Other measures include raw scores (the number of correct items), scaled scores (transformed scores allowing for comparisons across different tests), and stanines (a standard nine-point scale).

Interpreting these scores requires careful consideration of the specific assessment and its norms. It’s essential to understand the distribution of scores and the meaning of different ranges within the scale. For example, a standard score of 115 on an IQ test indicates a significantly higher score than the average (100), whereas a percentile of 50 means the individual scored at the average.

Reliability and validity are cornerstones of assessment quality. Reliability refers to the consistency of an assessment. A reliable assessment will produce similar results under consistent conditions. For example, if a person takes the same test twice under similar conditions, their scores should be relatively close. We evaluate reliability through various methods, such as test-retest reliability (consistency over time), internal consistency (consistency of items within the test), and inter-rater reliability (consistency across different raters).

Validity refers to the extent to which an assessment measures what it is intended to measure. A valid assessment accurately reflects the construct it aims to assess. For instance, a test designed to measure mathematical ability should actually measure mathematical ability, not reading comprehension. We evaluate validity through content validity (does the test cover the relevant content?), criterion validity (does the test correlate with other measures of the same construct?), and construct validity (does the test measure the theoretical construct it’s intended to measure?).

In practice, I look for assessments with strong psychometric properties, including evidence of both reliability and validity. I examine the technical manuals and research supporting the assessments to ensure their suitability for the intended purpose. A high reliability score doesn’t automatically mean high validity. Both are important for accurate and useful assessment results.

Integrating assessment results with other information is crucial for a holistic view of a candidate. Think of it like building a puzzle; assessments provide one piece, but resumes and interviews offer other crucial components.

I start by analyzing the assessment data objectively, identifying strengths and weaknesses according to the specific assessment’s design and norms. Then, I cross-reference these findings with the resume, verifying skills and experience claims. For instance, if an assessment highlights strong problem-solving skills, I’d look for evidence of this in their work history or projects described in their resume. The interview allows me to delve deeper. I’d use the assessment insights to guide my interview questions, probing specific areas where the assessment showed either exceptional ability or areas for development. For example, if the assessment indicates a weakness in teamwork, I would ask behavioral interview questions designed to assess collaborative skills in real-world scenarios.

Finally, I weigh the information from each source, considering their relative strengths and limitations. A resume might highlight achievements but not reveal actual work style. An interview can offer insights into personality and motivation, but it’s susceptible to social desirability bias. The assessment data provides a more objective measure of skills and aptitudes. The combination of these sources creates a much richer and more reliable picture.

My experience encompasses a wide variety of assessment feedback, including:

• Norm-referenced feedback: This compares an individual’s score to a larger group’s performance, providing a percentile rank. This is helpful for understanding where a candidate stands relative to others.
• Criterion-referenced feedback: This compares an individual’s score to a predetermined standard or benchmark, indicating whether they meet specific criteria. This is useful for determining competency levels against specific job requirements.
• Qualitative feedback: This involves narrative descriptions of behavior and performance, often from simulations, role-plays, or structured interviews. This adds context and richness to quantitative scores.
• 360-degree feedback: Gathering feedback from multiple sources (supervisors, peers, subordinates) provides a comprehensive view of an individual’s strengths and weaknesses in a work environment. This is particularly useful in leadership assessments.

I’m proficient in interpreting the nuances of each type, understanding their limitations, and integrating them to form a complete picture.

Explaining complex assessment results requires tailoring the communication to the audience’s level of understanding. I avoid jargon and use clear, concise language. For someone with limited knowledge, I’d focus on the key takeaways, using analogies to make abstract concepts understandable. For example, instead of explaining percentile ranks, I might say, “Your score places you in the top 25% of candidates for this skill.”

For individuals with more technical expertise, I can delve deeper, explaining statistical concepts and providing detailed breakdowns of the assessment’s methodology and psychometric properties. Visual aids like charts and graphs can be extremely helpful for everyone. Crucially, I ensure that the explanation focuses on the implications of the results for the individual and the job role, rather than just the raw scores.

In all cases, I encourage questions and create a safe space for open dialogue. This two-way communication ensures the individual understands the results and feels supported throughout the process.

Discrepancies between assessment results and other information require careful investigation. It’s rarely a case of simply dismissing one piece of information in favor of another. Instead, I follow a systematic approach:

• Review the assessment methodology: Ensure the assessment was administered and scored correctly, and that it’s appropriate for the specific job requirements. Are there any limitations to the assessment?
• Re-examine the other data: Scrutinize the resume and interview notes for potential biases or inaccuracies. Were the interview questions effective? Were there inconsistencies in the candidate’s responses?
• Seek additional information: Could further data collection resolve the conflict? This might include reference checks, additional interviews, or even a different type of assessment.
• Consider contextual factors: Were there any external factors (e.g., stress, illness) that may have impacted the assessment performance? How does the candidate’s overall performance profile factor in?

Ultimately, I aim to build a coherent narrative that accounts for all available data. Sometimes, the discrepancy highlights important information about the candidate’s strengths and weaknesses, prompting a deeper understanding that would have been missed otherwise.

Ethical considerations are paramount in the use and interpretation of assessment results. The key principles include:

• Fairness and equity: Assessments should be unbiased and free from discrimination based on race, gender, religion, or other protected characteristics. This requires careful selection of assessments and ensuring they are appropriately validated for diverse populations.
• Transparency and informed consent: Candidates should be fully informed about the purpose and nature of the assessments, and provide their consent before participation. They should also have access to their results and an explanation of their meaning.
• Confidentiality and privacy: Assessment data is sensitive and must be handled with strict confidentiality. Only authorized personnel should have access to the results.
• Purposeful use: Assessments should be used only for their intended purpose and not for purposes that are not job-related. Results should never be used to discriminate or unfairly penalize individuals.
• Accuracy and validity: It’s crucial to use reliable and valid assessments to ensure accurate and meaningful results, using assessments that are regularly updated and have established psychometric properties.

Adherence to these ethical principles ensures fairness, promotes trust, and protects the rights of individuals.

In a previous role, I interpreted assessment data for a senior management position. The assessment suggested a candidate, ‘Sarah,’ had moderate leadership potential, but her interview and resume portrayed her as a highly successful and driven leader. This discrepancy initially raised concerns. I meticulously reviewed the assessment, confirming the scoring and methodology were sound. I then revisited Sarah’s interview and resume, discovering that her accomplishments were largely within a very specific, highly controlled environment, lacking the broader leadership challenges typical of the senior role. The assessment, although not highlighting exceptional leadership across the board, did reveal a strength in adaptability.

During the hiring committee meeting, I presented my findings, emphasizing the assessment’s strengths and limitations, and how Sarah’s accomplishments reflected specific situational factors. I explained that her adaptability score was actually a critical asset for the new role, requiring leadership in more ambiguous and varied contexts. My detailed explanation, demonstrating my comprehensive understanding of the assessment data, and the ability to integrate multiple data points persuaded the committee, ultimately leading to her selection and her subsequent success in the role.

Staying current in this field is critical. I achieve this through several strategies:

• Professional development: I regularly attend conferences and workshops focused on assessment and selection, often organized by professional bodies like SHRM or SIOP.
• Professional memberships: My membership in professional organizations provides access to cutting-edge research, publications, and networking opportunities.
• Continuing education: I pursue certifications and training programs to expand my knowledge of new assessment methods and best practices. I also actively engage in relevant online courses.
• Journal articles and research: I regularly review relevant academic journals and research papers to stay abreast of the latest findings in assessment methodology and interpretation.
• Networking: I engage in professional discussions with colleagues and experts in the field, exchanging insights and learning from their experiences.

This continuous learning ensures that my practices remain aligned with the highest standards of the profession.

I’m proficient in several software and tools for assessment data analysis. My core competency lies in using statistical software packages like SPSS and R. SPSS offers a user-friendly interface for conducting various statistical analyses, including descriptive statistics, correlations, t-tests, and ANOVAs, which are crucial for understanding assessment results. R, on the other hand, provides a more flexible and powerful environment for advanced statistical modeling and custom data visualizations. I also have experience with dedicated psychometric software such as IRTPRO and Winsteps, which are specifically designed for analyzing item response theory (IRT) models, allowing for a more nuanced understanding of item difficulty and examinee ability.

Beyond statistical software, I’m adept at using spreadsheet programs like Microsoft Excel and Google Sheets for data cleaning, manipulation, and creating basic visualizations. For reporting and presentation, I utilize tools such as PowerPoint and Tableau to effectively communicate findings to stakeholders.

Ensuring fair and unbiased assessments is paramount. This involves a multi-faceted approach. Firstly, I carefully review the assessment content for any potential bias related to gender, race, ethnicity, age, or disability. This includes examining wording, imagery, and examples used in the assessment to ensure they are culturally sensitive and inclusive.

Secondly, I utilize statistical techniques to detect differential item functioning (DIF). DIF analysis identifies items that function differently for various subgroups of examinees, indicating potential bias. For example, an item might be significantly harder for female candidates than for male candidates, even if their overall ability levels are similar. Items exhibiting DIF are then reviewed and either revised or removed.

Thirdly, I consider the assessment environment. Factors such as testing location, time constraints, and the clarity of instructions can influence performance. Providing clear and consistent instructions, equitable testing conditions, and appropriate accommodations for candidates with disabilities are crucial for fairness.

Finally, regular review and updates of assessments are essential to ensure they remain current and free from bias. This is an ongoing process involving careful monitoring and analysis of assessment data.

I have extensive experience using assessment data to drive training and development programs. In a previous role, we used pre- and post-training assessments to evaluate the effectiveness of a leadership development program. The pre-assessment identified areas where participants needed improvement, such as communication skills or conflict resolution. The post-assessment demonstrated significant improvements in these areas after the training, proving its impact. This data informed future iterations of the program, allowing us to refine content and delivery methods based on the actual learning outcomes.

In another instance, we used performance data from 360-degree feedback assessments to create targeted development plans for individual employees. These plans focused on specific skills and behaviors identified as needing improvement. This personalized approach, guided by assessment data, led to increased employee engagement and improved overall performance.

Test anxiety and other extraneous factors can significantly influence assessment performance. To mitigate these effects, I employ several strategies. First, I ensure that the assessment instructions are clear, concise, and easy to understand. This reduces uncertainty and minimizes stress.

Secondly, I provide ample time for candidates to complete the assessment, reducing the pressure of time constraints. If appropriate, I offer practice tests or sample questions to help candidates become familiar with the format and content of the assessment.

Furthermore, I understand that creating a comfortable and supportive testing environment is crucial. This includes providing a quiet space with minimal distractions. For individuals known to experience significant test anxiety, I explore the possibility of reasonable accommodations, such as extended time or a separate testing environment. Finally, I thoroughly analyze the data for outliers or unusual patterns that could indicate factors beyond the candidate’s abilities, such as unusual levels of omissions or rapid response times, which could be signs of test anxiety or other issues.

Selecting the right assessment method depends heavily on the specific job role and selection purpose. There’s no one-size-fits-all solution. For example, if I’m assessing for a customer service role, I might use a combination of methods. A structured interview would assess communication skills and empathy, while a situational judgment test could evaluate decision-making under pressure. A personality inventory may provide insight into traits like extraversion and agreeableness, which are relevant to the role.

For a technical role requiring programming skills, a practical coding test would be essential. For leadership positions, assessment centers offering simulations and role-plays may be more appropriate. My approach involves carefully analyzing the job description and identifying the key competencies and skills required. I then select assessment methods that can accurately and reliably measure these traits. The choice also considers factors like cost, time constraints, and the availability of resources.

My experience encompasses a wide range of statistical methods for analyzing assessment data. I routinely use descriptive statistics (means, standard deviations, frequencies) to summarize and understand the distribution of scores. Inferential statistics, such as t-tests and ANOVAs, help me determine if there are significant differences between groups. Correlation analysis helps understand the relationships between different assessment measures.

For more sophisticated analysis, I employ regression techniques to predict performance outcomes based on assessment scores. Factor analysis helps identify underlying constructs or latent traits measured by the assessment. I also have expertise in applying item response theory (IRT) models to analyze individual item characteristics and examinee abilities, enabling more precise and nuanced interpretations of assessment data.

Differential Item Functioning (DIF) refers to the phenomenon where an item on an assessment functions differently for different subgroups of test-takers, even when they have the same underlying ability. For example, an item might be easier for men than women, despite both groups having the same overall score. This suggests bias in the item.

Identifying DIF usually involves using statistical techniques such as Mantel-Haenszel or logistic regression. These methods compare the performance of different subgroups on each item after controlling for overall ability. If a significant difference is found, the item exhibits DIF.

Addressing DIF involves several steps. First, the item needs careful scrutiny to identify potential sources of bias in the item’s wording, imagery, or context. The item might be revised to remove any ambiguity or potentially offensive content. If the item cannot be adequately revised, it might be removed from the assessment. Regular monitoring and analysis of DIF in assessments are crucial for ensuring fairness and validity.

Interpreting and utilizing assessment results in a team setting requires a collaborative and transparent approach. It’s not simply about presenting raw data; it’s about facilitating a shared understanding and leveraging the insights to make informed decisions.

• Data Sharing and Discussion: I begin by sharing the assessment results with the team in a clear and concise manner, ensuring everyone understands the methodology and limitations of the assessments used. This often involves visualizations like charts and graphs to make the data more accessible.

• Collaborative Interpretation: We then engage in a facilitated discussion, exploring the patterns and trends revealed by the data. Each team member contributes their perspective, bringing in their knowledge of the candidates or team members being assessed. This avoids bias from any single individual’s interpretation.

• Contextualization: It’s crucial to consider the context of the assessment. For example, a low score on a specific skill might be offset by strong performance in other areas, or explained by external factors like lack of experience in that particular domain. The team collaboratively analyzes the data in the context of the specific role and individual circumstances.

• Actionable Insights: The goal is not just to understand the data but to translate it into actionable steps. This might involve targeted training, assigning individuals to roles that align with their strengths, or providing mentorship to address skill gaps.

For example, if a team assessment reveals a lack of collaborative skills within a group, we can address this by implementing team-building activities or providing training on effective communication strategies. This collaborative approach ensures buy-in from the team and leads to more effective interventions.

Several pitfalls can hinder the accurate interpretation of assessment data. Avoiding these is crucial for making fair and effective decisions.

• Confirmation Bias: This is the tendency to interpret data in a way that confirms pre-existing beliefs. To mitigate this, I employ a structured approach to data analysis, focusing on objective measures and involving multiple team members in the interpretation process to ensure diverse perspectives.

• Over-Reliance on a Single Assessment: Relying on only one type of assessment can lead to an incomplete picture. A comprehensive approach utilizes multiple assessment methods to gain a holistic understanding of the individual’s abilities and potential.

• Ignoring Contextual Factors: As mentioned earlier, neglecting external factors that might influence assessment results, such as test anxiety or cultural differences, can lead to inaccurate conclusions. It is important to consider these factors when interpreting the data.

• Misinterpreting Correlations as Causation: Just because two variables correlate doesn’t mean one causes the other. Carefully examining the relationship between variables is essential to avoid drawing inaccurate conclusions.

• Lack of Validity and Reliability: Using assessments that lack validity (measuring what they intend to measure) and reliability (providing consistent results) is a major flaw. Always ensure the assessments used have been rigorously validated and demonstrate high reliability.

In a recent hiring process for a senior project manager role, we used a combination of cognitive ability tests, personality assessments, and situational judgment tests. One candidate scored exceptionally high on the cognitive tests, indicating strong analytical and problem-solving skills. However, their personality assessment revealed a tendency towards micromanagement and a lack of teamwork. The situational judgment test further supported this, showing poor decision-making in collaborative scenarios.

Based on this holistic interpretation, we opted not to hire this candidate despite their impressive cognitive abilities. We recognized that while strong analytical skills are essential for a project manager, the lack of collaborative skills and tendency towards micromanagement would significantly hinder their effectiveness in the role. Hiring a candidate with a better balance of cognitive abilities and interpersonal skills ultimately proved to be a more successful decision.

Ensuring legal compliance in assessment practices involves adhering to relevant legislation and guidelines. This includes:

• Fairness and Non-Discrimination: Assessments must be designed and implemented in a way that does not discriminate against any protected group (e.g., based on race, gender, religion, age, disability). This requires careful consideration of the assessment content and the process used to administer and interpret the results. Regular audits help ensure compliance.

• Job-Relatedness: Assessments must measure skills and abilities that are directly relevant to the job requirements. This involves conducting a thorough job analysis to identify the key competencies required for successful job performance, ensuring the assessment methods accurately reflect these competencies.

• Privacy and Data Security: Protecting the privacy of candidates’ data is crucial. This means complying with data protection laws and regulations, such as GDPR or CCPA. This involves storing data securely, only collecting necessary information, and obtaining informed consent from candidates.

• Accommodation for Disabilities: Reasonable accommodations must be provided to candidates with disabilities to ensure they have equal opportunities to demonstrate their abilities. This might involve modifying the assessment format or providing assistive technology.

• Transparency and Communication: Candidates should be informed about the assessment process, the purpose of the assessments, and how the results will be used. This ensures transparency and allows candidates to ask questions and address any concerns.

Regular training for assessment users on legal compliance and best practices is also essential to avoid unintentional violations.

These three assessment types tap into different aspects of a candidate’s potential:

• Cognitive Ability Tests: These measure general mental abilities such as reasoning, problem-solving, and verbal and numerical comprehension. Examples include Raven’s Progressive Matrices or the Wonderlic Personnel Test. They are often used to predict overall job performance across a variety of roles.

• Personality Tests: These assess personality traits, such as conscientiousness, extraversion, and agreeableness. Examples include the Myers-Briggs Type Indicator (MBTI) or the Big Five personality inventory. These tests help understand an individual’s work style, preferences, and how they might interact with others. It’s crucial to use validated personality tests and avoid making assumptions based solely on personality traits.

• Situational Judgment Tests (SJTs): These present candidates with realistic work scenarios and ask them to choose the best course of action. They assess judgment, decision-making, and problem-solving skills in context. SJTs are particularly useful in assessing practical skills and how candidates apply their knowledge in real-world situations.

The key difference lies in what each test measures: cognitive tests measure general mental abilities, personality tests assess traits, and SJTs evaluate judgment and decision-making in context. A comprehensive assessment strategy often utilizes a combination of these methods for a more complete understanding of a candidate’s strengths and weaknesses.

Balancing objective assessment data with subjective information requires a nuanced approach. Objective data from assessments provides a quantifiable measure of skills and abilities, while subjective information (e.g., interview feedback, references) offers context, nuances, and qualitative insights.

I use a weighted approach, where the weight given to each source of information depends on the specific context and the reliability of the information. For instance, highly reliable and valid assessments might carry more weight than less structured interview feedback.

I create a holistic profile of the candidate by integrating both objective and subjective data. Discrepancies between the two are investigated further to understand the underlying reasons. For example, if a candidate scores high on a leadership assessment but receives negative feedback on leadership skills during an interview, this discrepancy needs further investigation. Maybe the interview focused on a specific leadership style not measured by the assessment, or perhaps the interview itself was biased.

Ultimately, the decision-making process involves thoughtful consideration of all available data, prioritizing the reliability and validity of the sources. A clear and transparent rationale should be documented for every decision based on the integrated data, ensuring a fair and defensible process.