Skip to main content

Ticket QA

The "Ticket QA" action evaluates PSA tickets against a set of custom instructions to provide a quality assurance (QA) score and detailed reasoning. This helps ensure tickets are handled according to your company's standards.

Why is it useful?

This action helps maintain and improve the quality of your ticket handling by:

  • Automating QA: Systematically reviewing tickets based on your defined criteria.
  • Ensuring Consistency: Applying a standardized scoring mechanism across tickets and technicians.
  • Providing Feedback: Offering clear reasoning for QA scores, which can be used for technician coaching and process improvement.
  • Tracking Quality: Enabling the storage of QA scores in PSA custom fields for reporting and trend analysis.
info

This action is available only as a Triggered action in a workflow. Let us know if you'd like to use it as a Scheduled one.

What It Does​

The "Ticket QA" action performs the following key functions:

  • Analyzes one or more PSA tickets.
  • Uses AI to evaluate the ticket against custom instructions you provide, which define the scoring criteria and scale.
  • Generates a numerical QA score for each ticket.
  • Provides a detailed reasoning explaining how the score was derived based on the instructions.
  • Can optionally save the QA score and reasoning into a specified custom field on the ticket in your PSA.
  • Can optionally include internal notes created by Neo in its analysis.

How It Works​

The action follows a structured process to evaluate tickets:

1. Input Data​

The action takes a list of PSA tickets to be QAd and a set of instructions defining the QA criteria and scoring scale. It also considers whether to include Neo's internal notes and if/where to store the QA score.

2. AI-Powered Analysis​

For each ticket, Neo's TicketQAAgent processes the ticket content (including notes and optionally Neo's internal notes). It uses an AI model guided by a system prompt that incorporates your custom instructions. The AI then determines a score and generates a textual reasoning based on these instructions.

3. Output Structure​

The action outputs messages containing the QA score and reasoning for each ticket. If configured, it also prepares field update information to store the QA score in a custom field on the ticket.

Configuration Fields​

When you add the "Ticket QA" action to a workflow, you'll configure the following:

default_data​

The list of PSA tickets to be analyzed and scored.

instructions​

The detailed criteria and scoring scale Neo should use for the QA process. You must define how to score a ticket and on what scale (e.g., 1-5, 0-100). (default:

Score the ticket on a scale of 1-5 based on the following criteria:
- **Clarity and Completeness of Resolution (Weight: 50%)**:
  - 5: Resolution is exceptionally clear, comprehensive, and easy to follow. All steps taken are documented.
  - 4: Resolution is clear and mostly complete. Minor details might be missing but overall understandable.
  - 3: Resolution is understandable but lacks some clarity or important details.
  - 2: Resolution is difficult to understand or significantly incomplete.
  - 1: Resolution is unclear, incorrect, or missing.
- **Timeliness of Updates (Weight: 30%)**: (Assess based on note timestamps if available, otherwise focus on other criteria)
  - 5: Updates were consistently prompt and kept the user/team well-informed.
  - 4: Updates were generally timely with minor delays.
  - 3: Some noticeable delays in updates.
  - 2: Significant delays in updates.
  - 1: No updates or extremely delayed updates.
- **Professionalism and Tone (Weight: 20%)**:
  - 5: Communication is highly professional, empathetic, and courteous throughout.
  - 4: Communication is professional and courteous.
  - 3: Communication is generally acceptable but could be more professional or empathetic.
  - 2: Communication lacks professionalism or is unhelpful.
  - 1: Communication is unprofessional, rude, or inappropriate.

Provide a final score and a brief reasoning for each criterion.

)

store_score_in_custom_field​

Determines if the calculated QA score should be saved into a custom field on the ticket in your PSA. (default: False)

score_custom_field_name​

(Optional) The name of the text-based custom field in your PSA where the QA score will be stored. This field is available if store_score_in_custom_field is enabled. (default: Neo QA Score)

store_reasoning_in_custom_field​

Determines if the detailed reasoning behind the QA score should be saved into a separate custom field on the ticket in your PSA. This provides valuable context for understanding how the score was calculated. (default: False)

reasoning_custom_field_name​

(Optional) The name of the text-based custom field in your PSA where the QA reasoning will be stored. This field is available if store_reasoning_in_custom_field is enabled. (default: Neo QA Reasoning)

include_neo_internal_notes​

If enabled, Neo will include internal ticket notes created by itself during the QA analysis. This is useful if you want to QA Neo's work or check if technicians are following Neo's instructions. (default: False)

Output Fields​

This action makes the following information available to subsequent actions in the workflow:

default_fields_to_update​

A list of field update objects. If store_score_in_custom_field or store_reasoning_in_custom_field was enabled, this will include the updates for the specified custom fields with the QA score and/or reasoning.

default_messages_for_msp​

A list of messages, where each message contains the QA score and the detailed reasoning for a processed ticket. These are suitable for internal notes or notifications.

default_messages_for_dashboard​

A list of strings, summarizing the QA score and reasoning for each ticket, intended for display in the Neo Dashboard Event History.

Use Cases​

1. Post-Resolution Ticket QA​

  • Trigger: Ticket status changed to "Resolved" or "Closed".
  • Action: "Ticket QA" configured with your standard QA checklist and scoring. Both store_score_in_custom_field and store_reasoning_in_custom_field are enabled to capture the score and detailed justification.
  • Next Action (Optional): "Add Ticket Note" to append the QA score and reasoning to the ticket.
  • Next Action (Optional): "Notify Users" to alert a manager if a score is below a certain threshold.
  • Goal: Ensure all resolved tickets meet quality standards and track QA scores with detailed feedback for continuous improvement.

2. QA for Specific Ticket Categories​

  • Trigger: New ticket created.
  • Action: "Categorize Ticket" to identify if the ticket belongs to a critical category (e.g., "Major Incident", "VIP Customer").
  • Conditional Logic: If the ticket is in a critical category:
    • Action: "Ticket QA" with specific instructions tailored for that category.
  • Goal: Apply more rigorous QA to high-impact tickets.

3. Auditing Neo's Automated Actions​

  • Trigger: After a workflow where Neo performed automated actions (e.g., "L1 Engineer" sends a resolution).
  • Action: "Ticket QA" with include_neo_internal_notes enabled. Instructions focus on the quality of Neo's automated responses or actions.
  • Goal: Monitor and ensure the quality of Neo's automated interventions.

Best Practices​

  • Be Specific in Instructions: Clearly define each criterion, the scoring scale (e.g., 1-5, 1-100), and what constitutes different score levels. Vague instructions will lead to inconsistent results.
  • Use Text-Based Custom Fields: If you choose to store_score_in_custom_field or store_reasoning_in_custom_field, ensure the specified field names correspond to text-based custom fields in your PSA, as both the score and reasoning are stored as strings.
  • Store Both Score and Reasoning: Consider enabling both store_score_in_custom_field and store_reasoning_in_custom_field to capture not only the numerical score but also the detailed justification. This provides valuable context for understanding how the score was calculated and helps with technician coaching.
  • Iterate on Instructions: Start with a baseline set of QA instructions and refine them over time based on the results and feedback.
  • Educate Technicians: Share the QA criteria with your technicians so they understand how their work is being evaluated.
  • Review Event History: Regularly check the Neo Dashboard Event History to see the QA scores and reasoning. This helps in understanding how Neo is interpreting your instructions and identifying areas for process improvement.
  • Combine with Other Actions: Use "Ticket QA" in conjunction with notification actions to alert relevant personnel about QA results, especially for scores that require attention.