AI Testing & Quality Engineering — 4 Weeks | Pytest, Playwright, CI/CD

Full Curriculum

4-Week Course Curriculum

Week-by-week practical QA training — from prompt testing to CI/CD pipelines for AI-based applications.

📅 Week 1

Foundations of AI Testing + Prompt Engineering

Topics

AI Testing vs Traditional Testing: key differences
Basics of LLMs: GPT, Claude, Gemini — how they work
Key metrics: Accuracy, Hallucination, Relevance, Tone
Prompt Engineering: zero-shot, few-shot, chain-of-thought
Prompt variation testing strategies
JSON / JSONL basics for building evaluation datasets

Hands-On

Test prompts on ChatGPT / OpenAI API
Create evaluation dataset with 20–30 diverse prompts
Identify and document hallucination cases

📝 Assignment

Build Prompt Testing Sheet (Excel/JSON). Document 10 real AI response defects with severity.

📅 Week 2

API Testing + LLM Evaluation Automation

Topics

API Testing basics: HTTP methods, status codes, headers
Postman: collections, environments, assertions, pre-scripts
Python Requests & JS fetch for API automation
LLM API testing: response validation, token usage tracking
Latency measurement & performance benchmarking

Hands-On

Test OpenAI API & mock AI endpoints in Postman
Write Python scripts to validate response structure
Check keyword relevance & response time programmatically

📝 Assignment

Create complete API Test Suite (Postman + Python). Validate: status codes, response correctness, response time < 3s.

📅 Week 3

Automation + UI Testing (AI Apps)

Topics

Python Automation with pytest: fixtures, assertions, reports
Playwright / Selenium for browser automation
Testing AI-integrated UIs: chat interfaces, response rendering
AI defect taxonomy: wrong answer, toxic output, UI mismatch
Building reusable test utilities for prompt-based apps

Hands-On

Automate chat input → response validation flow
Build script to batch-test 50+ prompts automatically
Capture screenshots on failures for bug reports

📝 Assignment

Create Automation Script: input prompt list → pass/fail report. Include response time, accuracy flag, and screenshot capture.

📅 Week 4

CI/CD + Real Project + Monitoring

Topics

Jenkins basics: pipeline setup, stages, triggers
Git + GitHub workflow: branching, PRs, code review
Logs & Monitoring: CloudWatch / Datadog basics
AI Pipeline Testing: RAG overview & end-to-end validation
Jira bug reporting format for AI defects

Hands-On

Set up Jenkins pipeline running automated test suite
Integrate GitHub Actions for continuous testing
Monitor AI app logs and alert on anomalies

🏆 Final Project

Test a Chatbot OR Q&A AI System. Deliver: test cases, automation scripts, full bug report (Jira format).

Tech Stack

Tools & Technologies You'll Use

🐍Python

📬Postman

🧪Pytest

🎭Playwright

🌐Selenium

🤖OpenAI API

⚙️Jenkins

🔄GitHub Actions

📊CloudWatch / Datadog

🐛Jira

📁Git & GitHub

📄JSON / JSONL

What You'll Get

Student Deliverables

Every student completing this program receives the following tangible deliverables.

📊

Prompt Dataset (JSONL)

Curated evaluation dataset with 30+ diverse prompts

📬

API Test Collections

Postman collections with full API test suite

🐍

Python Automation Scripts

Reusable scripts for prompt & UI testing

🧪

AI Testing Framework

Basic testing framework for LLM-based applications

🏆

Certificate + Portfolio

Program certificate + GitHub project portfolio

📄

Offer Letter

Official internship offer letter from APS Solutions

FAQ

Frequently Asked Questions

Do I need prior QA or testing experience?

▼

No prior QA experience is required. We start from the basics of AI testing concepts and build up to automation. Basic Python knowledge is helpful but Week 1 covers the fundamentals needed.

Is this different from regular software testing courses?

▼

Yes — this program is specifically focused on testing AI and LLM-based products. Traditional testing courses don't cover prompt testing, hallucination evaluation, LLM API validation or AI defect taxonomy which are in high demand.

Will I test a real AI application?

▼

Yes. In Week 4 you will test a real chatbot or Q&A AI system end-to-end — creating test cases, writing automation scripts, setting up a CI/CD pipeline and delivering a full bug report in Jira format.

What job roles does this prepare me for?

▼

AI QA Engineer, LLM Tester, Automation Test Engineer, AI Product Analyst, QA Consultant for AI startups. This is a highly emerging and in-demand skill set in 2025 and beyond.

Is this available online?

▼

Yes — both online and offline at our Lucknow center. You can choose your preferred mode when you apply. Live sessions, mentor support and recordings are all included.

Apply Now

Join the AI Testing Program

Fill the form and our team will contact you within 24 hours with batch details and next steps.

📞

+91 9560078913

✉️

brajendrasingh@techvyuh.com

🌐

www.techvyuh.com

📍

Trinity Onex 3rd Floor, Office Space 5,6, Opp. Akash Enclave, Near Petrol Pump, Sector 6 Vrindavan Yojna, Lucknow — 226029

AI Testing & QualityEngineering — 4 Weeks

4-Week Course Curriculum

Tools & Technologies You'll Use

Student Deliverables

Frequently Asked Questions

Do I need prior QA or testing experience?

Is this different from regular software testing courses?

Will I test a real AI application?

What job roles does this prepare me for?

Is this available online?

Join the AI Testing Program

Become an AI QA Engineer

AI Testing & Quality
Engineering — 4 Weeks