QA Automation Engineer (Senior)

<ul>
<li>Build evaluation suites for LLM and ML features: curated test sets, scoring against reference answers, LLM-as-judge where it is justified, and human review where it is not.</li>
<li>Define what "good enough" means with the delivery team before launch, in numbers a client can accept, and hold releases to it.</li>
<li>Test retrieval pipelines end to end, covering chunking and indexing, retrieval quality, grounding of the final answer, and behaviour when the right document is simply not there.</li>
<li>Write regression tests that tolerate non-deterministic output, using semantic similarity, structured-output validation, tolerance bands, and repeated sampling rather than exact string matching.</li>
<li>Cover failure modes that matter to clients: hallucination, prompt injection through user or document content, leakage of data across tenants or sessions, refusal and over-refusal, and degradation on Thai-language input.</li>
<li>Track cost and latency alongside accuracy, so that a quality improvement that triples token spend is visible before it ships.</li>
<li>Detect drift after release by comparing production samples against the baseline, and raise it early rather than after the client does.</li>
<li>Design and maintain automated test suites for APIs, services, data pipelines, and web applications built for client projects.</li>
<li>Run tests in CI, keep them fast and stable, and treat a flaky suite as a defect in its own right.</li>
<li>Set the test strategy at the start of an engagement: what gets automated, what stays manual, what the exit criteria are, and what evidence the client will receive.</li>
</ul><h2>Requirements</h2><ul>
<li>
<strong>4-5 years in test automation</strong>, QA engineering, or software engineering with substantial testing ownership.</li>
<li>Strong programming ability in<strong>Python</strong>, or in TypeScript or Java with the willingness to work primarily in Python.</li>
<li>Hands-on experience<strong>testing an LLM or ML-based product</strong>in production or close to it.</li>
<li>Practical command of at least one automation framework such as Pytest, Playwright, Cypress, or RestAssured, and experience running suites in CI.</li>
<li>Working knowledge of API and data testing, including SQL and validating data through a pipeline.</li>
<li>The judgement to decide what deserves automation and what does not, and to say so when a suite is producing noise rather than signal.</li>
<li>Clear written and spoken communication in Thai and English.</li>
</ul><h2>Benefits</h2><ul>
<li>
<strong>Hybrid Working</strong>options with<strong>Flexible Working Hours</strong>.</li>
<li>
<strong>Your Choice of Gear:</strong>Mac or Windows? You pick your preferred work equipment.</li>
<li>Comprehensive Group Health Insurance.</li>
<li>Monthly<strong>Grab Transportation</strong>credits.</li>
<li>Job-related training courses and certificate fee coverage.</li>
<li>
<strong>10+ days of annual leave</strong>.</li>
<li>Free snacks and drinks available at the office.</li>
</ul>