AQA Engineer with LLM Testing experience

1 неделя назад

Украина, Украина CommIT Полный рабочий день 40 000 € - 80 000 € в год

The company is a pioneering AI-driven research company providing in-depth stock analysis globally in multiple languages. Our mission is to bridge the knowledge gap in the investment world, empowering investors with AI-powered insights. By integrating our advanced AI technology into brokerage platforms and financial institutions, we enable millions of investors to make informed investment decisions.

We are looking for a Mid-level Automation QA Engineer to join our team in Poland. The role will primarily focus on API testing (80%) with UI testing (20%), alongside testing LLM-powered AI solutions. You will help design, enhance, and maintain automation frameworks, ensuring efficient and scalable testing—especially for AI/LLM-driven features.

Key Responsibilities:

Automation Framework & Strategy

  • Contribute to designing and improving automation testing strategies.
  • Develop and maintain automation frameworks for API, UI, and LLM validation.
  • Ensure scalability, maintainability, and reusability of automation code.

Test Development & Execution

  • API Testing: Develop automated API tests in Python using the Requests library.
  • UI Testing: Create and maintain UI tests with Playwright.
  • LLM Testing: Design and execute automated test suites for **Large Language Model (LLM)**based features, including prompt validation, output accuracy, and bias/error detection.
  • Integrate automated tests with Postman collections where applicable.
  • Execute performance, regression, and functional tests.

Continuous Integration & Deployment

  • Integrate automation tests into CI/CD pipelines (Jenkins preferred).
  • Work with Docker and Kubernetes to manage and scale test environments.

Collaboration & Documentation

  • Collaborate with developers, QA, and DevOps teams to improve the QA process.
  • Write test plans, strategies, and design documents.
  • Track issues using Jira and manage test cases (Xray optional).
  • Analyze logs, investigate failures, and troubleshoot in staging and production.
Requirements
  • 5 years of experience in test automation or a related QA role.
  • Strong programming skills in Python.
  • Solid experience in API automation (Requests library) and UI automation (Playwright).
  • Hands-on experience testing LLM-based applications (prompt engineering, output evaluation, hallucination/bias testing).
  • Experience with CI/CD tools (Jenkins preferred).
  • Experience with Docker & Kubernetes for test environments.
  • Strong analytical, problem-solving, and debugging skills.
  • Experience with Jira for tracking and reporting.
  • Ability to work independently and proactively in a fast-paced environment.

Nice to Have:

  • Experience with Kafka.
  • Familiarity with Xray for test management.
  • Knowledge of AWS cloud services.
  • Experience with ML/AI model evaluation metrics.

  • Junior AQA Engineer

    1 неделя назад

    Украина, Украина SOMBRA Полный рабочий день 20 000 ₴ - 60 000 ₴ в год

    Junior AQA Engineer (Python)Hybrid, Office, RemoteUkraineOur client is solar and energy storage technology company specializing in developing and manufacturing solar battery systems, hybrid inverters, and chargers.Requirements:Strong experience with Python.Hands-on experience with API testing, Pytest, and Playwright.Focus on test coverage, combined with...

  • AQA Tech Lead

    2 недель назад

    Украина, Украина CommIT Полный рабочий день 30 000 ₴ - 60 000 ₴ в год

    We are seeking an experienced Automation QA (AQA) Tech Lead with strong management and technical leadership skills. The ideal candidate will have hands-on expertise in test automation frameworks, deep knowledge of Playwright, GitHub Copilot, and the ability to set up automation environments from scratch (0 → production). You will be responsible for...

  • Trainee AQA

    1 неделя назад

    Украина, Украина Eleks Полный рабочий день 20 000 ₴ - 25 000 ₴ в год

    ELEKS Quality Assurance Office is looking for a Trainee AQA in our offices in Ukraine. ABOUT PROJECT The company is spread all over the world, but the primary location is the USA. It is committed to helping financial institutions worldwide grow assets efficiently with its innovative portfolio management suite of data management, investment accounting,...

  • AI Data Engineer

    3 дней назад

    Украина, Украина Seven-Pro Полный рабочий день 300 000 ₴ - 600 000 ₴ в год

    About projectThe customer's company implements several client projects across various industries, working in a practical and dynamic environment. One of our client`s projects has several initiatives that are already in progress with a functioning infrastructure, while some new pipelines, microservices, and integrations need to be built from the ground up....

  • Automation QA Engineer

    7 дней назад

    Украина, Украина Transcenda Полный рабочий день 80 000 ₴ - 120 000 ₴ в год

    Own your futureOur culture isn't something people join, it's something they build and shape. We believe that every person deserves to be heard and empowered. If you're on the fence about whether you're a fit, we say go for it. Let's build something great together.‍‍Must haveStrong QA automation expertise, combined with solid manual testing...

  • Automation QA Engineer

    5 дней назад

    Украина, Украина Seven-Pro Полный рабочий день 40 000 ₴ - 80 000 ₴ в год

    About projectOur client's project is an advanced cybersecurity platform that helps organizations protect their AI, GenAI, and LLM-based systems. The platform enables companies to manage AI-related risks through a native security layer that integrates seamlessly into their infrastructure. It performs two key functions — risk assessment and AI firewall —...

  • AI/ML Engineer

    2 недель назад

    Украина, Украина Genesis Полный рабочий день 250 000 ₴ - 500 000 ₴ в год

    At Promova, we're redefining language education to make it accessible, personal, and effective for today's fast-paced world. Our growing team of 170 professionals is on a mission to connect people, bridge cultures, and empower lifelong learners — reaching every country except aggressor states (yes, even Antarctica). We blend AI-driven innovation with...

  • Java Engineer

    1 день назад

    Украина, Украина Globaldev Group Полный рабочий день 120 000 ₴ - 240 000 ₴ в год

    As an Experienced Backend Engineer at Windward, you will hold a crucial role in designing, developing, and maintaining our data platform, supporting critical data-driven decision-making processes across the organization.You will collaborate closely with cross-functional teams, including data scientists, analysts, and software engineers, ensuring smooth data...

  • Senior GenAI Engineer

    2 недель назад

    Украина, Украина SOMBRA Полный рабочий день 40 000 ₴ - 80 000 ₴ в год

    Senior GenAI EngineerHybrid, RemoteUkraineWe are seeking a highly skilled Senior Generative AI Specialist to design, develop, and deploy advanced AI solutions, including AI agents and multi-agent systems, leveraging the latest Generative AI technologies. The ideal candidate will have deep expertise in LLMs, NLP, and modern AI frameworks, with a strong...

  • Software Engineer in Test

    2 недель назад

    Украина, Украина AMC Bridge Полный рабочий день 40 000 ₴ - 80 000 ₴ в год

    POSITION DESCRIPTION:AMC Bridge is contributing to the development of a cloud-based platform for real-time design and construction collaboration. The system is built with TypeScript and React, follows microservices architecture on AWS, and utilizes Playwright for automated testing. The platform is already in production and used by a limited number of...