AQA Engineer with LLM Testing experience

2 недель назад

Украина, Украина CommIT Полный рабочий день 40 000 € - 80 000 € в год

The company is a pioneering AI-driven research company providing in-depth stock analysis globally in multiple languages. Our mission is to bridge the knowledge gap in the investment world, empowering investors with AI-powered insights. By integrating our advanced AI technology into brokerage platforms and financial institutions, we enable millions of investors to make informed investment decisions.

We are looking for a Mid-level Automation QA Engineer to join our team in Poland. The role will primarily focus on API testing (80%) with UI testing (20%), alongside testing LLM-powered AI solutions. You will help design, enhance, and maintain automation frameworks, ensuring efficient and scalable testing—especially for AI/LLM-driven features.

Key Responsibilities:

Automation Framework & Strategy

  • Contribute to designing and improving automation testing strategies.
  • Develop and maintain automation frameworks for API, UI, and LLM validation.
  • Ensure scalability, maintainability, and reusability of automation code.

Test Development & Execution

  • API Testing: Develop automated API tests in Python using the Requests library.
  • UI Testing: Create and maintain UI tests with Playwright.
  • LLM Testing: Design and execute automated test suites for **Large Language Model (LLM)**based features, including prompt validation, output accuracy, and bias/error detection.
  • Integrate automated tests with Postman collections where applicable.
  • Execute performance, regression, and functional tests.

Continuous Integration & Deployment

  • Integrate automation tests into CI/CD pipelines (Jenkins preferred).
  • Work with Docker and Kubernetes to manage and scale test environments.

Collaboration & Documentation

  • Collaborate with developers, QA, and DevOps teams to improve the QA process.
  • Write test plans, strategies, and design documents.
  • Track issues using Jira and manage test cases (Xray optional).
  • Analyze logs, investigate failures, and troubleshoot in staging and production.
Requirements
  • 5 years of experience in test automation or a related QA role.
  • Strong programming skills in Python.
  • Solid experience in API automation (Requests library) and UI automation (Playwright).
  • Hands-on experience testing LLM-based applications (prompt engineering, output evaluation, hallucination/bias testing).
  • Experience with CI/CD tools (Jenkins preferred).
  • Experience with Docker & Kubernetes for test environments.
  • Strong analytical, problem-solving, and debugging skills.
  • Experience with Jira for tracking and reporting.
  • Ability to work independently and proactively in a fast-paced environment.

Nice to Have:

  • Experience with Kafka.
  • Familiarity with Xray for test management.
  • Knowledge of AWS cloud services.
  • Experience with ML/AI model evaluation metrics.

  • Junior AQA Engineer

    2 недель назад

    Украина, Украина SOMBRA Полный рабочий день 20 000 ₴ - 60 000 ₴ в год

    Junior AQA Engineer (Python)Hybrid, Office, RemoteUkraineOur client is solar and energy storage technology company specializing in developing and manufacturing solar battery systems, hybrid inverters, and chargers.Requirements:Strong experience with Python.Hands-on experience with API testing, Pytest, and Playwright.Focus on test coverage, combined with...

  • Trainee AQA

    1 неделя назад

    Украина, Украина Eleks Полный рабочий день 20 000 ₴ - 25 000 ₴ в год

    ELEKS Quality Assurance Office is looking for a Trainee AQA in our offices in Ukraine. ABOUT PROJECT The company is spread all over the world, but the primary location is the USA. It is committed to helping financial institutions worldwide grow assets efficiently with its innovative portfolio management suite of data management, investment accounting,...

  • AI Data Engineer

    4 дней назад

    Украина, Украина Seven-Pro Полный рабочий день 300 000 ₴ - 600 000 ₴ в год

    About projectThe customer's company implements several client projects across various industries, working in a practical and dynamic environment. One of our client`s projects has several initiatives that are already in progress with a functioning infrastructure, while some new pipelines, microservices, and integrations need to be built from the ground up....

  • Automation QA Engineer

    1 неделя назад

    Украина, Украина Transcenda Полный рабочий день 80 000 ₴ - 120 000 ₴ в год

    Own your futureOur culture isn't something people join, it's something they build and shape. We believe that every person deserves to be heard and empowered. If you're on the fence about whether you're a fit, we say go for it. Let's build something great together.‍‍Must haveStrong QA automation expertise, combined with solid manual testing...

  • Automation QA Engineer

    6 дней назад

    Украина, Украина Seven-Pro Полный рабочий день 40 000 ₴ - 80 000 ₴ в год

    About projectOur client's project is an advanced cybersecurity platform that helps organizations protect their AI, GenAI, and LLM-based systems. The platform enables companies to manage AI-related risks through a native security layer that integrates seamlessly into their infrastructure. It performs two key functions — risk assessment and AI firewall —...

  • Java Engineer

    2 дней назад

    Украина, Украина Globaldev Group Полный рабочий день 120 000 ₴ - 240 000 ₴ в год

    As an Experienced Backend Engineer at Windward, you will hold a crucial role in designing, developing, and maintaining our data platform, supporting critical data-driven decision-making processes across the organization.You will collaborate closely with cross-functional teams, including data scientists, analysts, and software engineers, ensuring smooth data...

  • Manual QA Engineer

    2 дней назад

    Украина, Украина Solvd Полный рабочий день 30 000 ₴ - 60 000 ₴ в год

    Solvd Inc. is a rapidly growing AI-native consulting and technology services firm delivering enterprise transformation across cloud, data, software engineering, and artificial intelligence. We work with industry-leading organizations to design, build, and operationalize technology solutions that drive measurable business outcomes.Following the acquisition of...

  • Junior–Middle QA Engineer

    6 дней назад

    Украина, Украина MGID Полный рабочий день 20 000 ₴ - 25 000 ₴ в год

    Job DescriptionMGID is a global advertising platform that helps publishers monetize their audiences and enables brands to promote their products and services across the open web effectively. Using AI-powered technology, we deliver high-quality native, display, and video ads in brand-safe environments, balancing user experience and performance.Every month,...

  • Middle/Senior QA Engineer

    1 неделя назад

    Украина, Украина Softengi Полный рабочий день 900 000 ₴ - 1 200 000 ₴ в год

    Description of the project:High-load project in the telecom industry.Your qualification:4+ years of experience as a QA engineer with a focus on the backend side;Critical thinking and strong requirements analysis skills;Quick learning and desire for self-development;Strong knowledge and real experience with API testing;Experience with performance...

  • Junior Test Automation Engineer

    2 недель назад

    Украина, Украина SoftServe Полный рабочий день 250 000 ₴ - 350 000 ₴ в год

    DIRECTION:SOFTWARE TESTINGTECHNICAL LEVEL:JUNIORLOCATION:UKRAINEPERSONAL RECRUITER:OLESIA LOTOTSKAWE ARESoftServe is a team of thinkers and doers, true professionals and good people, solving business challenges with innovative technology solutions.Our customer is an American company that develops software to help businesses manage their networks, systems,...