ISTQB Certified Tester Advanced Level - Test Automation Engineering CTAL-TAE (Syllabus v2.0) Questions and Answers
You have been recruited as a TAE by a large travel agency and are now responsible for a mature TAS that is dedicated to functional testing of the company's booking system, which covers flights, hotels, and packaged holidays. This application contains a complex UI and a lot of functionality. It is undergoing constant maintenance and enhancement by an Agile team that adds tests to the regression test suite at the end of every iteration.
The TAS supports structured scripting and this has been implemented to a high standard, with a well-managed script library and minimal code duplication. The TAA will not support any more advanced scripting techniques. The test automation that has been achieved so far was well received at first but is now the target of increasing criticism. It is perceived as being old-fashioned, poorly documented and, in particular:
1. The test script maintenance overhead is becoming excessive because the test analysts frequently want data values to be changed.
2. Test analysts are complaining that a new test which is similar to an existing one, except for data values, requires a new high-level script, and that they must ask a technical test analyst to do this for them.
3. The nightly full regression run is taking a long time and there is a risk that, as more tests are added, it will soon take too long to run in the available window.
4. Some tests break frequently because of a slow response from the SUT.
5. Some transactions are difficult to verify because they rely on variables that cannot be seen through the UI of the SUT.
6. The UI of the TAF is difficult to learn.
Your new manager wants quick improvements to address items 1, 2 and 3. She has accepted that the TAA may need to be enhanced to support the improvements.
Which TWO improvements will BEST fit this scenario?
a) Transition the automation to data-driven testing
b) Transition the automation to keyword-driven testing
c) Adopt a set of standard verification methods
d) Review the test automation scripts for functional duplication
e) Implement hard-coded waits in scripts that have been affected by slow SUT response
You are working for a clothing retailer that has big stores in many cities. Your TAS, for the first time, has just been used for reliability and stress testing of the overnight batch processing that collects sales transactions from the stores and adds them to a data warehouse; functional suitability tests have been reused for these runs.
All tests have finished normally but their results have not been satisfactory, and it has been difficult to understand the nature of the defect or defects that have been causing the failures.
What data, if added to the test logs, would MOST likely help with debugging in this situation?
a) Dynamic data about the status of the test platforms
b) Screenshots taken during test execution
c) Crash dumps and stack traces
d) Values of randomly generated test input data
e) Counts of the quantity of test cases executed during each run
You are working as a TAE for a small start-up insurance broker and your next project will be a new system for generating motor insurance quotes. The project will follow an Agile methodology. The stakeholders are keen to get it into production, but are realistic about the need to allow enough time for thorough testing and support automation of tests where appropriate.
Which approach for automating the tests would be MOST suitable for this project?
Which statement about static analysis is TRUE?
In a first possible implementation, the automated test scripts within a suite locate and interact with elements of a web UI indirectly through the browsers using browser-specific drivers and APIs, provided by an automated test tool used as part of the TAS. In an alternative implementation, these test scripts locate and interact with elements of the same web UI directly at the HTML level by accessing the DOM (Document Object Model) and internal JavaScript code. The first possible implementation:
Which of the following recommendations can help improve the maintainability of test automation code?
Which of the following statements about the relationship between TAA, TAS and TAF is true?
(Which of the following aspects of “design for testability” is MOST directly associated with the need to define precisely which interfaces are available in the SUT for test automation at different test levels?)
An API's response to a request made to the corresponding endpoint should return some specific data about a payment transaction in JSON format. In particular, your goal is to write the test automation code, keeping it as short as possible, aimed at determining whether that response includes certain properties (transaction_id, amount, status, timestamp) with the data types and formats expected. Assuming that the TAF provides all the necessary support to validate the specified API response, how would you BEST achieve your goal?
Consider a TAS aimed at implementing and running automated test scripts at the UI level on web apps. The TAS must support cross-browser compatibility for a variety of supported browsers, by ensuring that the same test script will run on such browsers in the same way without making any changes to it. This is achieved by introducing appropriate abstractions into the TAA for connection and interaction with different browsers. Because of this, the TAS will be able to make direct calls to the supported browsers using each different browser’s native support for automation. Which of the following SOLID principles was adopted?
As a TA-E, you have successfully verified that a test automation environment and all other components of the TAS are working as expected. Now your goal is to verify the correct behavior for a given automated test suite that will be run by the TAS. Which of the following should NOT be part of the verifications aimed at achieving your goal?
Which statement about layering in a test automation framework is TRUE?
Which of the following information in API documentation is LEAST relevant for implementing automated tests on that API?
The last few runs for a suite of automated keyword-driven tests on a SUT were never completed. The test where the run was aborted was not the same between runs. Currently, it is not possible to identify the root cause of these aborts, but only determine that test execution aborted when exceptions (e.g., NullPointerException, OutOfMemoryError) occurred on the SUT by analyzing its log files. Test execution log files are currently generated, in HTML format, by the TAS as follows: all expected logging data is logged for each keyword in intermediate log files. This data is then inserted into the final log file only for keywords that fail, while only a configurable subset of that data is logged for keywords that execute successfully. Which of the following actions (assuming it is possible to perform all of them) would you take FIRST to help find the root cause of the aborts?
A suite of automated test cases was run multiple times on the same release of the SUT in the same test environment. Consider analyzing a test histogram that shows the distribution of test results (pass, fail, etc.) for each test case across these runs. Which of the following potential issues is MOST likely to be identified as a result of such an analysis?
Which statements about a continuous integration/continuous deployment (CI/CD) pipeline are TRUE?
a) The pipeline can contain configuration tests for the TAF/TAS.
b) The pipeline cannot contain configuration tests for the TAF/TAS.
c) Component and component integration tests should be run separately.
d) The pipeline can contain system integration tests.
e) The pipeline cannot contain system integration tests.
Consider choosing an approach for the automated implementation of manual regression test suites written at the UI level for some already developed web apps. The TAS is based on a programming language that allows the creation of test libraries and provides a capture/playback feature that allows recognition and interaction with all widgets in the web UIs being tested. The automated tests will be implemented by team members with strong programming skills. The chosen approach should aim to reduce both the effort required to maintain automated tests and the effort required to add new automated tests. Which of the following approaches would you choose?
Your TAS is being used for automated functional suitability testing of a stock management system. Several tests have crashed. Despite the fact that the SUT creates detailed logs of all its activity, it has been difficult to determine the reason(s) for these crashes from the TAS and SUT test logs.
What actions are MOST likely to help with debugging?
a) Ensure that the TAS logs a count of how many times each test case is run.
b) Ensure that the SUT logs all available configuration data when it starts up.
c) Ensure that the SUT logs all available crash dumps and stack traces in the event of failure.
d) Ensure that the SUT logs all values of randomly generated test input data.
e) Ensure exact time synchronization of all test logs before any tests are run.
Which of the following should be done when checking the composition of the test suite?
Which ONE of the four layers of the test automation architecture provides an optional capability?
Which of the following statements about a test progress report produced for an automated test suite is TRUE?
Automated tests run by a TAS on a SUT can be subject to sudden bursts of messages to log during their execution. All log messages that occur during execution must be permanently stored in the corresponding test execution logs by the TAS for later analysis. If logging is not performed correctly, these bursts can reduce the execution speed of these automated tests, causing them to produce unreliable results. Which of the following solutions would you expect to be MOST useful to address this issue for TAS logging?
In which way would a feature toggle configuration be used?
A new TAS allows the implementation of automated data-driven test scripts. All the tasks planned for the initial deployment of this TAS, aimed at installing and configuring the TAS components and provisioning the infrastructure, will be performed manually by a dedicated, specialized team. This TAS is expected to be deployed in the future in other similar environments. As a TAE, you see a risk that the correct and reproducible deployment of the TAS cannot be guaranteed. Which of the following options is BEST suited for mitigating this risk?