Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A computer implemented system for automated test and retesting in a virtual test environment using an interactive interface provided by a computer processor, the system comprising: an input interface, comprising at least one processor, configured to receive one or more device independent commands from a user to execute on at least one virtual machine, where the at least one virtual machine represents a simulation of multiple physical devices, wherein the one or more device independent commands comprise at least one automated test procedure and at least one system version parameter, wherein the at least one system version parameter specifies a version of software to execute as a system under test; an automated test and retest engine, comprising at least one processor, configured to automatically execute the one or more device independent commands on the system under test that comprises at least one virtual machine and at least one native machine concurrently, where each virtual machine emulates multiple computer environments on a virtual environment, based at least in part on the at least one system version parameter wherein the automated test and retest engine is configured to execute the one or more device independent commands on a plurality of systems under test on one or more virtual environments located at one or more remote locations and at least one native environment concurrently; the automated test and retest engine further comprising: a test flow tool configured to generate a plurality of automated test flows associated with the automated test procedure responsive directly and exclusively from system modeling metadata generated by a System Model that represents an expected behavior of the system under test, where the device independent commands are based on the plurality of automated test flows, the test flow tool provides modeling capability through an interface comprising a canvas display and a palette display, the canvas display configured to automatically build a corresponding automated test flow graphically represented on the canvas display and the palette display configured to display a collection of modeling components for building automated test flows on the canvas display, so that the device independent commands are executed from the plurality of automated test flows via the virtual environment independent of specific physical devices; an output interface, comprising at least one processor, configured to receive automated test results data responsive to execution of the one or more device independent commands via a message based testing protocol and further configured to display the results to the user; and an analysis manager, responsive directly to system modeling metadata generated by the System Model, configured to automatically record test results, conduct automated analysis consistent with the System Model, and provide the results data in a predetermined format to the output interface based at least in part on extracted sensor data from the system under test; wherein the test and retest engine comprises a software library that stores and manages a plurality of versions of a software to execute as one or more systems under test.
This invention relates to automated testing and retesting of software systems in virtual environments. The system addresses the challenge of efficiently testing software across multiple versions and environments without relying on physical devices. It uses a virtual test environment where virtual machines simulate multiple physical devices, allowing for concurrent testing on both virtual and native machines. The system accepts device-independent commands from users, including automated test procedures and system version parameters, which specify the software version to test. An automated test and retest engine executes these commands across multiple systems under test, including virtual and native environments, at remote locations. The system generates automated test flows from system modeling metadata, which defines the expected behavior of the system under test. A test flow tool provides a graphical interface with a canvas and palette for building test flows, ensuring device-independent execution. Test results are collected via a message-based protocol, analyzed automatically, and displayed to the user. The system also includes a software library to manage multiple versions of the software under test, enabling version-specific testing and retesting. The analysis manager records test results, performs automated analysis based on the system model, and formats the results for output. This approach streamlines testing by abstracting physical device dependencies and automating test execution, analysis, and reporting.
2. The system of claim 1 , wherein the plurality of systems of tests further comprise at least one system under test executing on a native environment on hardware.
A system for testing software applications includes multiple test systems, where at least one of these systems involves a software application running in a native environment directly on hardware. This setup allows for real-world performance and compatibility testing of the application. The system may also include other test configurations, such as virtualized or emulated environments, to evaluate the application under different conditions. The native hardware execution ensures accurate assessment of hardware-specific behaviors, such as processor utilization, memory management, and peripheral interactions. This approach is particularly useful for validating applications that require direct hardware access or rely on low-level system operations. The system may further include monitoring and analysis tools to capture performance metrics, detect errors, and compare results across different test environments. By incorporating native hardware testing alongside other configurations, the system provides a comprehensive evaluation of the application's functionality, reliability, and efficiency in real-world deployment scenarios.
3. The system of claim 1 , wherein the output interface is automatically configured to display one or more displays associated with one or more systems under test during execution of the one or more device independent commands.
The system automatically shows the screens of the devices being tested while the test is running.
4. The system of claim 1 , wherein the automated test and retest engine comprises a test manager configured to manage a plurality of automated test procedures for one or more systems under test.
The automated testing system includes a "test manager" that organizes and runs multiple automated tests on one or more things being tested.
5. The system of claim 1 , wherein the analysis manager is further configured to receive the output data from the system under test and automatically generate one or more outputs based on the output data.
The system relates to automated testing and analysis of software or hardware systems. The problem addressed is the need for efficient and automated evaluation of system outputs during testing to identify errors, performance issues, or other anomalies without manual intervention. Traditional testing methods often require manual review of output data, which is time-consuming and prone to human error. The system includes an analysis manager that processes output data from a system under test. The analysis manager is configured to automatically generate one or more outputs based on the output data. These outputs may include reports, alerts, or other forms of feedback that indicate the system's performance, errors, or compliance with expected behavior. The analysis manager may apply predefined criteria, statistical analysis, or machine learning models to assess the output data and produce actionable insights. This automation reduces the need for manual review, speeds up the testing process, and improves accuracy by minimizing human oversight. The system may also integrate with other testing tools or databases to enhance data collection and analysis capabilities.
6. The system of claim 1 , wherein the input interface is configured to manage one or more files associated with the system under test.
The system has an input that can handle files related to the device being tested.
7. A computer implemented method for automated test and retesting in a virtual test environment using an interactive interface provided by a computer processor, the method comprising the steps of: receiving, via an input interface comprising at least one processor, one or more device independent commands from a user to execute on at least one virtual machine, where the at least one virtual machine represents a simulation of multiple physical devices, wherein the one or more device independent commands comprise at least one automated test procedure and at least one system version parameter, wherein the at least one system version parameter specifies a version of software to execute as a system under test; automatically executing, via an automated test and retest engine comprising at least one processor, the one or more device independent commands on the system under test that comprises at least one virtual machine and at least one native machine concurrently, where each virtual machine emulates multiple computer environments on a virtual environment, based at least in part on the at least one system version parameter wherein the automated test and retest engine is configured to execute the one or more device independent commands on a plurality of systems under test on one or more virtual environments located at one or more remote locations and at least one native environment concurrently; the automated test and retest engine further comprising a test flow tool configured to generate a plurality of automated test flows associated with the automated test procedure responsive directly and exclusively from system modeling metadata generated by a System Model that represents an expected behavior of the system under test, where the device independent commands are based on the plurality of automated test flows, the test flow tool provides modeling capability through an interface comprising a canvas display and a palette display, the canvas display configured to automatically build a corresponding automated test flow graphically represented on the canvas display and the palette display configured to display a collection of modeling components fbr building automated test flows on the canvas display, so that the device independent commands are executed from the plurality of automated test flows via the virtual environment independent of specific physical devices; receiving, via an output interface comprising at least one processor, automated test results data responsive to execution of the one or more device independent commands via a message based testing protocol and further configured to display the results to the user; and responsive directly to system modeling metadata generated by the System Model, automatically recording, via an analysis manager comprising at least one processor, test results; conducting automated analysis consistent with the System Model; and providing the results data in a predetermined format to the output interface based at least in part on extracted sensor data from the system under test; Wherein the test and retest engine comprises a software library that stores and manages a plurality of versions of a software to execute as one or more systems under test.
This invention relates to automated testing and retesting of software systems in virtual environments. The system addresses the challenge of efficiently testing software across multiple versions and environments without relying on physical devices. It uses a virtual test environment that simulates multiple physical devices, allowing for concurrent testing on both virtual and native machines. The method involves receiving device-independent commands from a user, which include automated test procedures and system version parameters specifying the software version to test. An automated test and retest engine executes these commands on the system under test, which may include multiple virtual machines and native machines simultaneously. The engine can operate across remote virtual environments and native environments. A test flow tool generates automated test flows directly from system modeling metadata, representing expected system behavior. The tool provides a graphical interface with a canvas and palette for building test flows, ensuring device-independent command execution. Test results are received via a message-based protocol and displayed to the user. An analysis manager automatically records results, conducts analysis based on the system model, and provides formatted results using extracted sensor data. The system includes a software library to manage multiple versions of the software under test. This approach enables scalable, version-specific testing without physical device dependencies.
8. The method of claim 7 , wherein the plurality of systems of tests further comprise at least one system under test executing on a native environment on hardware.
This invention relates to testing systems, specifically methods for executing multiple test systems, including at least one system under test running in a native environment on physical hardware. The technology addresses the challenge of efficiently validating software or hardware systems by combining different testing approaches. The method involves running a plurality of test systems, where at least one of these systems operates in a native environment directly on hardware, rather than in a virtualized or emulated setting. This ensures that the test accurately reflects real-world performance and behavior. The other test systems may include simulations, emulations, or other testing environments, allowing for comprehensive validation across different scenarios. By integrating native hardware testing with other testing methods, the approach improves reliability and accuracy in system validation, particularly for complex or performance-critical applications. The method is useful in industries where real-world hardware behavior is critical, such as embedded systems, automotive, aerospace, and industrial control systems. The invention ensures that tests account for hardware-specific factors like timing, power consumption, and physical interactions, which may not be fully captured in virtualized environments. This hybrid testing approach enhances confidence in system performance and reduces the risk of undetected issues during deployment.
9. The method of claim 7 , wherein the output interface is automatically configured to display one or more displays associated with one or more systems under test during execution of the one or more device independent commands.
This invention relates to automated testing systems, specifically improving the visualization of test results for multiple systems under test (SUTs). The problem addressed is the lack of real-time, unified display capabilities in existing test frameworks, which often require manual configuration or separate interfaces for monitoring different SUTs during test execution. The invention provides a method for automatically configuring an output interface to display one or more displays associated with one or more SUTs during the execution of device-independent commands. These commands are designed to be compatible with various types of SUTs, allowing a single test framework to control and monitor heterogeneous systems without requiring system-specific adjustments. The output interface dynamically generates and updates visual representations of test progress, status, and results for each SUT in real time, ensuring that test operators can observe the behavior of multiple systems simultaneously without manual intervention. This automation reduces setup time and minimizes errors associated with manual configuration, improving efficiency and reliability in testing environments. The system also supports customizable display layouts, allowing users to tailor the interface to their specific monitoring needs while maintaining the automated configuration benefits.
10. The method of claim 7 , wherein the automated test and retest engine comprises a test manager configured to manage a plurality of automated test procedures for one or more systems under test.
This invention relates to automated testing systems, specifically improving the management and execution of test procedures for software or hardware systems. The problem addressed is the inefficiency and complexity of manually coordinating multiple test procedures, which can lead to errors, delays, and inconsistent results. The solution involves an automated test and retest engine that includes a test manager to oversee and control a variety of automated test procedures for one or more systems under test. The test manager is designed to handle the scheduling, execution, and monitoring of multiple test procedures simultaneously. It ensures that tests are run in the correct sequence, with proper dependencies and configurations, and that results are collected and analyzed efficiently. The system may also support retesting, allowing for automated re-execution of failed or questionable tests to verify results. This approach reduces human intervention, improves test coverage, and enhances the reliability of testing outcomes. The invention is particularly useful in environments where frequent testing is required, such as software development, quality assurance, or hardware validation. By automating the management of test procedures, the system increases efficiency, reduces errors, and provides consistent, repeatable testing processes.
11. The method of claim 7 , wherein the analysis manager is further configured to receive the output data from the system under test and automatically generate one or more outputs based on the output data.
This invention relates to automated testing systems, specifically methods for analyzing output data from a system under test (SUT) to generate actionable insights. The problem addressed is the inefficiency and potential inaccuracies in manually interpreting test results, which can lead to missed defects or incorrect conclusions. The method involves an analysis manager that processes output data from the SUT, which may include logs, metrics, or other performance indicators. The analysis manager automatically generates outputs such as reports, alerts, or recommendations based on predefined criteria or machine learning models. These outputs help identify failures, performance bottlenecks, or compliance issues without manual intervention. The analysis manager may also compare the output data against expected results, historical baselines, or industry standards to detect anomalies. It can prioritize findings based on severity or impact, ensuring critical issues are addressed first. The system may integrate with other tools, such as debugging or monitoring systems, to provide a comprehensive testing solution. By automating the analysis of test outputs, the invention reduces human error, speeds up defect identification, and improves overall testing efficiency. This is particularly useful in continuous integration/continuous deployment (CI/CD) pipelines where rapid feedback is essential. The method ensures that test results are consistently and accurately interpreted, enhancing software quality and reliability.
12. The method of claim 7 , wherein the input interface is configured to manage one or more files associated with the system under test.
A system and method for managing files associated with a system under test (SUT) in a testing environment. The system includes an input interface designed to handle file operations for the SUT, such as creating, modifying, or deleting files. The interface may also support file transfer, storage, and retrieval, ensuring that test data and configurations are properly managed during testing. The method involves using the input interface to perform these file operations, which may include interacting with local or remote storage systems. The system may further include a processing unit that executes test scripts or commands to validate the SUT's functionality, with the input interface facilitating the necessary file interactions required for these tests. The method ensures that file operations are accurately tracked and logged, allowing for traceability and reproducibility in testing processes. This approach improves efficiency and reliability in software or hardware testing by automating file management tasks, reducing manual intervention, and minimizing errors. The system may be integrated into larger testing frameworks or used as a standalone tool for file management in testing environments.
Unknown
June 9, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.