Legal claims defining the scope of protection. Each claim is shown in both the original legal language and a plain English translation.
1. A computer-aided decision making system, comprising: a user input device; a user output device; and a processor programmed to evaluate decision problems available to a user, the programmed processor; (A) facilitating input of a historical data set from a decision maker via the user input device; (B) the programmed processor defining a decision problem to be solved, the decision problem defined by parameters generated using statistical techniques on the historical data set, the parameters including; (i) an action set, the action set has elements representing actions available to a subject and action costs to the subject of performing the actions, (ii) at least one state dimension representing conditions relevant to the subject of the decision problem, (iii) a reward set representing rewards received by the user when transitioning between states for actions in the action set, (iv) each state dimension having a corresponding transition matrix containing a probability of moving between the states for actions in the action set, (v) a time index and a discount factor, the time index containing decision points available to the subject where the subject selects an action from the action set, and the discount factor representing the subject's preference for rewards relative to time, (C) the programmed processor combining the reward set with the action costs to form a reward matrix and the programmed processor combining the transition matrices with the action set to form a total transition matrix; (D) the programmed processor forming a functional equation from the state dimensions, the reward matrix, the total transition matrix, and the time index and the discount factor; (E) the programmed processor evaluating the functional equation, including error-checking and validating the parameters and performing a convergence check to ensure that the functional equation will be solvable, and the programmed processor solving the functional equation; (F) the programmed processor generating an optimal policy by using the solved functional equation to find, for every point in the time index, an overall value-maximizing action; (G) the programmed processor outputting the optimal policy to the user through the user output device.
2. A computer-aided decision making system according to claim 1 , wherein the programmed processor generates the at least one state dimension, the action costs, the time index and the discount factor using at least one of: K-means, K nearest neighbors, or hierarchical clustering, or Bayes or Naive-Bayes classification.
A computer-aided decision-making system automates decision processes by evaluating possible actions and their outcomes. The system models decision-making as a sequential optimization problem, where actions are selected to minimize costs over time. The system generates state dimensions, action costs, time indices, and discount factors to evaluate decisions. These parameters are derived using machine learning techniques such as K-means, K-nearest neighbors, hierarchical clustering, Bayes classification, or Naive-Bayes classification. The system processes input data to identify relevant state dimensions, assigns costs to actions, and applies a discount factor to prioritize immediate outcomes over future ones. The time index tracks the sequence of decisions. By leveraging these techniques, the system provides data-driven decision support, improving efficiency and accuracy in decision-making processes. The approach is applicable in fields like finance, healthcare, and logistics, where optimal decision-making is critical. The system dynamically adapts to changing conditions by updating parameters based on new data, ensuring robust performance.
3. A computer-aided decision making system according to claim 1 , wherein the programmed processor receives the historical data and, before defining the decision problem by generating the parameters, at least one additional input of: an action, a state, a discount factor, a decision point, a reward for the reward set, an element of the transition matrix, or an action cost, the programmed processor generating all of the parameters not received as additional input and including the additional input in the statistical techniques.
A computer-aided decision-making system automates the process of defining and solving decision problems using statistical techniques. The system addresses the challenge of efficiently modeling complex decision scenarios by leveraging historical data and user-provided inputs to generate key parameters. These parameters include actions, states, discount factors, decision points, rewards, transition matrix elements, and action costs. The system receives at least one of these inputs from a user or external source and automatically generates the remaining parameters. The historical data and additional inputs are integrated into statistical techniques, such as reinforcement learning or Markov decision processes, to optimize decision-making. This approach reduces the manual effort required to define decision problems while improving the accuracy and adaptability of the system. The system is particularly useful in fields like finance, healthcare, and logistics, where dynamic decision-making is critical. By automating parameter generation, it enables faster deployment of decision models and supports real-time adjustments based on new data.
4. A computer-aided decision making system according to claim 1 , wherein the programmed processor receives the historical data and the at least one state dimension, the action set, the action costs, the discount factor, the time index and the programmed processor uses the historical data to generate the reward set and the transition matrices.
A computer-aided decision-making system processes historical data to optimize decision-making in dynamic environments. The system operates in a domain where decisions must be made under uncertainty, balancing immediate costs against long-term rewards. The system receives historical data, which includes past states, actions, and outcomes, along with key parameters such as state dimensions, available actions, action costs, a discount factor, and a time index. Using this data, the system generates a reward set, which quantifies the benefits of different actions, and transition matrices, which model how actions influence future states. The system then applies these models to evaluate potential decisions, ensuring that future actions align with long-term objectives while accounting for immediate costs. This approach enables automated decision-making in complex scenarios where manual analysis would be impractical. The system is particularly useful in fields like finance, logistics, and healthcare, where decisions must balance short-term and long-term consequences. By leveraging historical data, the system improves decision accuracy and efficiency, reducing reliance on human judgment in high-stakes environments.
5. A computer-aided decision making system according to claim 4 , wherein the programmed processor uses the historical data to generate the parameters including modifying at least one of the at least one state dimension, the action set, the action costs, the discount factor or the time index.
A computer-aided decision-making system is designed to optimize decision-making processes by leveraging historical data to refine key parameters in a decision model. The system operates within the domain of automated decision support, addressing challenges in dynamic environments where traditional static models fail to adapt to changing conditions. The core functionality involves a programmed processor that analyzes historical data to dynamically adjust critical model parameters, including state dimensions, action sets, action costs, discount factors, or time indices. These adjustments enhance the system's ability to make accurate and contextually relevant decisions. The state dimensions define the variables considered in decision-making, while the action set specifies available actions. Action costs quantify the resources or penalties associated with each action, and the discount factor determines the weight given to future outcomes. The time index ensures decisions align with temporal constraints. By modifying these parameters based on historical data, the system improves adaptability and performance in real-world applications, such as resource allocation, scheduling, or strategic planning. The historical data provides insights into past decision outcomes, enabling the system to refine its model for better future performance. This approach ensures decisions are data-driven and responsive to evolving conditions, enhancing efficiency and accuracy in automated decision-making processes.
6. A computer-aided decision making system according to claim 4 , wherein the programmed processor receives the historical data and at least one reward for the reward set or one element of a transition matrix and uses the historical data and all of the elements not received as additional input and including the additional input in the statistical techniques to generate the reward set and the set of transition matrices.
A computer-aided decision-making system processes historical data and incomplete reward or transition matrix information to generate a complete set of rewards and transition matrices. The system operates in the domain of decision optimization, where incomplete or partially specified models are common. The problem addressed is the need to derive a fully specified decision model from partial inputs, improving accuracy and reliability in automated decision-making processes. The system receives historical data and at least one reward from a predefined reward set or one element of a transition matrix. Using statistical techniques, it infers the missing elements of the reward set and transition matrices based on the provided data and partial inputs. This approach ensures that the decision-making model is complete and consistent, even when some parameters are initially unknown. The system leverages historical data to fill gaps in the model, enhancing its predictive and decision-making capabilities. This method is particularly useful in applications where full model specification is impractical or costly, such as in reinforcement learning, operations research, and automated control systems. The system's ability to infer missing parameters from partial inputs improves robustness and adaptability in dynamic environments.
7. A computer-aided decision making system according to claim 1 , wherein the programmed processor prompts the user to review and edit at least one of the parameters of the decision problem generated from the historical data and forming the functional equation, the programmed processor allowing the user to re-review and re-edit at least one of the historical data or the parameters after forming the functional equation, re-forming the functional equation when an edit is made.
A computer-aided decision-making system assists users in analyzing and solving complex decision problems by leveraging historical data. The system generates a functional equation representing the decision problem, which is derived from the historical data and includes parameters that define the problem's structure. The system prompts the user to review and edit these parameters, allowing adjustments to refine the decision model. Additionally, the user can re-examine and modify the historical data or parameters even after the functional equation is initially formed. When edits are made, the system dynamically re-forms the functional equation to reflect the updated inputs, ensuring the decision model remains accurate and aligned with the user's requirements. This iterative process enables users to fine-tune the decision-making framework, improving the system's ability to provide reliable and actionable insights. The system enhances decision-making by combining automated data processing with user-driven customization, ensuring flexibility and precision in problem-solving.
8. A computer-aided decision making system according to claim 4 , wherein the programmed processor prompts the user to review and edit at least one of the parameters of the decision problem generated from the historical data and forming the functional equation, the programmed processor allowing the user to re-review and re-edit at least one of the historical data or the parameters after forming the functional equation, re-forming the functional equation when an edit is made.
A computer-aided decision-making system assists users in analyzing and solving complex decision problems by leveraging historical data. The system generates a functional equation representing the decision problem, which is derived from the historical data and user-defined parameters. The system includes a user interface that allows the user to review and edit these parameters before the functional equation is formed. Additionally, after the functional equation is generated, the user can re-review and re-edit the historical data or parameters, triggering the system to re-form the functional equation based on the updated inputs. This iterative process ensures that the decision model remains accurate and aligned with the user's evolving requirements. The system dynamically adjusts the functional equation in response to edits, providing real-time feedback and refining the decision-making framework. This approach enhances the flexibility and precision of the decision-making process, allowing users to refine their models based on new insights or changing conditions. The system is particularly useful in fields requiring iterative analysis, such as financial forecasting, risk assessment, and strategic planning.
9. A computer-aided decision making system according to claim 1 , wherein the programmed processor prompts the user to review and edit at least one of the parameters of the decision problem generated from the historical data after viewing the optimal policy and the programmed processor reforming and solving the edited decision problem for the user.
A computer-aided decision-making system assists users in analyzing and solving complex decision problems using historical data. The system generates a decision problem model based on historical data, which includes parameters defining the problem's structure and constraints. The system then computes an optimal policy or solution for the decision problem and presents it to the user. The user can review and edit the parameters of the decision problem after viewing the optimal policy. The system then reformulates and resolves the edited decision problem, providing an updated optimal policy based on the revised parameters. This iterative process allows users to refine their decision-making by adjusting inputs and constraints, ensuring the solution aligns with their objectives. The system automates the computational aspects of decision analysis, reducing manual effort and improving the accuracy of decision-making. The iterative feedback loop between the user and the system enhances the adaptability and precision of the decision-making process.
10. A computer-aided decision making system according to claim 4 , wherein the programmed processor prompts the user to review and edit at least one of the parameters of the decision problem generated from the historical data after viewing the optimal policy and the programmed processor reforming and solving the edited decision problem for the user.
A computer-aided decision-making system assists users in analyzing and solving complex decision problems by leveraging historical data. The system generates a decision problem model based on this data, which includes parameters defining the problem's structure and constraints. The system then computes an optimal policy or solution for the user to review. After evaluating the proposed solution, the user can modify the parameters of the decision problem, such as constraints, objectives, or variables, to refine the model. The system then reformulates and resolves the edited decision problem, providing an updated optimal policy based on the revised parameters. This iterative process allows users to explore different scenarios, adjust assumptions, and refine decision-making strategies in real time. The system is particularly useful in fields requiring data-driven decision support, such as operations research, finance, and logistics, where multiple variables and constraints must be balanced to achieve optimal outcomes. By enabling dynamic parameter adjustments and rapid re-evaluation, the system enhances the user's ability to make informed, adaptive decisions.
11. A computer implemented method for assisting a user in making a decision comprising: providing a computer system having a user input device, a user output device and a processor programmed with instructions to evaluate decision problems available to the user, the instructions programming the processor and; (A) using the computer system to facilitate input of a historical data set from a decision maker via the user input device; (B) defining a decision problem to be solved, the decision problem defined by parameters generated using statistical techniques on the historical data set, the parameters including; (i) an action set, the action set has elements representing actions available to a subject and action costs to the subject of performing the actions, (ii) at least one state dimension representing conditions relevant to the subject of the decision problem, each state dimension has elements representing values of a condition relevant to the subject of the decision problem, (iii) a reward set representing rewards received by the user when transitioning between states for each action in the action set, (iv) each state dimension having a corresponding transition matrix containing a probability of moving between the states for actions in the action set, (v) a time index and a discount factor, the time index containing decision points available to the subject where the subject selects an action from the action set, and the discount factor representing the subject's preference for rewards relative to time, (C) combining the reward set with the action costs to form a reward matrix and combining the transition matrices with the action set to form a total transition matrix; (D) forming a functional equation from the state dimensions, the reward matrix, the total transition matrix, and the time index and the discount factor; (E) evaluating the functional equation, including error-checking and validating the parameters and performing a convergence check to ensure that the functional equation will be solvable, and the programmed processor solving the functional equation; (F) generating an optimal policy by using the solved functional equation to find, for every point in the time index, an overall value-maximizing action; (G) outputting the optimal policy to the user through the user output device.
This invention relates to a computer-implemented decision-making assistance system designed to help users evaluate and optimize decision problems using historical data. The system processes a historical data set provided by a decision maker to define a structured decision problem, which includes key parameters such as an action set, state dimensions, a reward set, transition matrices, a time index, and a discount factor. The action set lists available actions and their associated costs, while state dimensions represent relevant conditions with possible values. The reward set defines rewards for state transitions, and transition matrices contain probabilities of moving between states for each action. The time index marks decision points, and the discount factor reflects the user's time preference for rewards. The system combines these elements into a reward matrix and a total transition matrix, then forms and solves a functional equation to generate an optimal policy. This policy identifies the value-maximizing action for each decision point. The system includes error-checking, validation, and convergence checks to ensure solvability. The final optimal policy is then presented to the user. This approach leverages statistical techniques and probabilistic modeling to assist in structured decision-making across various domains.
12. A method as set forth in claim 11 , wherein the step of generating the at least one state dimension, the action costs, the time index and the discount factor using at least one of: K-means, K nearest neighbors, or hierarchical clustering, or Bayes or Naive-Bayes classification.
This invention relates to methods for optimizing decision-making processes in dynamic systems, particularly in applications involving sequential decision-making under uncertainty. The core problem addressed is the efficient generation of key parameters used in decision-making models, such as state dimensions, action costs, time indices, and discount factors, to improve the accuracy and performance of decision-making algorithms. The method involves generating these parameters using machine learning techniques, including clustering algorithms like K-means, K-nearest neighbors, or hierarchical clustering, as well as classification algorithms like Bayes or Naive-Bayes. These techniques are applied to input data to derive meaningful representations of system states, associated costs, and temporal factors. The generated parameters are then used to construct a decision-making framework that can evaluate possible actions and select optimal or near-optimal decisions based on the learned model. By leveraging these machine learning methods, the approach aims to automate and enhance the parameter generation process, reducing reliance on manual tuning and improving adaptability to different decision-making scenarios. The use of clustering and classification techniques allows for the extraction of patterns and relationships in the data, enabling more informed and efficient decision-making in complex environments. This method is particularly useful in fields such as robotics, autonomous systems, and reinforcement learning, where dynamic decision-making is critical.
13. A method as set forth in claim 11 , wherein the step of receiving the historical data and, before defining the decision problem by generating the parameters, at least one additional input of: an action, a state, a discount factor, a decision point, a reward for the reward set, an element of the transition matrix, or an action cost, the programmed processor generating all of the parameters not received as additional input and including the additional input in the statistical techniques.
This invention relates to decision-making systems that use statistical techniques to optimize outcomes based on historical data. The problem addressed is the need for flexible and efficient methods to define decision problems in dynamic environments where multiple variables influence outcomes. The invention provides a method for generating parameters used in statistical techniques to model decision problems, allowing for partial or complete user input while automatically deriving missing parameters. The method involves receiving historical data and additional inputs that may include actions, states, discount factors, decision points, rewards, transition matrix elements, or action costs. These inputs are used to define the decision problem by generating the necessary parameters for statistical analysis. If some parameters are not provided as additional input, the system automatically generates them. The generated parameters are then incorporated into statistical techniques to model and solve the decision problem. This approach allows for adaptability in defining decision problems, as users can provide specific inputs while the system fills in the remaining parameters. The method ensures that all necessary parameters are available for statistical analysis, enabling accurate decision-making in complex scenarios. The invention is particularly useful in fields such as reinforcement learning, operations research, and automated decision systems where dynamic parameter adjustments are required.
14. A method as set forth in claim 11 , wherein the step of receiving the historical data also includes the at least one state dimension, the action set, the action costs, the discount factor, the time index and using the historical data to generate the reward set and the transition matrices.
This invention relates to reinforcement learning systems, specifically methods for processing historical data to improve decision-making in sequential decision processes. The problem addressed is the efficient extraction and utilization of historical data to generate key components for reinforcement learning models, such as reward sets, transition matrices, and other parameters like state dimensions, action sets, action costs, discount factors, and time indices. The method involves receiving historical data that includes at least one state dimension, an action set, action costs, a discount factor, and a time index. This data is then used to generate a reward set and transition matrices, which are essential for modeling the environment in reinforcement learning. The reward set defines the expected outcomes of actions, while the transition matrices represent the probabilities of moving between states based on actions taken. The method ensures that the historical data is structured in a way that allows for accurate modeling of the decision-making process, enabling the reinforcement learning system to make informed decisions based on past experiences. By incorporating these components, the method provides a framework for training reinforcement learning agents more effectively, reducing the need for extensive real-time data collection and improving the efficiency of policy optimization. This approach is particularly useful in applications where historical data is abundant but not fully utilized, such as in robotics, autonomous systems, and game-playing AI.
15. A method as set forth in claim 14 , wherein the step of using the historical data to generate the parameters further includes modifying at least one of the at least one state dimension, the action set, the action costs, the discount factor or the time index.
This invention relates to reinforcement learning systems, specifically methods for optimizing decision-making processes by adjusting key parameters based on historical data. The core problem addressed is improving the adaptability and performance of reinforcement learning models by dynamically modifying their foundational components. The method involves using historical data to generate or refine parameters that define the reinforcement learning model. These parameters include state dimensions, which represent the variables used to describe the system's condition; the action set, which lists possible decisions the model can make; action costs, which quantify the penalties or expenses associated with each action; the discount factor, which determines the importance of future rewards; and the time index, which tracks the sequence of decisions. The key innovation is the ability to modify these parameters during the learning process. By analyzing historical data, the system can adjust state dimensions to better capture relevant information, expand or restrict the action set based on observed outcomes, update action costs to reflect real-world consequences, modify the discount factor to prioritize short-term or long-term rewards appropriately, and adjust the time index to align with the temporal structure of the problem. This dynamic adjustment enhances the model's ability to make informed decisions in complex, evolving environments. The approach ensures that the reinforcement learning model remains aligned with real-world conditions, improving its accuracy and effectiveness over time.
16. A method as set forth in claim 14 , wherein the step of receiving the historical data further includes at least one reward for the reward set or one element of a transition matrix and using the historical data and all of the elements not received as additional input and including the additional input in the statistical techniques to generate the reward set and the set of transition matrices.
This invention relates to methods for generating reward sets and transition matrices in reinforcement learning systems using historical data. The problem addressed is the need to efficiently construct these components when some elements are missing or incomplete, ensuring robust model training even with partial data. The method involves receiving historical data that includes at least one reward from a predefined reward set or one element of a transition matrix. The received data is combined with additional input derived from the missing elements, which are inferred or estimated based on the available information. Statistical techniques are then applied to this combined input to generate the complete reward set and the set of transition matrices. This approach ensures that the reinforcement learning model can be trained effectively even when some data is unavailable, improving adaptability and performance in real-world applications where data may be incomplete or noisy. The method leverages statistical methods to fill gaps in the data, ensuring accurate and reliable model outputs.
17. A method as set forth in claim 11 , wherein an additional step prompts the user to review and edit at least one of the parameters of the decision problem generated from the historical data and forming the functional equation and further includes the step of allowing the user to re-review and re-edit at least one of the historical data or the parameters after forming the functional equation, re-forming the functional equation when an edit is made.
This invention relates to decision-making systems that analyze historical data to generate and refine decision models. The problem addressed is the need for user interaction in refining decision models derived from historical data to ensure accuracy and relevance. The method involves generating a decision problem from historical data, forming a functional equation representing the decision model, and allowing the user to review and edit parameters of the decision problem. The user can also re-review and re-edit the historical data or parameters after the functional equation is formed. If edits are made, the functional equation is re-formed to reflect the changes. This iterative process ensures the decision model is refined based on user input, improving its reliability and adaptability. The system supports dynamic adjustments to the decision-making framework, allowing for continuous improvement as new data or insights are incorporated. The method enhances the flexibility and accuracy of decision models by enabling user-driven modifications at multiple stages of the process.
18. A method as set forth in claim 14 wherein an additional step prompts the user to review and edit at least one of the parameters of the decision problem generated from the historical data and forming the functional equation and further includes the step of allowing the user to re-review and re-edit at least one of the historical data or the parameters after forming the functional equation, re-forming the functional equation when an edit is made.
This invention relates to decision-making systems that analyze historical data to generate functional equations for solving decision problems. The core challenge addressed is the need for flexibility and user control in refining decision models derived from data. The method involves generating a decision problem from historical data, forming a functional equation based on this data, and then prompting the user to review and edit parameters of the decision problem. The user can modify these parameters or the underlying historical data, triggering a reformation of the functional equation to reflect the changes. This iterative process ensures the decision model aligns with user expectations and domain-specific requirements. The system allows for continuous refinement, improving the accuracy and relevance of the decision-making process. The invention is particularly useful in fields where decision models must adapt to evolving data or user preferences, such as financial forecasting, risk assessment, or operational optimization. By enabling dynamic adjustments, the method enhances the robustness and applicability of data-driven decision systems.
19. A method as set forth in claim 11 wherein an additional step prompts the user to review and edit at least one of the parameters of the decision problem generated from the historical data after viewing the optimal policy and reforming and solving the edited decision problem for the user.
This invention relates to decision-making systems that analyze historical data to generate and optimize decision policies. The problem addressed is the lack of user interaction in automated decision-making processes, which can lead to policies that do not align with user expectations or domain-specific constraints. The solution involves a method where a decision-making system first processes historical data to generate a decision problem, including parameters such as objectives, constraints, and variables. The system then solves this problem to determine an optimal policy. After presenting this policy to the user, the system prompts the user to review and edit the parameters of the decision problem. The edited problem is then reformulated and resolved to generate a revised optimal policy. This iterative process allows the user to refine the decision-making model based on their expertise or preferences, ensuring the final policy better reflects real-world requirements. The method is particularly useful in fields like operations research, finance, and healthcare, where decision models must balance automation with human oversight. By incorporating user feedback, the system improves the accuracy and applicability of the generated policies.
20. A method as set forth in claim 14 wherein the programmed processor prompts the user to review and edit at least one of the parameters of the decision problem generated from the historical data after viewing the optimal policy and reforming and solving the edited decision problem for the user.
This invention relates to decision-making systems that analyze historical data to generate and optimize decision policies. The problem addressed is the need for users to refine decision models based on real-world outcomes and expert judgment. The system uses a programmed processor to process historical data, generate a decision problem model, and compute an optimal policy. The processor then presents this policy to the user, who can review and edit the underlying parameters of the decision problem. After editing, the system reforms and resolves the decision problem to produce an updated optimal policy. This iterative process allows users to incorporate domain knowledge and refine the model based on observed results, improving the accuracy and practicality of the decision-making framework. The system supports dynamic adjustments to parameters such as constraints, objectives, or variables, ensuring the decision model aligns with evolving requirements or new insights. The invention enhances decision-making by combining automated optimization with human expertise, leading to more robust and adaptable decision policies.
Unknown
January 28, 2020
Browse 5M+ US patents with plain-English claim translations and AI-generated analysis.