|
Active Set Method
In mathematical optimization, the active-set method is an algorithm used to identify the active constraints in a set of inequality constraints. The active constraints are then expressed as equality constraints, thereby transforming an inequality-constrained problem into a simpler equality-constrained subproblem. An optimization problem is defined using an objective function to minimize or maximize, and a set of constraints : g_1(x) \ge 0, \dots, g_k(x) \ge 0 that define the feasible region, that is, the set of all ''x'' to search for the optimal solution. Given a point x in the feasible region, a constraint : g_i(x) \ge 0 is called active at x_0 if g_i(x_0) = 0, and inactive at x if g_i(x_0) > 0. Equality constraints are always active. The active set at x_0 is made up of those constraints g_i(x_0) that are active at the current point . The active set is particularly important in optimization theory, as it determines which constraints will influence the final result of optim ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Optimization (mathematics)
Mathematical optimization (alternatively spelled ''optimisation'') or mathematical programming is the selection of a best element, with regard to some criterion, from some set of available alternatives. It is generally divided into two subfields: discrete optimization and continuous optimization. Optimization problems of sorts arise in all quantitative disciplines from computer science and engineering to operations research and economics, and the development of solution methods has been of interest in mathematics for centuries. In the more general approach, an optimization problem consists of maximizing or minimizing a real function by systematically choosing input values from within an allowed set and computing the value of the function. The generalization of optimization theory and techniques to other formulations constitutes a large area of applied mathematics. More generally, optimization includes finding "best available" values of some objective function given a defi ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Constraint (mathematics)
In mathematics, a constraint is a condition of an optimization problem that the solution must satisfy. There are several types of constraints—primarily equality constraints, inequality constraints, and integer constraints. The set of candidate solutions that satisfy all constraints is called the feasible set. Example The following is a simple optimization problem: :\min f(\mathbf x) = x_1^2+x_2^4 subject to :x_1 \ge 1 and :x_2 = 1, where \mathbf x denotes the vector (''x''1, ''x''2). In this example, the first line defines the function to be minimized (called the objective function, loss function, or cost function). The second and third lines define two constraints, the first of which is an inequality constraint and the second of which is an equality constraint. These two constraints are hard constraints, meaning that it is required that they be satisfied; they define the feasible set of candidate solutions. Without the constraints, the solution would be (0,0), ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Inequality (mathematics)
In mathematics, an inequality is a relation which makes a non-equal comparison between two numbers or other mathematical expressions. It is used most often to compare two numbers on the number line by their size. There are several different notations used to represent different kinds of inequalities: * The notation ''a'' ''b'' means that ''a'' is greater than ''b''. In either case, ''a'' is not equal to ''b''. These relations are known as strict inequalities, meaning that ''a'' is strictly less than or strictly greater than ''b''. Equivalence is excluded. In contrast to strict inequalities, there are two types of inequality relations that are not strict: * The notation ''a'' ≤ ''b'' or ''a'' ⩽ ''b'' means that ''a'' is less than or equal to ''b'' (or, equivalently, at most ''b'', or not greater than ''b''). * The notation ''a'' ≥ ''b'' or ''a'' ⩾ ''b'' means that ''a'' is greater than or equal to ''b'' (or, equivalently, at least ''b'', or not less than ''b''). The r ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Feasible Region
In mathematical optimization, a feasible region, feasible set, search space, or solution space is the set of all possible points (sets of values of the choice variables) of an optimization problem that satisfy the problem's constraints, potentially including inequalities, equalities, and integer constraints. This is the initial set of candidate solutions to the problem, before the set of candidates has been narrowed down. For example, consider the problem of minimizing the function x^2+y^4 with respect to the variables x and y, subject to 1 \le x \le 10 and 5 \le y \le 12. \, Here the feasible set is the set of pairs (''x'', ''y'') in which the value of ''x'' is at least 1 and at most 10 and the value of ''y'' is at least 5 and at most 12. The feasible set of the problem is separate from the objective function, which states the criterion to be optimized and which in the above example is x^2+y^4. In many problems, the feasible set reflects a constraint that one or m ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Linear Programming
Linear programming (LP), also called linear optimization, is a method to achieve the best outcome (such as maximum profit or lowest cost) in a mathematical model whose requirements are represented by linear relationships. Linear programming is a special case of mathematical programming (also known as mathematical optimization). More formally, linear programming is a technique for the optimization of a linear objective function, subject to linear equality and linear inequality constraints. Its feasible region is a convex polytope, which is a set defined as the intersection of finitely many half spaces, each of which is defined by a linear inequality. Its objective function is a real-valued affine (linear) function defined on this polyhedron. A linear programming algorithm finds a point in the polytope where this function has the smallest (or largest) value if such a point exists. Linear programs are problems that can be expressed in canonical form as : \begin & \t ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Hyperplane
In geometry, a hyperplane is a subspace whose dimension is one less than that of its '' ambient space''. For example, if a space is 3-dimensional then its hyperplanes are the 2-dimensional planes, while if the space is 2-dimensional, its hyperplanes are the 1-dimensional lines. This notion can be used in any general space in which the concept of the dimension of a subspace is defined. In different settings, hyperplanes may have different properties. For instance, a hyperplane of an -dimensional affine space is a flat subset with dimension and it separates the space into two half spaces. While a hyperplane of an -dimensional projective space does not have this property. The difference in dimension between a subspace and its ambient space is known as the codimension of with respect to . Therefore, a necessary and sufficient condition for to be a hyperplane in is for to have codimension one in . Technical description In geometry, a hyperplane of an ''n''-dime ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Quadratic Programming
Quadratic programming (QP) is the process of solving certain mathematical optimization problems involving quadratic functions. Specifically, one seeks to optimize (minimize or maximize) a multivariate quadratic function subject to linear constraints on the variables. Quadratic programming is a type of nonlinear programming. "Programming" in this context refers to a formal procedure for solving mathematical problems. This usage dates to the 1940s and is not specifically tied to the more recent notion of "computer programming." To avoid confusion, some practitioners prefer the term "optimization" — e.g., "quadratic optimization." Problem formulation The quadratic programming problem with variables and constraints can be formulated as follows. Given: * a real-valued, -dimensional vector , * an -dimensional real symmetric matrix , * an -dimensional real matrix , and * an -dimensional real vector , the objective of quadratic programming is to find an -dimensional vector , that ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Lagrange Multipliers
In mathematical optimization, the method of Lagrange multipliers is a strategy for finding the local maxima and minima of a function subject to equality constraints (i.e., subject to the condition that one or more equations have to be satisfied exactly by the chosen values of the variables). It is named after the mathematician Joseph-Louis Lagrange. The basic idea is to convert a constrained problem into a form such that the derivative test of an unconstrained problem can still be applied. The relationship between the gradient of the function and gradients of the constraints rather naturally leads to a reformulation of the original problem, known as the Lagrangian function. The method can be summarized as follows: in order to find the maximum or minimum of a function f(x) subjected to the equality constraint g(x) = 0, form the Lagrangian function :\mathcal(x, \lambda) = f(x) + \lambda g(x) and find the stationary points of \mathcal considered as a function of x and the Lagrange m ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Successive Linear Programming
Successive Linear Programming (SLP), also known as Sequential Linear Programming, is an optimization technique for approximately solving nonlinear optimization problems. Starting at some estimate of the optimal solution, the method is based on solving a sequence of first-order approximations (i.e. linearizations) of the model. The linearizations are linear programming problems, which can be solved efficiently. As the linearizations need not be bounded, trust regions or similar techniques are needed to ensure convergence in theory. SLP has been used widely in the petrochemical industry since the 1970s. See also * Sequential quadratic programming * Sequential linear-quadratic programming * Augmented Lagrangian method Augmented Lagrangian methods are a certain class of algorithms for solving constrained optimization problems. They have similarities to penalty methods in that they replace a constrained optimization problem by a series of unconstrained problems a ... References ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Sequential Quadratic Programming
Sequential quadratic programming (SQP) is an iterative method for constrained nonlinear optimization. SQP methods are used on mathematical problems for which the objective function and the constraints are twice continuously differentiable. SQP methods solve a sequence of optimization subproblems, each of which optimizes a quadratic model of the objective subject to a linearization of the constraints. If the problem is unconstrained, then the method reduces to Newton's method for finding a point where the gradient of the objective vanishes. If the problem has only equality constraints, then the method is equivalent to applying Newton's method to the first-order optimality conditions, or Karush–Kuhn–Tucker conditions, of the problem. Algorithm basics Consider a nonlinear programming problem of the form: :\begin \min\limits_ & f(x) \\ \mbox & b(x) \ge 0 \\ & c(x) = 0. \end The Lagrangian for this problem is :\mathcal(x,\lambda,\sigma) = f(x) - \lambda b(x) - \sigma c(x), wh ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Sequential Linear-quadratic Programming
Sequential linear-quadratic programming (SLQP) is an iterative method for nonlinear optimization problems where objective function and constraints are twice continuously differentiable. Similarly to sequential quadratic programming (SQP), SLQP proceeds by solving a sequence of optimization subproblems. The difference between the two approaches is that: * in SQP, each subproblem is a quadratic program, with a quadratic model of the objective subject to a linearization of the constraints * in SLQP, two subproblems are solved at each step: a linear program (LP) used to determine an active set, followed by an equality-constrained quadratic program (EQP) used to compute the total step This decomposition makes SLQP suitable to large-scale optimization problems, for which efficient LP and EQP solvers are available, these problems being easier to scale than full-fledged quadratic programs. Algorithm basics Consider a nonlinear programming problem of the form: :\begin \min\limits_ & f(x ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Frank–Wolfe Algorithm
The Frank–Wolfe algorithm is an iterative first-order optimization algorithm for constrained convex optimization. Also known as the conditional gradient method, reduced gradient algorithm and the convex combination algorithm, the method was originally proposed by Marguerite Frank and Philip Wolfe in 1956. In each iteration, the Frank–Wolfe algorithm considers a linear approximation of the objective function, and moves towards a minimizer of this linear function (taken over the same domain). Problem statement Suppose \mathcal is a compact convex set in a vector space and f \colon \mathcal \to \mathbb is a convex, differentiable real-valued function. The Frank–Wolfe algorithm solves the optimization problem :Minimize f(\mathbf) :subject to \mathbf \in \mathcal. Algorithm :''Initialization:'' Let k \leftarrow 0, and let \mathbf_0 \! be any point in \mathcal. :Step 1. ''Direction-finding subproblem:'' Find \mathbf_k solving ::Minimize \mathbf^T \nabla f(\mathbf_k ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
