|
Multitask Optimization
Multi-task optimization is a paradigm in the optimization literature that focuses on solving multiple self-contained tasks simultaneously.Gupta, A., Ong, Y. S., & Feng, L. (2018)Insights on transfer optimization: Because experience is the best teacher IEEE Transactions on Emerging Topics in Computational Intelligence, 2(1), 51-64.Gupta, A., Ong, Y. S., & Feng, L. (2016)Multifactorial evolution: toward evolutionary multitasking.IEEE Transactions on Evolutionary Computation, 20(3), 343-357. The paradigm has been inspired by the well-established concepts of transfer learning and multi-task learning in predictive analytics. The key motivation behind multi-task optimization is that if optimization tasks are related to each other in terms of their optimal solutions or the general characteristics of their function landscapes, the search progress can be transferred to substantially accelerate the search on the other. The success of the paradigm is not necessarily limited to one-way kno ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Paradigm
In science and philosophy, a paradigm () is a distinct set of concepts or thought patterns, including theories, research methods, postulates, and standards for what constitute legitimate contributions to a field. Etymology ''Paradigm'' comes from Greek παράδειγμα (''paradeigma''), "pattern, example, sample" from the verb παραδείκνυμι (''paradeiknumi''), "exhibit, represent, expose" and that from παρά (''para''), "beside, beyond" and δείκνυμι (''deiknumi''), "to show, to point out". In classical (Greek-based) rhetoric, a paradeigma aims to provide an audience with an illustration of a similar occurrence. This illustration is not meant to take the audience to a conclusion, however it is used to help guide them get there. One way of how a ''paradeigma'' is meant to guide an audience would be exemplified by the role of a personal accountant. It is not the job of a personal accountant to tell a client exactly what (and what not) to spend money on ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Gradient Descent
In mathematics, gradient descent (also often called steepest descent) is a first-order iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point, because this is the direction of steepest descent. Conversely, stepping in the direction of the gradient will lead to a local maximum of that function; the procedure is then known as gradient ascent. Gradient descent is generally attributed to Augustin-Louis Cauchy, who first suggested it in 1847. Jacques Hadamard independently proposed a similar method in 1907. Its convergence properties for non-linear optimization problems were first studied by Haskell Curry in 1944, with the method becoming increasingly well-studied and used in the following decades. Description Gradient descent is based on the observation that if the multi-variable function F(\mathbf) is def ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Multicriteria Classification
In multiple criteria decision aiding (MCDA), multicriteria classification (or sorting) involves problems where a finite set of alternative actions should be assigned into a predefined set of preferentially ordered categories (classes). For example, credit analysts classify loan applications into risk categories (e.g., acceptable/unacceptable applicants), customers rate products and classify them into attractiveness groups, candidates for a job position are evaluated and their applications are approved or rejected, technical systems are prioritized for inspection on the basis of their failure risk, clinicians classify patients according to the extent to which they have a complex disease or not, etc. Problem statement In a multicriteria classification problem (MCP) a set : X=\ of ''m'' alternative actions is available. Each alternative is evaluated over a set of ''n'' criteria. The scope of the analysis is to assign each alternative into a given set of categories (classes) ''C'' ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Multi-task Learning
Multi-task learning (MTL) is a subfield of machine learning in which multiple learning tasks are solved at the same time, while exploiting commonalities and differences across tasks. This can result in improved learning efficiency and prediction accuracy for the task-specific models, when compared to training the models separately. Early versions of MTL were called "hints". In a widely cited 1997 paper, Rich Caruana gave the following characterization:Multitask Learning is an approach to inductive transfer that improves generalization by using the domain information contained in the training signals of related tasks as an inductive bias. It does this by learning tasks in parallel while using a shared representation; what is learned for each task can help other tasks be learned better. In the classification context, MTL aims to improve the performance of multiple classification tasks by learning them jointly. One example is a spam-filter, which can be treated as distinct but relate ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Multi-objective Optimization
Multi-objective optimization (also known as multi-objective programming, vector optimization, multicriteria optimization, multiattribute optimization or Pareto optimization) is an area of multiple criteria decision making that is concerned with mathematical optimization problems involving more than one objective function to be optimized simultaneously. Multi-objective optimization has been applied in many fields of science, including engineering, economics and logistics where optimal decisions need to be taken in the presence of trade-offs between two or more conflicting objectives. Minimizing cost while maximizing comfort while buying a car, and maximizing performance whilst minimizing fuel consumption and emission of pollutants of a vehicle are examples of multi-objective optimization problems involving two and three objectives, respectively. In practical problems, there can be more than three objectives. For a nontrivial multi-objective optimization problem, no single solutio ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Ensemble Learning
In statistics and machine learning, ensemble methods use multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone. Unlike a statistical ensemble in statistical mechanics, which is usually infinite, a machine learning ensemble consists of only a concrete finite set of alternative models, but typically allows for much more flexible structure to exist among those alternatives. Overview Supervised learning algorithms perform the task of searching through a hypothesis space to find a suitable hypothesis that will make good predictions with a particular problem. Even if the hypothesis space contains hypotheses that are very well-suited for a particular problem, it may be very difficult to find a good one. Ensembles combine multiple hypotheses to form a (hopefully) better hypothesis. The term ''ensemble'' is usually reserved for methods that generate multiple hypotheses using the same base learne ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Machine Learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence. Machine learning algorithms build a model based on sample data, known as training data, in order to make predictions or decisions without being explicitly programmed to do so. Machine learning algorithms are used in a wide variety of applications, such as in medicine, email filtering, speech recognition, agriculture, and computer vision, where it is difficult or unfeasible to develop conventional algorithms to perform the needed tasks.Hu, J.; Niu, H.; Carrasco, J.; Lennox, B.; Arvin, F.,Voronoi-Based Multi-Robot Autonomous Exploration in Unknown Environments via Deep Reinforcement Learning IEEE Transactions on Vehicular Technology, 2020. A subset of machine learning is closely related to computational statistics, which focuses on making predicti ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Cooperative Bargaining
Cooperative bargaining is a process in which two people decide how to share a surplus that they can jointly generate. In many cases, the surplus created by the two players can be shared in many ways, forcing the players to negotiate which division of payoffs to choose. Such surplus-sharing problems (also called bargaining problem) are faced by management and labor in the division of a firm's profit, by trade partners in the specification of the terms of trade, and more. The present article focuses on the ''normative'' approach to bargaining. It studies how the surplus ''should'' be shared, by formulating appealing axioms that the solution to a bargaining problem should satisfy. It is useful when both parties are willing to cooperate in implementing the fair solution. The five axioms, any Nash Bargaining Solution should satisfy are Pareto Optimality (PAR), Individual Rationality (IR), Independent of Expected Utility Representations (INV), Independence of Irrelevant Alternatives (IIA) ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Deep Neural Networks
Deep learning (also known as deep structured learning) is part of a broader family of machine learning methods based on artificial neural networks with representation learning. Learning can be supervised, semi-supervised or unsupervised. Deep-learning architectures such as deep neural networks, deep belief networks, deep reinforcement learning, recurrent neural networks, convolutional neural networks and Transformers have been applied to fields including computer vision, speech recognition, natural language processing, machine translation, bioinformatics, drug design, medical image analysis, climate science, material inspection and board game programs, where they have produced results comparable to and in some cases surpassing human expert performance. Artificial neural networks (ANNs) were inspired by information processing and distributed communication nodes in biological systems. ANNs have various differences from biological brains. Specifically, artificial neural ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Crossover (genetic Algorithm)
In genetic algorithms and evolutionary computation, crossover, also called recombination, is a genetic operator used to combine the genetic information of two parents to generate new offspring. It is one way to stochastically generate new solutions from an existing population, and is analogous to the crossover that happens during sexual reproduction in biology. Solutions can also be generated by cloning an existing solution, which is analogous to asexual reproduction. Newly generated solutions are typically mutated before being added to the population. Different algorithms in evolutionary computation may use different data structures to store genetic information, and each genetic representation can be recombined with different crossover operators. Typical data structures that can be recombined with crossover are bit arrays, vectors of real numbers, or trees. Examples Traditional genetic algorithms store genetic information in a chromosome represented by a bit array. Crossover m ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Transfer Learning
Transfer learning (TL) is a research problem in machine learning (ML) that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem. For example, knowledge gained while learning to recognize cars could apply when trying to recognize trucks. This area of research bears some relation to the long history of psychological literature on transfer of learning, although practical ties between the two fields are limited. From the practical standpoint, reusing or transferring information from previously learned tasks for the learning of new tasks has the potential to significantly improve the sample efficiency of a reinforcement learning agent. History In 1976, Stevo Bozinovski and Ante Fulgosi published a paper explicitly addressing transfer learning in neural networks training. The paper gives a mathematical and geometrical model of transfer learning. In 1981, a report was given on the application of transfer learning in training ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Implicit Parallelism
In computer science, implicit parallelism is a characteristic of a programming language that allows a compiler or interpreter to automatically exploit the parallelism inherent to the computations expressed by some of the language's constructs. A pure implicitly parallel language does not need special directives, operators or functions to enable parallel execution, as opposed to explicit parallelism. Programming languages with implicit parallelism include Axum, BMDFM, HPF, Id, LabVIEW, MATLAB M-code, NESL, SaC, SISAL, ZPL, and pH. Example If a particular problem involves performing the same operation on a group of numbers (such as taking the sine or logarithm of each in turn), a language that provides implicit parallelism might allow the programmer to write the instruction thus: numbers = 1 2 3 4 5 6 7 result = sin(numbers); The compiler or interpreter can calculate the sine of each element independently, spreading the effort across multiple processors if available. ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
