|
Hypercube (communication Pattern)
d-dimensional hypercube is a network topology for parallel computers with 2^d processing elements. The topology allows for an efficient implementation of some basic communication primitives such as Broadcast, All-Reduce, and Prefix sum. The processing elements are numbered 0 through 2^d - 1. Each processing element is adjacent to processing elements whose numbers differ in one and only one bit. The algorithms described in this page utilize this structure efficiently. Algorithm outline Most of the communication primitives presented in this article share a common template. Initially, each processing element possesses one message that must reach every other processing element during the course of the algorithm. The following pseudo code sketches the communication steps necessary. Hereby, Initialization, Operation, and Output are placeholders that depend on the given communication primitive (see next section). Input: message m. Output: depends on Initialization, Operation and Outp ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Hypercube
In geometry, a hypercube is an ''n''-dimensional analogue of a square () and a cube (). It is a closed, compact, convex figure whose 1- skeleton consists of groups of opposite parallel line segments aligned in each of the space's dimensions, perpendicular to each other and of the same length. A unit hypercube's longest diagonal in ''n'' dimensions is equal to \sqrt. An ''n''-dimensional hypercube is more commonly referred to as an ''n''-cube or sometimes as an ''n''-dimensional cube. The term measure polytope (originally from Elte, 1912) is also used, notably in the work of H. S. M. Coxeter who also labels the hypercubes the γn polytopes. The hypercube is the special case of a hyperrectangle (also called an ''n-orthotope''). A ''unit hypercube'' is a hypercube whose side has length one unit. Often, the hypercube whose corners (or ''vertices'') are the 2''n'' points in R''n'' with each coordinate equal to 0 or 1 is called ''the'' unit hypercube. Construction A hyp ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Broadcasting (networking)
In computer networking, telecommunication and information theory, broadcasting is a method of transferring a message to all recipients simultaneously. Broadcasting can be performed as a high-level operation in a program, for example, broadcasting in Message Passing Interface, or it may be a low-level networking operation, for example broadcasting on Ethernet. All-to-all communication is a computer communication method in which each sender transmits messages to all receivers within a group. In networking this can be accomplished using broadcast or multicast. This is in contrast with the point-to-point method in which each sender communicates with one receiver. Addressing methods There are four principal addressing methods in the Internet Protocol: Overview In computer networking, broadcasting refers to transmitting a packet that will be received by every device on the network. In practice, the scope of the broadcast is limited to a broadcast domain. Broadcasting is the ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Reduce (parallel Pattern)
In computer science, the reduction operator is a type of operator that is commonly used in parallel programming to reduce the elements of an array into a single result. Reduction operators are associative and often (but not necessarily) commutative.SolihinChandra p. 59 The reduction of sets of elements is an integral part of programming models such as Map Reduce, where a reduction operator is applied ( mapped) to all elements before they are reduced. Other parallel algorithms use reduction operators as primary operations to solve more complex problems. Many reduction operators can be used for broadcasting to distribute data to all processors. Theory A reduction operator can help break down a task into various partial tasks by calculating partial results which can be used to obtain a final result. It allows certain serial operations to be performed in parallel and the number of steps required for those operations to be reduced. A reduction operator stores the result of the partial ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Prefix Sum
In computer science, the prefix sum, cumulative sum, inclusive scan, or simply scan of a sequence of numbers is a second sequence of numbers , the sums of prefixes ( running totals) of the input sequence: : : : :... For instance, the prefix sums of the natural numbers are the triangular numbers: : Prefix sums are trivial to compute in sequential models of computation, by using the formula to compute each output value in sequence order. However, despite their ease of computation, prefix sums are a useful primitive in certain algorithms such as counting sort,. and they form the basis of the scan higher-order function in functional programming languages. Prefix sums have also been much studied in parallel algorithms, both as a test problem to be solved and as a useful primitive to be used as a subroutine in other parallel algorithms.. Abstractly, a prefix sum requires only a binary associative operator ⊕, making it useful for many applications from calculating well-separated pai ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Hypergraph Communication Pattern
In mathematics, a hypergraph is a generalization of a graph in which an edge can join any number of vertices. In contrast, in an ordinary graph, an edge connects exactly two vertices. Formally, an undirected hypergraph H is a pair H = (X,E) where X is a set of elements called ''nodes'' or ''vertices'', and E is a set of non-empty subsets of X called ''hyperedges'' or ''edges''. Therefore, E is a subset of \mathcal(X) \setminus\, where \mathcal(X) is the power set of X. The size of the vertex set is called the ''order of the hypergraph'', and the size of edges set is the ''size of the hypergraph''. A directed hypergraph differs in that its hyperedges are not sets, but ordered pairs of subsets of X, with each pair's first and second entries constituting the tail and head of the hyperedge respectively. While graph edges connect only 2 nodes, hyperedges connect an arbitrary number of nodes. However, it is often desirable to study hypergraphs where all hyperedges have the same c ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Prefix Sum
In computer science, the prefix sum, cumulative sum, inclusive scan, or simply scan of a sequence of numbers is a second sequence of numbers , the sums of prefixes ( running totals) of the input sequence: : : : :... For instance, the prefix sums of the natural numbers are the triangular numbers: : Prefix sums are trivial to compute in sequential models of computation, by using the formula to compute each output value in sequence order. However, despite their ease of computation, prefix sums are a useful primitive in certain algorithms such as counting sort,. and they form the basis of the scan higher-order function in functional programming languages. Prefix sums have also been much studied in parallel algorithms, both as a test problem to be solved and as a useful primitive to be used as a subroutine in other parallel algorithms.. Abstractly, a prefix sum requires only a binary associative operator ⊕, making it useful for many applications from calculating well-separated pai ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Hypergraph Communication Steps For Prefix Sum
In mathematics, a hypergraph is a generalization of a graph in which an edge can join any number of vertices. In contrast, in an ordinary graph, an edge connects exactly two vertices. Formally, an undirected hypergraph H is a pair H = (X,E) where X is a set of elements called ''nodes'' or ''vertices'', and E is a set of non-empty subsets of X called ''hyperedges'' or ''edges''. Therefore, E is a subset of \mathcal(X) \setminus\, where \mathcal(X) is the power set of X. The size of the vertex set is called the ''order of the hypergraph'', and the size of edges set is the ''size of the hypergraph''. A directed hypergraph differs in that its hyperedges are not sets, but ordered pairs of subsets of X, with each pair's first and second entries constituting the tail and head of the hyperedge respectively. While graph edges connect only 2 nodes, hyperedges connect an arbitrary number of nodes. However, it is often desirable to study hypergraphs where all hyperedges have the same c ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
