Jaql

Jaql (pronounced "jackal") is a functional data processing and query language most commonly used for JSON query processing on big data.

It started as an open source project at Google but the latest release was on 2010-07-12. IBM took it over as primary data processing language for their Hadoop software packag
BigInsights

Although having been developed for JSON it supports a variety of other data sources like CSV, TSV, XML.

A comparison to other BigData query languages like PIG Latin and Hive QL illustrates performance and usability aspects of these technologies.

Jaql supports lazy evaluation, so expressions are only materialized when needed.

Syntax

The basic concept of Jaql is

source -> operator(parameter) -> sink ;

where a sink can be a source for a downstream operator. So typically a Jaql program has to following structure, expressing a data processing graph:

source -> operator1(parameter) -> operator2(parameter) -> operator2(parameter) -> operator3(parameter) -> operator4(parameter) -> sink ;

Most commonly for readability reasons Jaql programs are linebreaked after the arrow, as is also a common idiom in Twitte
Scalding

source -> operator1(parameter)
-> operator2(parameter)
-> operator2(parameter)
-> operator3(parameter)
-> operator4(parameter)
-> sink ;

Core operators
/ref>

Expand

Use the EXPAND expression to flatten nested arrays. This expression takes as input an array of nested arrays [ T ">[_T_.html" ;"title="[ T ">[ T and produces an output array [ T ">[_T_.html"_;"title="[_T_">[_T_<_a>.html" ;"title="[_T_.html" ;"title="[ T ">[ T ">[_T_.html" ;"title="[ T ">[ T and produces an output array [ T by promoting the elements of each nested array to the top-level output array.

Filter

Use the FILTER operator to filter away elements from the specified input array. This operator takes as input an array of elements of type T and outputs an array of the same type, retaining those elements for which a predicate evaluates to true. It is the Jaql equivalent of the SQL WHERE clause.
Example:

data = Where (SQL)">SQL WHERE clause.
Example:

data = [
,
,
,

data -> filter $.manager;

[

"> ,
,
,

">Where (SQL)">SQL WHERE clause.
Example:

data = [
,
,
,

data -> filter $.manager;

[

data -> filter $.income < 30000;

[
,

]

Group

Use the GROUP expression to group one or more input arrays on a grouping key and applies an aggregate function per group.

Join

Use the JOIN operator to express a join between two or more input arrays. This operator supports multiple types of joins, including natural, left-outer, right-outer, and outer joins.

Sort

Use the SORT operator to sort an input by one or more fields.

Top

The TOP expression selects the first k elements of its input. If a comparator is provided, the output is semantically equivalent to sorting the input, then selecting the first elements.

Transform

Use the TRANSFORM operator to realize a projection or to apply a function to all items of an output.

See also

* jq
* JSONiq
* XPath
* XQuery

References

External links

Definition of the JAQL language

JAQL Introduction


and PIG]

Adaptive Processing of User-Defined Aggregates in Jaql

{{Query languages

2008 software
Query languages
JSON