grep
grep is a command-line utility for searching plain-text data sets for lines that match a regular expression. Its name comes from the ed command ''g/re/p'' (''globally search for a regular expression and print matching lines''), which has the sa ...
and awk
AWK (''awk'') is a domain-specific language designed for text processing and typically used as a data extraction and reporting tool. Like sed and grep, it is a filter, and is a standard feature of most Unix-like operating systems.
The AWK lang ...
can take input either as command-line arguments or from the standard input. However, others such as cp
and echo
In audio signal processing and acoustics, an echo is a reflection of sound that arrives at the listener with a delay after the direct sound. The delay is directly proportional to the distance of the reflecting surface from the source and the lis ...
can only take input as arguments, which is why xargs is necessary.
A port of an older version of GNU is available for Examples
One use case of the xargs command is to remove a list of files using the rm command.xargs
command to break the list of arguments into sublists small enough to be acceptable:
find
utility feeds the input of xargs
with a long list of file names. xargs
then splits this list into sublists and calls rm
once for every sublist.
Some implementations of xargs can also be used to parallelize operations with the -P maxprocs
argument to specify how many parallel processes should be used to execute the commands over the input argument lists. However, the output streams may not be synchronized. This can be overcome by using an --output file
argument where possible, and then combining the results after processing. The following example queues 24 processes and waits on each to finish before launching another.
`...`
or $(...)
). xargs is also a good companion for commands that output long lists of files such as find
Find, FIND or Finding may refer to:
Computing
* find (Unix), a command on UNIX platforms
* find (Windows), a command on DOS/Windows platforms
Books
* ''The Find'' (2010), by Kathy Page
* ''The Find'' (2014), by William Hope Hodgson
Film and t ...
, locate
and grep
grep is a command-line utility for searching plain-text data sets for lines that match a regular expression. Its name comes from the ed command ''g/re/p'' (''globally search for a regular expression and print matching lines''), which has the sa ...
, but only if one uses -0
(or equivalently --null
), since xargs
without -0
deals badly with file names containing '
, "
and space. '
, "
, and space (newline still requires -0
).
Placement of arguments
option: single argument
The xargs command offers options to insert the listed arguments at some position other than the end of the command line. The-I
option to xargs takes a string that will be replaced with the supplied input before the command is executed. A common choice is %
.
Shell trick: any number
Another way to achieve a similar effect is to use a shell as the launched command, and deal with the complexity in that shell, for example:$0
and the file wouldn't be copied to ~/backups
. One can also use any other word to fill in that blank, for example.
Since accepts multiple files at once, one can also simply do the following:
Separator problem
Many Unix utilities are line-oriented. These may work withxargs
as long as the lines do not contain '
, "
, or a space. Some of the Unix utilities can use NUL as record separator (e.g. -0
and \0
instead of \n
), locate
(requires using -0
), find
Find, FIND or Finding may refer to:
Computing
* find (Unix), a command on UNIX platforms
* find (Windows), a command on DOS/Windows platforms
Books
* ''The Find'' (2010), by Kathy Page
* ''The Find'' (2014), by William Hope Hodgson
Film and t ...
(requires using -print0
), grep
grep is a command-line utility for searching plain-text data sets for lines that match a regular expression. Its name comes from the ed command ''g/re/p'' (''globally search for a regular expression and print matching lines''), which has the sa ...
(requires -z
or -Z
), sort
(requires using -z
)). Using -0
for xargs
deals with the problem, but many Unix utilities cannot use NUL as separator (e.g. head
A head is the part of an organism which usually includes the ears, brain, forehead, cheeks, chin, eyes, nose, and mouth, each of which aid in various sensory functions such as sight, hearing, smell, and taste. Some very simple animals ...
, tail
The tail is the section at the rear end of certain kinds of animals’ bodies; in general, the term refers to a distinct, flexible appendage to the torso. It is the part of the body that corresponds roughly to the sacrum and coccyx in mammal ...
, ls
, echo
In audio signal processing and acoustics, an echo is a reflection of sound that arrives at the listener with a delay after the direct sound. The delay is directly proportional to the distance of the reflecting surface from the source and the lis ...
, sed
sed ("stream editor") is a Unix utility that parses and transforms text, using a simple, compact programming language. It was developed from 1973 to 1974 by Lee E. McMahon of Bell Labs,
and is available today for most operating systems.
sed w ...
, tar
Tar is a dark brown or black viscous liquid of hydrocarbons and free carbon, obtained from a wide variety of organic materials through destructive distillation. Tar can be produced from coal, wood, petroleum, or peat. "a dark brown or black bi ...
-v
, wc
, which
Which may refer to:
* a relative pronoun
* an interrogative word
* which (command), an operating system command
*Which?, a UK charity and its magazine
See also
* English relative clauses
* Interrogative clause
* Whicher (disambiguation) Which ...
).
But often people forget this and assume xargs
is also line-oriented, which is not the case (per default xargs
separates on newlines and blanks within lines, substrings with blanks must be single- or double-quoted).
The separator problem is illustrated here:
important_file
to be removed but will remove neither the directory called 12" records
, nor the file called not important_file
.
The proper fix is to use the GNU-specific -print0
option, but tail
(and other tools) do not support NUL-terminated strings:
-print0
option, entries are separated by a null character instead of an end-of-line. This is equivalent to the more verbose command:xargs
to (non-POSIX) line-oriented mode with the -d
(delimiter) option: -0
with -print0
should be preferred, since newlines in filenames are still a problem.
GNU parallel
Parallel is a geometric term of location which may refer to:
Computing
* Parallel algorithm
* Parallel computing
* Parallel metaheuristic
* Parallel (software), a UNIX utility for running programs in parallel
* Parallel Sysplex, a cluster o ...
is an alternative to xargs
that is designed to have the same options, but is line-oriented. Thus, using GNU Parallel instead, the above would work as expected.
For Unix environments where xargs
does not support the -0
nor the option (e.g. Solaris, AIX), the POSIX standard states that one can simply backslash-escape every character:Operating on a subset of arguments at a time
One might be dealing with commands that can only accept one or maybe two arguments at a time. For example, thediff
command operates on two files at a time. The -n
option to xargs
specifies how many arguments at a time to supply to the given command. The command will be invoked repeatedly until all input is exhausted. Note that on the last invocation one might get fewer than the desired number of arguments if there is insufficient input. Use xargs
to break up the input into two arguments per line:
-L 1
option. One can use an arbitrary number of lines at a time, but one is most common. Here is how one might diff
every git commit against its parent.
Encoding problem
The argument separator processing ofxargs
is not the only problem with using the xargs
program in its default mode. Most Unix tools which are often used to manipulate filenames (for example sed
, basename
, sort
, etc.) are text processing tools. However, Unix path names are not really text. Consider a path name /aaa/bbb/ccc. The /aaa directory and its bbb subdirectory can in general be created by different users with different environments. That means these users could have a different locale setup, and that means that aaa and bbb do not even necessarily have to have the same character encoding. For example, aaa could be in UTF-8 and bbb in Shift JIS. As a result, an absolute path name in a Unix system may not be correctly processable as text under a single character encoding. Tools which rely on their input being text may fail on such strings.
One workaround for this problem is to run such tools in the C locale, which essentially processes the bytes of the input as-is. However, this will change the behavior of the tools in ways the user may not expect (for example, some of the user's expectations about case-folding behavior may not be met).
References
External links
*Manual pages
* * * * * {{Unix commands Unix text processing utilities Unix SUS2008 utilities Plan 9 commands IBM i Qshell commands