AWK(1) | General Commands Manual | AWK(1) |
awk
—
pattern-directed scanning and processing
language
awk |
[-safe ] [-V ]
[-d [n]]
[-F fs]
[-v
var=value]
[prog | -f
progfile] file ... |
awk
scans each input
file for lines that match any of a set of patterns
specified literally in prog or in one or more files
specified as -f
progfile. With
each pattern there can be an associated action that will be performed when a
line of a file matches the pattern. Each line is
matched against the pattern portion of every pattern-action statement; the
associated action is performed for each matched pattern. The file name
‘-’ means the standard input. Any file
of the form var=value is treated
as an assignment, not a filename, and is executed at the time it would have
been opened if it were a filename.
The options are as follows:
-d
[n]awk
to dump core on fatal errors.-F
fs-f
progfile-safe
print
>, print
>>), process creation
(cmd | getline
,
print
|, system
) and
access to the environment (ENVIRON; see the section
on variables below). This is a first (and not very reliable) approximation
to a “safe” version of awk
.-V
awk
to standard output
and exit.-v
var=value-v
options may be present.The input is normally made up of input lines (records) separated by newlines, or by the value of RS. If RS is null, then any number of blank lines are used as the record separator, and newlines are used as field separators (in addition to the value of FS). This is convenient when working with multi-line records.
An input line is normally made up of fields separated by whitespace, or by the value of the field separator FS at the time the line is read. The fields are denoted $1, $2, ..., while $0 refers to the entire line. FS may be set to either a single character or a regular expression. As a special case, if FS is a single space (the default), fields will be split by one or more whitespace characters. If FS is null, the input line is split into one field per character.
Normally, any number of blanks separate fields. In order to set
the field separator to a single blank, use the -F
option with a value of ‘[ ]’. If a field separator of
‘t’ is specified, awk
treats it as if
‘\t’ had been specified and uses ⟨TAB⟩ as the
field separator. In order to use a literal ‘t’ as the field
separator, use the -F
option with a value of
‘[t]’. The field separator is usually set via the
-F
option or from inside a
BEGIN
block so that it takes effect before the input
is read.
A pattern-action statement has the form:
{
action
}
A missing {
action
}
means print the line; a missing pattern always
matches. Pattern-action statements are separated by newlines or
semicolons.
Newlines are permitted after a terminating statement or following a comma (‘,’), an open brace (‘{’), a logical AND (‘&&’), a logical OR (‘||’), after the ‘do’ or ‘else’ keywords, or after the closing parenthesis of an ‘if’, ‘for’, or ‘while’ statement. Additionally, a backslash (‘\’) can be used to escape a newline between tokens.
An action is a sequence of statements. A statement can be one of the following:
if
(expression) statement
[else
statement]while
(expression) statementfor
(expression; expression;
expression) statementfor
(var in
array) statementdo
statement while
(expression)break
continue
{
[statement ...] }
print
[expression-list]
[>expression]printf
format [...,
expression-list]
[>expression]return
[expression]next
# skip remaining patterns on this input linenextfile
# skip rest of this file, open next, start at
topdelete
array[
expression]
# delete an array elementdelete
array # delete all elements of
arrayexit
[expression] # exit processing, and
perform END
processing; status is
expressionStatements are terminated by semicolons, newlines or right braces.
An empty expression-list stands for
$0. String constants are quoted
""
, with the usual C escapes recognized
within (see printf(1) for a complete list
of these). Expressions take on string or numeric values as appropriate, and
are built using the operators + - * / % ^
(exponentiation), and concatenation (indicated by whitespace). The operators
! ++ -- += -= *= /= %= ^=
>
>= < <= == != ?:
are also available in expressions.
Variables may be scalars, array elements (denoted
x[i]
) or fields. Variables are initialized to the
null string. Array subscripts may be any string, not necessarily numeric;
this allows for a form of associative memory. Multiple subscripts such as
[i,j,k]
are permitted; the constituents are
concatenated, separated by the value of SUBSEP (see
the section on variables below).
The print
statement prints its arguments
on the standard output (or on a file if
> file or
>> file is present or on a pipe if
| cmd is present), separated by the current
output field separator, and terminated by the output record separator.
file and cmd may be literal
names or parenthesized expressions; identical string values in different
statements denote the same open file. The printf
statement formats its expression list according to the
format (see
printf(1)).
Patterns are arbitrary Boolean combinations (with
! || &&
) of regular expressions and
relational expressions. awk
supports extended
regular expressions (EREs). See
re_format(7) for more information on
regular expressions. Isolated regular expressions in a pattern apply to the
entire line. Regular expressions may also occur in relational expressions,
using the operators ~
and
!~
. /re/ is a constant regular
expression; any string (constant or variable) may be used as a regular
expression, except in the position of an isolated regular expression in a
pattern.
A pattern may consist of two patterns separated by a comma; in this case, the action is performed for all lines from an occurrence of the first pattern through an occurrence of the second.
A relational expression is one of the following:
in
array-name(
expr,
expr, ...)
in
array-namewhere a relop is any of the six relational
operators in C, and a matchop is either
~
(matches) or !~
(does not
match). A conditional is an arithmetic expression, a relational expression,
or a Boolean combination of these.
The special pattern BEGIN
may be used to
capture control before the first input line is read. The special pattern
END
may be used to capture control after processing
is finished. BEGIN
and END
do not combine with other patterns. They may appear multiple times in a
program and execute in the order they are read by
awk
.
Variable names with special meanings:
%.6g
").-F
fs.%.6g
").match
()
function.match
() function.The awk language has a variety of built-in functions: arithmetic, string, input/output, general, and bit-operation.
Functions may be defined (at the position of a pattern-action statement) thusly:
function foo(a, b, c) { ...; return x
}
Parameters are passed by value if scalar, and by reference if array name; functions may be called recursively. Parameters are local to the function; all other variables are global. Thus local variables may be created by providing excess parameters in the function definition.
atan2
(y,
x)cos
(x)exp
(x)int
(x)log
(x)rand
()srand
().sin
(x)sqrt
(x)srand
(expr)rand
() to expr
and returns the previous seed. If expr is omitted,
rand
() will return non-deterministic random
numbers.gensub
(r,
s, h,
[t])g
or
G
, then replace all matches of
r with s. Otherwise,
h is a number indicating which match of
r to replace. If no t is
supplied, $0 is used instead. Unlike
sub
()
and
gsub
(),
the modified string is returned as the result of the function, and the
original target is not changed. Note that
\n sequences within the replacement string
s, as supported by GNU awk
,
are not supported at this time.gsub
(r,
t, s)sub
() except that all occurrences of
the regular expression are replaced. gsub
()
returns the number of replacements.index
(s,
t)length
(s)match
(s,
r)split
(s,
a, fs)sprintf
(fmt,
expr, ...)sub
(r,
t, s)sub
() returns the number of
replacements.substr
(s,
m, n)tolower
(str)toupper
(str)This version of awk
provides the following
functions for obtaining and formatting time stamps.
mktime
(datespec)systime
(). The
datespec is a string composed of six or seven
numbers separated by whitespace:
YYYY MM DD HH MM SS [DST]
The fields in datespec are as follows:
mktime
()
will attempt to determine the correct value.strftime
([format
[, timestamp]])mktime
() and
systime
().
If timestamp is not specified, the current time is
used. If format is not specified, a default format
equivalent to the output of date(1) is
used.systime
()close
(expr)getline
[var]close
().
getline
returns 1 for a successful input, 0 for
end of file, and -1 for an error.fflush
([expr])getline
getline
sets the
variables NF, NR, and
FNR. getline
returns 1 for a
successful input, 0 for end of file, and -1 for an error.getline
vargetline
sets the variables
NR and FNR.
getline
returns 1 for a successful input, 0 for
end of file, and -1 for an error.getline
[var] < fileclose
().system
(cmd)compl
(x)and
(x,
y)or
(x,
y)xor
(x,
y)lshift
(x,
n)rshift
(x,
n)The following environment variables affect the execution of
awk
:
POSIXLY_CORRECT
The awk
utility exits 0 on success,
and >0 if an error occurs.
But note that the exit
expression can
modify the exit status.
Print lines longer than 72 characters:
length($0) > 72
Print first two fields in opposite order:
{ print $2, $1 }
Same, with input fields separated by comma and/or spaces and tabs:
BEGIN { FS = ",[ \t]*|[ \t]+" } { print $2, $1 }
Add up first column, print sum and average:
{ s += $1 } END { print "sum is", s, " average is", s/NR }
Print all lines between start/stop pairs:
/start/, /stop/
Simulate echo(1):
BEGIN { # Simulate echo(1) for (i = 1; i < ARGC; i++) printf "%s ", ARGV[i] printf "\n" exit }
Print an error message to standard error:
{ print "error!" > "/dev/stderr" }
awk
was designed before IEEE 754
arithmetic defined Not-A-Number (NaN) and Infinity values, which are
supported by all modern floating-point hardware.
Because awk
uses
strtod(3) and
atof(3) to convert string values to
double-precision floating-point values, modern C libraries also convert
strings starting with inf
and
nan
into infinity and NaN values respectively. This
led to strange results, with something like this:
echo nancy
| awk '{ print
$1 + 0 }'
printing nan
instead of zero.
awk
now follows GNU
awk
, and prefilters string values before attempting
to convert them to numbers, as follows:
cut(1), date(1), grep(1), lex(1), printf(1), sed(1), strftime(3), re_format(7), script(7)
A. V. Aho, P. J. Weinberger, and B. W. Kernighan, AWK — A Pattern Scanning and Processing Language, Software — Practice and Experience, 9:4, pp. 267-279, April 1979.
A. V. Aho, B. W. Kernighan, and P. J. Weinberger, The AWK Programming Language, Addison-Wesley, 1988, ISBN 0-201-07981-X.
The awk
utility is compliant with the
IEEE Std 1003.1-2008 (“POSIX.1”)
specification except that consecutive backslashes in the replacement string
argument for sub
() and
gsub
() are not collapsed and a slash
(‘/
’) does not need to be escaped in a
bracket expression. Also, the behaviour of rand
()
and srand
() has been changed to support
non-deterministic random numbers.
The flags [-dV
] and
[-safe
], support for regular expressions in
RS, as well as the functions
fflush
(), gensub
(),
compl
(), and
(),
or
(), xor
(),
lshift
(), rshift
(),
mktime
(), strftime
() and
systime
() are extensions to that specification.
An awk
utility appeared in
Version 7 AT&T UNIX.
There are no explicit conversions between numbers and strings. To
force an expression to be treated as a number add 0 to it; to force it to be
treated as a string concatenate ""
to
it.
The scope rules for variables in functions are a botch; the syntax is worse.
Only eight-bit character sets are handled correctly.
November 8, 2021 | OpenBSD-current |