The query language allows you to run mapreduce queries on log files. This page is the reference to the language.
For this to work, DTail needs to understand your log format. DTail already understands its own log format. You can have a look at all examples of the examples page using
-query (these would be all examples of the
dmap command, and some examples using the
DTail also ships with a generic log format, which only allows you to run very basic queries. Check out the log format documentation for this. That page also documents how to implement your own log format parser.
This are the fundamental types of the query language:
NUMBER := A whole number (e.g. 42) FLOAT := A float number, e.g. 3.14 STRING := A quoted string, e.g. "foo" FIELD := BAREWORD|$VARIABLE BAREWORD := A bare string without quotes, e.g. foo. This usually contains a value extracted from a log line. $VARIABLE := Like a bareword, but with a $ prefix, e.g. $foo. This usually contains a special value set by DTail itself (not necessary from the log line).
This is the overall structure of a query:
QUERY := select SELECT1[,SELECT2...] [from TABLE] [where CONDITION1[,CONDITION2...]] [group by FIELD1[,FIELD2...]] [order|rorder by ORDERFIELD] [set SET1,[,SET2...]] [interval NUMBER] [limit NUMBER] [outfile [append] STRING] [logformat LOGFORMAT]
TABLE := The mapreduce table name, e.g. STATS in MAPREDUCE:STATS SELECT := FIELD|AGGREGATION(FIELD) CONDITION := ARG1 OPERATOR ARG2 ARG := FIELD|FLOAT|STRING OPERATOR := FLOATOPERATOR|STRINGOPERATOR FLOATOPERATOR := One of: == != < <= > >= STRINGOPERATOR := eq|ne|contains|ncontains|lacks|hasprefix|nhasprefix|hassuffix|nhassuffix ORDERFIELD := FIELD|AGGREGATION(FIELD) SET := $VARIABLE = FLOAT|STRING|FIELD|FUNCTION(FIELD) LOGFORMAT := default|generic|generickv|... AGGREGATION := count|sum|min|max|avg|last|len FUNCTION := md5sum|maskdigits
rorderstands for reverse order.
lacksis an alias for