Class Parser
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescription(package private) static enum
private class
An operator-precedence parser for the binary operations we understand. -
Field Summary
FieldsModifier and TypeFieldDescriptionprivate static final com.google.common.base.CharMatcher
private static final com.google.common.base.CharMatcher
private int
The invariant of this parser is thatc
is always the next character of interest.private static final com.google.common.collect.ImmutableListMultimap<Integer,
Parser.Operator> Maps a code point to the operators that begin with that code point.private static final int
private static final com.google.common.base.CharMatcher
private int
A single character of pushback.private final LineNumberReader
private final String
private final Template.ResourceOpener
-
Constructor Summary
ConstructorsConstructorDescriptionParser
(Reader reader, String resourceName, Template.ResourceOpener resourceOpener) -
Method Summary
Modifier and TypeMethodDescriptionprivate void
expect
(char expected) Skips any space in the reader, and then throws an exception if the first non-space character found is not the expected one.private static boolean
isAsciiDigit
(int c) private static boolean
isAsciiLetter
(int c) private static boolean
isIdChar
(int c) private int
private void
next()
Gets the next character from the reader and assigns it toc
.private void
Gets the next character from the reader, and if it is a space character, keeps reading until a non-space character is found.(package private) Template
parse()
Parse the input completely to produce aTemplate
.private Node
Parses and discards a block comment, which is#*
followed by everything up to and including the next*#
.private ExpressionNode
Parses a boolean literal, eithertrue
orfalse
.private Node
Parses a single directive token from the reader.private ParseException
parseException
(String message) Returns an exception to be thrown describing a parse error with the given message, and including information about where it occurred.private ExpressionNode
Parses an expression, which can occur within a directive like#if
or#set
, or within a reference like$x[$a + $b]
or$x.m($a + $b)
.private Node
Parses a#foreach
token from the reader.private Node
private String
Parse an identifier as specified by the VTL .private Node
parseIfOrElseIf
(String directive) Parses the condition following#if
or#elseif
.private ExpressionNode
parseIntLiteral
(String prefix) private Node
Parses and discards a line comment, which is##
followed by any number of characters up to and including the next newline.private Node
Parses a#macro
token from the reader.private Node
Parses a single node from the reader, as part of the first parsing phase.private Node
Parses a single non-directive node from the reader.private Node
Parses a#parse
token from the reader.private Node
parsePlainText
(int firstChar) Parses plain text, which is text that contains neither$
nor#
.private Node
private Node
parsePossibleMacroCall
(String directive) Parses an identifier after#
that is not one of the standard directives.private ExpressionNode
Parses an expression containing only literals or references.private Node
Parses a reference, which is everything that can start with a$
.private ReferenceNode
Parses an index suffix to a method, like$x[$i]
.private ReferenceNode
Parses a reference member, which is either a property reference like$x.y
or a method call like$x.y($z)
.private ReferenceNode
Parses the parameters to a method reference, like$foo.bar($a, $b)
.private ReferenceNode
Parses a reference, in the simple form without braces.private ReferenceNode
Parses the modifiers that can appear at the tail of a reference.private ReferenceNode
Same asparseReference()
, except it really must be a reference.private Node
parseSet()
Parses a#set
token from the reader.private ExpressionNode
private com.google.common.collect.ImmutableList<Node>
private ExpressionNode
Parses an expression not containing any operators (except inside parentheses).private void
pushback
(int c1) Saves the current characterc
to be read again, and setsc
to the givenc1
.private String
private void
Ifc
is a space character, keeps reading untilc
is a non-space character or there are no more characters.
-
Field Details
-
EOF
private static final int EOF- See Also:
-
reader
-
resourceName
-
resourceOpener
-
c
private int cThe invariant of this parser is thatc
is always the next character of interest. This means that we almost never have to "unget" a character by reading too far. For example, after we parse an integer,c
will be the first character after the integer, which is exactly the state we will be in when there are no more digits.Sometimes we need to read two characters ahead, and in that case we use
pushback
. -
pushback
private int pushbackA single character of pushback. If this is not negative, thenext()
method will return it instead of reading a character. -
CODE_POINT_TO_OPERATORS
private static final com.google.common.collect.ImmutableListMultimap<Integer,Parser.Operator> CODE_POINT_TO_OPERATORSMaps a code point to the operators that begin with that code point. For example, maps<
toLESS
andLESS_OR_EQUAL
. -
ASCII_LETTER
private static final com.google.common.base.CharMatcher ASCII_LETTER -
ASCII_DIGIT
private static final com.google.common.base.CharMatcher ASCII_DIGIT -
ID_CHAR
private static final com.google.common.base.CharMatcher ID_CHAR
-
-
Constructor Details
-
Parser
Parser(Reader reader, String resourceName, Template.ResourceOpener resourceOpener) throws IOException - Throws:
IOException
-
-
Method Details
-
parse
Parse the input completely to produce aTemplate
.Parsing happens in two phases. First, we parse a sequence of "tokens", where tokens include entire references such as
${x.foo()[23]}
or entire directives such as#set ($x = $y + $z)
But tokens do not span complex constructs. For example,#if ($x == $y) something #end
is three tokens:#if ($x == $y) (literal text " something ") #end
The second phase then takes the sequence of tokens and constructs a parse tree out of it. Some nodes in the parse tree will be unchanged from the token sequence, such as the
${x.foo()[23]} #set ($x = $y + $z)
examples above. But a construct such as the#if ... #end
mentioned above will become a single IfNode in the parse tree in the second phase.The main reason for this approach is that Velocity has two kinds of lexical contexts. At the top level, there can be arbitrary literal text; references like
${x.foo()}
; and directives like#if
or#set
. Inside the parentheses of a directive, however, neither arbitrary text nor directives can appear, but expressions can, so we need to tokenize the inside of#if ($x == $a + $b)
as the five tokens "$x", "==", "$a", "+", "$b". Rather than having a classical parser/lexer combination, where the lexer would need to switch between these two modes, we replace the lexer with an ad-hoc parser that is the first phase described above, and we define a simple parser over the resultant tokens that is the second phase.- Throws:
IOException
-
parseTokens
- Throws:
IOException
-
lineNumber
private int lineNumber() -
next
Gets the next character from the reader and assigns it toc
. If there are no more characters, setsc
toEOF
if it is not already.- Throws:
IOException
-
pushback
private void pushback(int c1) Saves the current characterc
to be read again, and setsc
to the givenc1
. Suppose the text containsxy
and we have just ready
. Soc == 'y'
. Now if we executepushback('x')
, we will havec == 'x'
and the next call tonext()
will setc == 'y'
. Subsequent calls tonext()
will continue reading fromreader
. So the pushback essentially puts us back in the state we were in before we ready
. -
skipSpace
Ifc
is a space character, keeps reading untilc
is a non-space character or there are no more characters.- Throws:
IOException
-
nextNonSpace
Gets the next character from the reader, and if it is a space character, keeps reading until a non-space character is found.- Throws:
IOException
-
expect
Skips any space in the reader, and then throws an exception if the first non-space character found is not the expected one. Setsc
to the first character after that expected one.- Throws:
IOException
-
parseNode
Parses a single node from the reader, as part of the first parsing phase.<template> -> <empty> | <directive> <template> | <non-directive> <template>
- Throws:
IOException
-
parseHashSquare
- Throws:
IOException
-
parseNonDirective
Parses a single non-directive node from the reader.<non-directive> -> <reference> | <text containing neither $ nor #>
- Throws:
IOException
-
parseDirective
Parses a single directive token from the reader. Directives can be spelled with or without braces, for example#if
or#{if}
. We omit the brace spelling in the productions here:<directive> -> <if-token> | <else-token> | <elseif-token> | <end-token> | <foreach-token> | <set-token> | <parse-token> | <macro-token> | <macro-call> | <comment>
- Throws:
IOException
-
parseIfOrElseIf
Parses the condition following#if
or#elseif
.<if-token> -> #if ( <condition> ) <elseif-token> -> #elseif ( <condition> )
- Parameters:
directive
- either"if"
or"elseif"
.- Throws:
IOException
-
parseForEach
Parses a#foreach
token from the reader.<foreach-token> -> #foreach ( $<id> in <expression> )
- Throws:
IOException
-
parseSet
Parses a#set
token from the reader.<set-token> -> #set ( $<id> = <expression>)
- Throws:
IOException
-
parseParse
Parses a#parse
token from the reader.<parse-token> -> #parse ( <string-literal> )
The way this works is inconsistent with Velocity. In Velocity, the
#parse
directive is evaluated when it is encountered during template evaluation. That means that the argument can be a variable, and it also means that you can use#if
to choose whether or not to do the#parse
. Neither of those is true in EscapeVelocity. The contents of the#parse
are integrated into the containing template pretty much as if they had been written inline. That also means that EscapeVelocity allows forward references to macros inside#parse
directives, which Velocity does not.- Throws:
IOException
-
parseMacroDefinition
Parses a#macro
token from the reader.<macro-token> -> #macro ( <id> <macro-parameter-list> ) <macro-parameter-list> -> <empty> | $<id> <macro-parameter-list>
Macro parameters are not separated by commas, though method-reference parameters are.
- Throws:
IOException
-
parsePossibleMacroCall
Parses an identifier after#
that is not one of the standard directives. The assumption is that it is a call of a macro that is defined in the template. Macro definitions are extracted from the template during the second parsing phase (and not during evaluation of the template as you might expect). This means that a macro can be called before it is defined.<macro-call> -> # <id> ( <expression-list> ) <expression-list> -> <empty> | <expression> <optional-comma> <expression-list> <optional-comma> -> <empty> | ,
- Throws:
IOException
-
parseLineComment
Parses and discards a line comment, which is##
followed by any number of characters up to and including the next newline.- Throws:
IOException
-
parseBlockComment
Parses and discards a block comment, which is#*
followed by everything up to and including the next*#
.- Throws:
IOException
-
parsePlainText
Parses plain text, which is text that contains neither$
nor#
. The givenfirstChar
is the first character of the plain text, andc
is the second (if the plain text is more than one character).- Throws:
IOException
-
parsePlainText
- Throws:
IOException
-
parseReference
Parses a reference, which is everything that can start with a$
. References can optionally be enclosed in braces, so$x
and${x}
are the same. Braces are useful when text after the reference would otherwise be parsed as part of it. For example,${x}y
is a reference to the variable$x
, followed by the plain texty
. Of course$xy
would be a reference to the variable$xy
.<reference> -> $<reference-no-brace> | ${<reference-no-brace>}
On entry to this method,
c
is the character immediately after the$
.- Throws:
IOException
-
parseRequiredReference
Same asparseReference()
, except it really must be a reference. A$
in normal text doesn't start a reference if it is not followed by an identifier. But in an expression, for example in#if ($x == 23)
,$
must be followed by an identifier.- Throws:
IOException
-
parseReferenceNoBrace
Parses a reference, in the simple form without braces.<reference-no-brace> -> <id><reference-suffix>
- Throws:
IOException
-
parseReferenceSuffix
Parses the modifiers that can appear at the tail of a reference.<reference-suffix> -> <empty> | <reference-member> | <reference-index>
- Parameters:
lhs
- the reference node representing the first part of the reference$x
in$x.foo
or$x.foo()
, or later$x.y
in$x.y.z
.- Throws:
IOException
-
parseReferenceMember
Parses a reference member, which is either a property reference like$x.y
or a method call like$x.y($z)
.<reference-member> -> .<id><reference-property-or-method><reference-suffix> <reference-property-or-method> -> <id> | <id> ( <method-parameter-list> )
- Parameters:
lhs
- the reference node representing what appears to the left of the dot, like the$x
in$x.foo
or$x.foo()
.- Throws:
IOException
-
parseReferenceMethodParams
Parses the parameters to a method reference, like$foo.bar($a, $b)
.<method-parameter-list> -> <empty> | <non-empty-method-parameter-list> <non-empty-method-parameter-list> -> <expression> | <expression> , <non-empty-method-parameter-list>
- Parameters:
lhs
- the reference node representing what appears to the left of the dot, like the$x
in$x.foo()
.- Throws:
IOException
-
parseReferenceIndex
Parses an index suffix to a method, like$x[$i]
.<reference-index> -> [ <expression> ]
- Parameters:
lhs
- the reference node representing what appears to the left of the dot, like the$x
in$x[$i]
.- Throws:
IOException
-
parseExpression
Parses an expression, which can occur within a directive like#if
or#set
, or within a reference like$x[$a + $b]
or$x.m($a + $b)
.<expression> -> <and-expression> | <expression> || <and-expression> <and-expression> -> <relational-expression> | <and-expression> && <relational-expression> <equality-exression> -> <relational-expression> | <equality-expression> <equality-op> <relational-expression> <equality-op> -> == | != <relational-expression> -> <additive-expression> | <relational-expression> <relation> <additive-expression> <relation> -> < | <= | > | >= <additive-expression> -> <multiplicative-expression> | <additive-expression> <add-op> <multiplicative-expression> <add-op> -> + | - <multiplicative-expression> -> <unary-expression> | <multiplicative-expression> <mult-op> <unary-expression> <mult-op> -> * | / | %
- Throws:
IOException
-
parseUnaryExpression
Parses an expression not containing any operators (except inside parentheses).<unary-expression> -> <primary> | ( <expression> ) | ! <unary-expression>
- Throws:
IOException
-
parsePrimary
Parses an expression containing only literals or references.<primary> -> <reference> | <string-literal> | <integer-literal> | <boolean-literal>
- Throws:
IOException
-
parseStringLiteral
- Throws:
IOException
-
readStringLiteral
- Throws:
IOException
-
parseIntLiteral
- Throws:
IOException
-
parseBooleanLiteral
Parses a boolean literal, eithertrue
orfalse
. <boolean-literal> -> true | false- Throws:
IOException
-
isAsciiLetter
private static boolean isAsciiLetter(int c) -
isAsciiDigit
private static boolean isAsciiDigit(int c) -
isIdChar
private static boolean isIdChar(int c) -
parseId
Parse an identifier as specified by the VTL . Identifiers are ASCII: starts with a letter, then letters, digits,-
and_
.- Throws:
IOException
-
parseException
Returns an exception to be thrown describing a parse error with the given message, and including information about where it occurred.- Throws:
IOException
-