Add option to preserve comments when parsing templates #2037
Conversation
| elif token == TOKEN_LINECOMMENT_BEGIN: | ||
| token = TOKEN_COMMENT_BEGIN | ||
| elif token == TOKEN_LINECOMMENT_END: | ||
| token = TOKEN_COMMENT_END |
There was a problem hiding this comment.
note: Possible (small?) performance hit. Not sure how to avoid it.
|
Not sure to understand the readthedocs build failure. |
| @@ -715,6 +715,13 @@ def as_const(self, eval_ctx: t.Optional[EvalContext] = None) -> t.Any: | |||
| return self.expr2.as_const(eval_ctx) | |||
|
|
|||
|
|
|||
| class Comment(Stmt): | |||
| """A template comment.""" | |||
There was a problem hiding this comment.
suggestion: Could maybe add .. versionadded:: 3.2, is that a valid directive?
|
The docs failure is unrelated and something I'm actively working on, you can ignore it. What if we just added these nodes to the AST unconditionally? I'm not sure that would be a breaking change, as an existing walker would skip those nodes. I'd rather not have more configuration if possible. |
Totally fine by me! I definitely understand not wanting more configuration. I'll run a few code searches on GitHub to see if this would break anything, and then do the change. |
|
This would break jinja2schema: https://github.com/aromanovich/jinja2schema. Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/media/data/dev/jinja/.venv/lib/python3.12/site-packages/jinja2schema/core.py", line 49, in infer_from_ast
rv = visit(ast, {}, config)
^^^^^^^^^^^^^^^^^^^^^^
File "/media/data/dev/jinja/.venv/lib/python3.12/site-packages/jinja2schema/visitors/util.py", line 20, in visit
structure = visit_many(node.body, macroses, config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/data/dev/jinja/.venv/lib/python3.12/site-packages/jinja2schema/visitors/util.py", line 37, in visit_many
structure = visit(node, macroses, config, predicted_struct_cls, return_struct_cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/data/dev/jinja/.venv/lib/python3.12/site-packages/jinja2schema/visitors/util.py", line 14, in visit
structure = visit_stmt(node, macroses, config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/data/dev/jinja/.venv/lib/python3.12/site-packages/jinja2schema/visitors/stmt.py", line 53, in visit_stmt
raise Exception('stmt visitor for {0} is not found'.format(type(ast)))
Exception: stmt visitor for <class 'jinja2.nodes.Comment'> is not foundIt hasn't been updated in 7 years though. |
|
Other than that, almost all uses of |
| if not isinstance(body_node, (nodes.Output, nodes.Comment)) or ( | ||
| isinstance(body_node, nodes.Output) |
There was a problem hiding this comment.
note: This was needed to fix the logic and two failing tests:
FAILED tests/test_inheritance.py::TestInheritance::test_level1_required - jinja2.exceptions.TemplateSyntaxError: Required blocks can only contain comments or whitespace
FAILED tests/test_inheritance.py::TestInheritance::test_invalid_required - jinja2.exceptions.TemplateSyntaxError: Required blocks can only contain comments or whitespace
|
I've pushed a fixup commit that removes the new parameter and changes the code so that comments are always preserved in ASTs. Makes the overall change much smaller 🙂 |
Add thepreserve_commentsparameter toEnvironment.parseto preserve comments in template ASTs.Create a new
Commentnode, and always preserve those in ASTs obtained fromEnvironment.parse.Fixes #1967.
should fail without the change.
.. versionchanged::entries in any relevant code docs.