Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ng code generation #332

Merged
merged 51 commits into from
Dec 11, 2023
Merged
Changes from 1 commit
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
a25f0b9
[tool] build path for new code generation
apalala Nov 28, 2023
ad1f639
[walkrers] make walk_children public
apalala Nov 28, 2023
d87ac12
[grammar] fix name to TatSu
apalala Nov 28, 2023
caf280a
[ngcodegen] up to Buffer definition
apalala Nov 28, 2023
d59d2f4
[mixins][indent] allow control over the amount of indentation
apalala Nov 28, 2023
75d43b6
[ngcodegen] refactor
apalala Nov 28, 2023
e8cfa3b
[mixins][indent] clarify and refactor
apalala Nov 29, 2023
feff967
[lint] resolve warnings
apalala Nov 29, 2023
3a849fb
cleanup
apalala Nov 29, 2023
21cbaf9
[ngcodegen] allow naming the parser
apalala Nov 29, 2023
7074718
[python] generate up to rule templates
apalala Nov 29, 2023
fda5915
[lint] solve warnings
apalala Nov 29, 2023
e71087f
[ngcodegen] add more node types to walker
apalala Nov 29, 2023
0bdd945
[ngcodegen] add more node types to walker
apalala Nov 29, 2023
a057888
[ngcodegen] add more node types to walker
apalala Nov 29, 2023
8563ef1
[ngcodegen] add more node types to walker
apalala Nov 29, 2023
c3ab556
[ngcodegen] add more node types to walker
apalala Nov 29, 2023
367f730
[ngcodegen] add more node types to walker
apalala Nov 29, 2023
37c0dc4
[ngcodegen] bug fixes
apalala Nov 29, 2023
82a5b90
[ngcodegen] fix bugs
apalala Nov 29, 2023
09d6139
[mixins][indent] allways trim left spacing in arguments
apalala Nov 29, 2023
18462a5
[ngcodegen][model] bootstrap model generation
apalala Nov 29, 2023
e38a3dc
[ngcodegen][objectmodel] add model generator
apalala Nov 29, 2023
0cc8f0d
[ngcodegen][lint] clear linter warnings
apalala Nov 29, 2023
1deaee3
[test] solve lint warnings and update unit tests
apalala Nov 29, 2023
e6aebf1
[mixins][indent] honor print() kwargs
apalala Nov 30, 2023
b8f4bdd
[ngcodegen][python] debut with what unit tests say
apalala Nov 30, 2023
6064a71
[tool] arg documentation
apalala Dec 8, 2023
4b6d2db
[lint] resolve warnings
apalala Dec 8, 2023
cf7fdfa
[walkers] fix long standing bug
apalala Dec 8, 2023
bae9710
[lint] fix warnings
apalala Dec 8, 2023
0bcdf05
some refactoring
apalala Dec 9, 2023
1d1d059
[docs] bug fix
apalala Dec 9, 2023
58e73aa
[docs] deprecate declarative translation
apalala Dec 9, 2023
48df36e
[walkers] refactor and cleanup
apalala Dec 9, 2023
2f9d487
[docs] update for walkers and refactor
apalala Dec 9, 2023
1d84b25
[docs] refactor translation
apalala Dec 9, 2023
40e9dd2
[ngcodegen] fix bugs
apalala Dec 9, 2023
9cc66af
[tool] replace code generation with ng
apalala Dec 9, 2023
1db6655
[ngcodegen][model] debug
apalala Dec 9, 2023
908549f
[ngcodegen] fix gut with rendering whitespace
apalala Dec 9, 2023
ad1bc9c
[ngcodegen][model] use topological sort for order of model classes
apalala Dec 10, 2023
ef23c64
[util][misc] document topological_sort
apalala Dec 10, 2023
8cefbae
remove debugging statements
apalala Dec 10, 2023
8e3004c
remove debug code
apalala Dec 10, 2023
b356d2a
[lint] resolve warnings
apalala Dec 10, 2023
db369b0
[ngcodegen][model] do not generate model classes for builtins
apalala Dec 10, 2023
3d4b066
remove reference to Py27
apalala Dec 10, 2023
224612c
[bootstrap] make the generated parser be the bootstrap parser
apalala Dec 10, 2023
c48c19f
[ngcodegen][model] refactor and optimize
apalala Dec 10, 2023
59a1ad5
[dist] bump up version number
apalala Dec 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[ngcodegen] up to Buffer definition
apalala committed Nov 28, 2023
commit caf280aa8d781f54e650a20139d677b16db7dcc0
6 changes: 5 additions & 1 deletion tatsu/ngcodegen/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
from ..objectmodel import ParseModel

from .python import PythonCodeGenerator


def codegen(model: ParseModel) -> str:
return '** NG CODEGEN NOT IMPLEMENTED YET **'
generator = PythonCodeGenerator()
generator.walk(model)
return generator.printed_text()
Binary file modified tatsu/ngcodegen/__pycache__/__init__.cpython-312.pyc
Binary file not shown.
95 changes: 95 additions & 0 deletions tatsu/ngcodegen/python.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
from __future__ import annotations

from .. import grammars
from ..mixins.indent import IndentPrintMixin
from ..util import trim
from ..walkers import NodeWalker


HEADER= """\
#!/usr/bin/env python

# WARNING:
#
# CAVEAT UTILITOR
#
# This file was automatically generated by TatSu.
#
# https://pypi.python.org/pypi/tatsu/
#
# Any changes you make to it will be overwritten the next time
# the file is generated.

# ruff: noqa: C405, I001, F401, SIM117

import sys
from pathlib import Path

from tatsu.buffering import Buffer
from tatsu.parsing import Parser
from tatsu.parsing import tatsumasu
from tatsu.parsing import leftrec, nomemo, isname
from tatsu.infos import ParserConfig
from tatsu.util import re, generic_main
"""


class PythonCodeGenerator(IndentPrintMixin, NodeWalker):

def print(self, *args, **kwargs):
args = [trim(arg) for arg in args]
super().print(*args, **kwargs)

def walk_Grammar(self, grammar: grammars.Grammar):
self.print(HEADER)
self.print()
self.print()

keywords = grammar.keywords
keywords = ['HERE']
keywords = [str(k) for k in keywords if k is not None]
if not keywords:
self.print('KEYWORDS: set[str] = set()')
else:
keywords = '\n'.join(f' {k!r},' for k in keywords)
keywords = '{\n%s\n}' % keywords
self.print(f'KEYWORDS: set[str] = {keywords}')

self.print()
self.print()

self._gen_buffering(grammar)

self.print(
'''
** AT GRAMMAR
'''
)

def _gen_buffering(self, grammar: grammars.Grammar):
self.print(f'class {grammar.name}Buffer(Buffer):')
start = grammar.config.start or grammar.rules[0].name

with self.indent():
self.print('def __init__(self, text, /, config: ParserConfig | None = None, **settings):')
with self.indent():
self.print(
f'''
config = ParserConfig.new(
config,
owner=self,
whitespace={grammar.config.whitespace},
nameguard={grammar.config.nameguard},
ignorecase={grammar.config.ignorecase},
namechars={grammar.config.namechars or None},
parseinfo={grammar.config.parseinfo},
comments_re={grammar.config.comments_re!r},
eol_comments_re={grammar.config.eol_comments_re!r},
keywords=KEYWORDS,
start={start!r},
)
config = config.replace(**settings)
super().__init__(text, config=config)
'''
)