Request: Generate Python-specific documentation

[parker-research]

The PyPI package's documentation link points to https://sv-lang.com.

It would be nice if there was specific documentation built for the Python project to make using pyslang easier.

Even something as simple as a "pyslang Getting Started" section on that docs website that mirror's the pyslang README would be very helpful in reinforcing that sv-lang.com is the best there is currently for docs.

[MikePopoloski]

I agree; more documentation is always better. Happy to accept PRs if you want to contribute.

[parker-research]

Would you mind pointing me to where those docs live?

A summary in a CONTRIBUTING.md file that explains the dev process could be helpful! The Developer Guide on sv-lang.com doesn't have info about contributing to the docs either (which is getting rather meta :) )

Thanks!

[MikePopoloski]

All of the API / website docs are here: https://github.com/MikePopoloski/slang/tree/master/docs

There is a CONTRIBUTING file in the upstream repo: https://github.com/MikePopoloski/slang/blob/master/CONTRIBUTING.md

[parker-research]

I'm taking a look at how to build Python documentation from what's currently available here.

It doesn't seem like there's a very good way to build it with Doxygen, unfortunately. Would require hacks like find-and-replace in the cpp files, and/or exporting as XML and re-ingesting it from Sphynx-to-Doxygen or vice versa.

It seems like the best and most-suggested way to manage this is by generating the Python docs in Sphynx. Would you be willing to host and link the Sphynx-generated Python API docs? If so, do you agree that the generation scripts for them should probably go in this repository (and not in slang)?

[parker-research]

I setup a proof-of-concept Sphinx document generation in this branch - https://github.com/parker-research/pyslang/tree/binding-docs

Reading through the docs, without any docstrings, I'm not too sure if it's actually that helpful. But at least we know what it looks like now.

[MikePopoloski]

The documentation platform I use for the C++ docs & website is m.css, which has this Python generator as well. I had planned to look at that more if I ever found the time. The author is in a similar position of having a C++ library exposed to Python and wanting docs for both, so I assume he's solved some of the obvious problems.

[parker-research]

I'm figuring this out the m.css way, and it's going pretty well. It also supports generating function stubs (#96, #382), so I'm taking a look at how that can be setup as well.

A few issues with/a few opportunities to contribute to the m.css Python docs generator (especially in error handling and typing edge cases). Also, catching a few bugs in the current bindings (code which is not runnable, but is only caught as an error at runtime).

Which repo would you like this documentation generation in? I think it could go in either. Might make more sense in the pyslang repo, other than the fact that the docs website is published from a GitHub Action in the main slang repo; I'm not too sure if there's an easy way to push to the website from different actions in both repos.

The more I think about it, the more I sorta think the two repos should be merged, but that's a totally different topic.

[MikePopoloski]

Yeah, this probably needs to be in the slang repo for the reason you mentioned -- wanting to upload the whole website in one go from the doc build.

I tried to put the Python packaging stuff into the slang repo when I first started building the bindings, but the Python packaging infrastructure really did not like being a subdirectory and I simply couldn't make it work. Maybe things have improved since then but I kind of doubt it.

[parker-research]

Re documentation build - sounds good, will move my PR over to that repo when it's fully ready.

Re Python packaging location -

1. Would it be that big of deal to keep the pyproject.toml file in the root of the repo? I get that it's not ideal, but it doesn't seem like a blocker to me?
2. Not a complete parallel, but Polars, a Rust+Python bindings project, has the Python packaging in a subdirectory. So at the very least, there exists a way to make it work.

Maybe best for a different channel, but I'd be open to giving it a try. Curious if you remember specifically in what ways "the Python packaging infrastructure really did not like being a subdirectory"? Assuming it's not a total dealbreaker to have just the pyproject.toml in the repo root (like how Pytorch does it), then I'm certain it's possible as pyproject.toml lets you pick any subdirectory.

[MikePopoloski]

Yes, putting pyproject.toml in the root of the slang repo is fine. IIRC, scikit-build refused to work unless the target CMakeLists was in a subdirectory, though I have in the meantime moved to the newer scikit-build-core system. Something about the build was unhappy that it had paths leading up above it, since in the final package there's nothing above the root of the thing you're deploying.

Anyway, if you manage to make it work in the main repo without too much junk added I'd be happy to just kill this repo.