Ability to configure skipping of types and specs in doc generation #1882
Comments
|
Hi @onno-vos-dev! Thanks for your work on maintaining those. Given those are generated, what do you think about having a flag on aws-elixir or aws-codegen itself that skips the types/specs? And then you can define the docs alias to do something like this: So now you regenerate the code right before generating the docs/publish the package. Then you do The benefit of doing the above is that you get a bit more of control. For example, you can skip the docs for some modules but leave it on others. WDYT? |
|
@josevalim Thank you for your reply and kind words! 🙇 While I'm not one to shy away from a challenge, I do think it's gonna be pretty non-trivial... 😄 At the time of publishing these packages, the code which generates the code (aws-codegen) is no longer available to that particular package be it In I only briefly looked through the |
|
I'd be OK and I think Eric would be too to make a size exception in Hex for this particular package. So my question is how having or not having typespecs and other things in docs changes the developer experience of package consumers. If it doesn't matter, then I'd like to pursue these ExDoc changes to have smaller tarballs indeed. |
|
Maybe a simpler idea that we could apply to both Elixir and Erlang is to do a pre pass on the .beam files and remove spec/type attributes? That should be only few LOC for each. I can share a snippet for Elixir tomorrow. |
|
here's a POC for Elixir. :) #!/usr/bin/env mix run
Path.wildcard("#{Mix.Project.app_path()}/ebin/*.beam")
|> Enum.each(fn path ->
{:ok, _module, chunks} = :beam_lib.all_chunks(String.to_charlist(path))
{_, dbgi} = List.keyfind(chunks, ~c"Dbgi", 0)
{:debug_info_v1, :elixir_erl, {:elixir_v1, code, typespecs}} = :erlang.binary_to_term(dbgi)
# keep just callbacks, remove types and specs
typespecs = Enum.filter(typespecs, &(elem(&1, 2) == :callback))
dbgi = :erlang.term_to_binary({:debug_info_v1, :elixir_erl, {:elixir_v1, code, typespecs}})
chunks = List.keyreplace(chunks, ~c"Dbgi", 0, {~c"Dbgi", dbgi})
{:ok, binary} = :beam_lib.build_module(chunks)
File.write!(path, binary)
end) |
|
Sweet! You can use a similar script for Erlang too. :) it can be even in Elixir. Let me know if that’s acceptable @onno-vos-dev! |
|
Sorry for the delay on this but I only just got around to deal with this. Running above Elixir script followed by and with latest ex_doc: |
|
Ah, we need to prune the doc entries for those types as well. Maybe just pruning specs is enough? You can try this version: |
|
😢 Unfortunately it's not small enough. There are three types inside the |
|
I managed to publish the docs now using below script to remove all types and specs. It's better than leaving the docs unpublished at least. #!/usr/bin/env mix run
Path.wildcard("#{Mix.Project.app_path()}/ebin/*.beam")
|> Enum.each(fn path ->
{:ok, _module, chunks} = :beam_lib.all_chunks(String.to_charlist(path))
# Remove docs from types which contain the examples
{_, docs} = List.keyfind(chunks, ~c"Docs", 0)
{:docs_v1, anno, language, fmt, a, b, c} = :erlang.binary_to_term(docs)
new_c = Enum.filter(c,
fn doc ->
elem(elem(doc, 0), 0) != :type
end)
new_docs = :erlang.term_to_binary({:docs_v1, anno, language, fmt, a, b, new_c})
new_chunks1 = List.keyreplace(chunks, ~c"Docs", 0, {~c"Docs", new_docs})
{_, dbgi} = List.keyfind(new_chunks1, ~c"Dbgi", 0)
{:debug_info_v1, :elixir_erl, {:elixir_v1, code, typespecs}} = :erlang.binary_to_term(dbgi)
# remove specs
new_typespecs = Enum.filter(typespecs, &(elem(&1, 2) != :spec))
new_dbgi = :erlang.term_to_binary({:debug_info_v1, :elixir_erl, {:elixir_v1, code, new_typespecs}})
new_chunks = List.keyreplace(new_chunks1, ~c"Dbgi", 0, {~c"Dbgi", new_dbgi})
{:ok, binary} = :beam_lib.build_module(new_chunks)
File.write!(path, binary)
end) |
As a maintainer of aws-elixir and aws-erlang we've had some issues with doc generation due to the size of these two packages.
While initially resolved thanks to @wojtekmach (🙏) this again led to issues with the generation of types and specs for these two packages (aws-beam/aws-codegen#109) causing the docs size to grow up to 152MB for
aws-erlang.Since I do not believe they're THAT important in these two packages to have available on hexdocs.pm I'd like the ability in ex_doc to disable this.
Would you be open to such a PR where a configuration would allow one to skip the generation of docs for types/specs?
The text was updated successfully, but these errors were encountered: