Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revise TOML format #2

Merged
merged 1 commit into from
Jan 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .typos.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
[default]
extend-ignore-identifiers-re = [
"[A-Z]T",
"[0-9abcdef]{8}",
"[A-Z][a-z]+[A-Z]{2,5}[0-9]{4}",
]

[files]
extend-exclude = ["*.bib", "*.toml", "test_inventory.jl"]
17 changes: 9 additions & 8 deletions docs/src/formats.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,23 +35,24 @@ Note that `DocInventories` internally uses the `text/x-intersphinx` MIME type fo

## TOML Format

The TOML format is another text output format that is optimized for human readability. It starts with a header section of the form
The TOML format is a text output format that is optimized for human readability. It starts with a header section of the form

```
[Inventory]
format = "DocInventories v0"
```toml
# DocInventory version 0
project = "<project>"
version = "<version>"
```

The `format` line is mandatory and identifies the file as containing inventory data in the format described here.
The comment in the first line is mandatory and identifies the file as containing inventory data in the format described here.

!!! warning
As indicated by the `v0` in the `format` line, the format described here is currently experimental and may change without notice
As indicated by the "version 0" in the header comment line, the format described here is currently experimental and may change without notice

The `project` line must specify the name of the project described by the inventory. It is mandatory. The `version` line may specify the version of the project. It is optional, but recommended.

After that, each [`InventoryItem`](@ref) is represented by a multi-line block of the form

```
```toml
[[<domain>.<role>]]
name = "<name>"
uri = "<uri>"
Expand All @@ -61,7 +62,7 @@ priority = <priority>

The four lines for `name`, `uri`, `dispname`, and `priority` may occur in any order. Also, for items with the default priority (`-1` for the `std` domain, `1` otherwise), the `priority` line may be omitted. If `dispname` is equal to `name` (usually indicated by `dispname="-"`), the `dispname` line may also be omitted.

The item-blocks may be grouped/separated by blank lines. In `.toml` file generated by `DocInventories.save("inventory.toml", inventory)` items will be grouped into blocks with the same `[[<domain>.<role>]]` with a blank line between each block.
The item-blocks may be grouped/separated by blank lines. In `.toml` files generated by `DocInventories.save("inventory.toml", inventory)` items will be grouped into blocks with the same `[[<domain>.<role>]]` with a blank line between each block.

Any TOML parser should read a `.toml` file with the above structure into a nested dictionary, so that `item_dict = toml_data[domain][role][i]` corresponds to the `i`'th inventory item with the given `domain` and `role`. That `item_dict` will then map `"name"`, `"uri"`, and potentially `"dispname"` and `"priority"` to their respective values.

Expand Down
3 changes: 1 addition & 2 deletions docs/src/inventories/Documenter.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
[Inventory]
format = "DocInventories v0"
# DocInventory version 0
project = "Documenter.jl"
version = "1.2.1"

Expand Down
4 changes: 2 additions & 2 deletions docs/src/inventories/Julia.toml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[Inventory]
format = "DocInventories v0"
# DocInventory version 0

project = "The Julia Language"
version = "1.10.0"

Expand Down
4 changes: 1 addition & 3 deletions docs/src/inventories/JuliaDocs.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
[Inventory]
format = "DocInventories v0"
# DocInventory version 0
project = "JuliaDocs"
version = ""

[[std.doc]]
name = "DocumenterInterLinks"
Expand Down
59 changes: 31 additions & 28 deletions src/toml_format.jl
Original file line number Diff line number Diff line change
Expand Up @@ -25,47 +25,50 @@ function Base.show(io::IO, ::MIME"application/toml", inventory::Inventory)
push!(domains[item.domain][item.role], item_data)
end

toml_dict = Dict(
"Inventory" => Dict(
"format" => "DocInventories v0",
"project" => inventory.project,
"version" => inventory.version,
),
domains...
)
toml_dict = Dict("project" => inventory.project, domains...)
if !isempty(inventory.version)
toml_dict["version"] = inventory.version
end

TOML.println(io, "# DocInventory version 0")
TOML.print(io, toml_dict; sorted=true)

end


function read_inventory(io::IO, ::MIME"application/toml")
format_header = readline(io)
if !startswith(format_header, "#")
@warn "Invalid format_header: $format_header"
# TODO: verify format_header more strictly once the format has settled.
end
data = TOML.parse(io)
try
inventory_format = data["Inventory"]["format"]
if inventory_format != "DocInventories v0"
msg = "Invalid inventory format: $(repr(inventory_format))"
throw(InventoryFormatError(msg))
end
project = data["Inventory"]["project"]
version = data["Inventory"]["version"]
project = string(pop!(data, "project")) # mandatory
version = string(pop!(data, "version", "")) # optional
items = InventoryItem[]
for (domain, domain_data) in data
(domain == "Inventory") && continue # that's the header, not a domain
for (role, role_data) in domain_data
for item_data in role_data
push!(
items,
InventoryItem(
item_data["name"],
domain,
role,
get(item_data, "priority", (domain == "std") ? -1 : 1),
item_data["uri"],
get(item_data, "dispname", "-")
if domain_data isa Dict
for (role, role_data) in domain_data
for item_data in role_data
push!(
items,
InventoryItem(
item_data["name"],
domain,
role,
get(item_data, "priority", (domain == "std") ? -1 : 1),
item_data["uri"],
get(item_data, "dispname", "-")
)
)
)
end
end
elseif domain == "format"
# For backward-compatibility, remove this `elseif` in v1.0
@warn "Unexpected key: $domain"
else
throw(InventoryFormatError("Unexpected key: $domain"))
end
end
return project, version, items
Expand Down
143 changes: 143 additions & 0 deletions test/test_inventory.jl
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,118 @@ end
@test c.value isa InventoryFormatError
@test contains(c.output, "Unexpected line")

filename = joinpath(tempdir, "missing_project.toml")
#!format: off
write(filename, """
# DocInventory version 0
[[std.doc]]
name = "DocumenterInterLinks"
uri = "DocumenterInterLinks.jl#readme"
""")
#!format: on
c = IOCapture.capture(rethrow=Union{}) do
Inventory(filename; root_url="")
end
@test c.value isa InventoryFormatError
@test contains(c.output, "key \"project\" not found")

filename = joinpath(tempdir, "old_format.toml")
#!format: off
write(filename, """
[Inventory]
format = "DocInventories v0"
project = "Test"
version = "0.1.0"
[[std.doc]]
name = "DocumenterInterLinks"
uri = "DocumenterInterLinks.jl#readme"
""")
#!format: on
c = IOCapture.capture(rethrow=Union{}) do
Inventory(filename; root_url="")
end
@test_broken c.value isa InventoryFormatError
@test contains(c.output, "Unexpected key: format")
@test contains(c.output, "Invalid format_header: [Inventory]")

filename = joinpath(tempdir, "old_format2.toml")
#!format: off
write(filename, """
# This is an old inventory file, and because of this comment, the
# `[Inventory]` won't be stripped as a format_header
[Inventory]
format = "DocInventories v0"
project = "Test"
version = "0.1.0"
[[std.doc]]
name = "DocumenterInterLinks"
uri = "DocumenterInterLinks.jl#readme"
""")
#!format: on
c = IOCapture.capture(rethrow=Union{}) do
Inventory(filename; root_url="")
end
@test c.value isa InventoryFormatError
@test contains(c.output, "key \"project\" not found")

filename = joinpath(tempdir, "no_header_line.toml")
#!format: off
write(filename, """
project = "Test"
version = "0.1.0"
[[std.doc]]
name = "DocumenterInterLinks"
uri = "DocumenterInterLinks.jl#readme"
""")
#!format: on
c = IOCapture.capture(rethrow=Union{}) do
Inventory(filename; root_url="")
end
@test c.value isa InventoryFormatError
@test contains(c.output, "Invalid format_header: project = \"Test\"")
@test contains(c.output, "key \"project\" not found")

filename = joinpath(tempdir, "typo1.toml")
#!format: off
write(filename, """
# DocInventory version 0
project = "Test"
verison = "0.1.0"
[[std.doc]]
name = "DocumenterInterLinks"
uri = "DocumenterInterLinks.jl#readme"
""")
#!format: on
c = IOCapture.capture(rethrow=Union{}) do
Inventory(filename; root_url="")
end
@test c.value isa InventoryFormatError
@test contains(c.output, "Unexpected key: verison")

filename = joinpath(tempdir, "typo2.toml")
#!format: off
write(filename, """
# DocInventory version 0
project = "Test"
version = "0.1.0"
[[std_doc]]
name = "DocumenterInterLinks"
uri = "DocumenterInterLinks.jl#readme"
""")
#!format: on
c = IOCapture.capture(rethrow=Union{}) do
Inventory(filename; root_url="")
end
@test c.value isa InventoryFormatError
@test contains(c.output, "Unexpected key: std_doc")

end

end
Expand Down Expand Up @@ -484,6 +596,37 @@ end
@test contains(c.output, "Only v2 objects.inv files currently supported")
@test c.value isa InventoryFormatError

filename = joinpath(tempdir, "wrong_version_type.toml")
#!format: off
write(filename, """
# DocInventory version 0
project = "Test"
version = 1.0
[[std.doc]]
name = "DocumenterInterLinks"
uri = "DocumenterInterLinks.jl#readme"
""")
#!format: on
readinv = Inventory(filename; root_url="")
# this still works because the float "1.0" is converted to string
@test readinv.version == "1.0"

filename = joinpath(tempdir, "future_header.toml")
#!format: off
write(filename, """
# Documenter Inventory version 1
project = "Test"
version = "1.0"
[[std.doc]]
name = "DocumenterInterLinks"
uri = "DocumenterInterLinks.jl#readme"
""")
#!format: on
readinv = Inventory(filename; root_url="")
@test readinv.project == "Test"

end

end
Loading