Add Typesense #44
Add Typesense #44
Conversation
ruslandoga
force-pushed
the
add-typesense
branch
from
ca70475
to
859dd13
Compare
|
Thank you @ruslandoga! I am currently on holidays but I will try to carve some time sooner than later to give you feedback. /cc @wojtekmach |
Erlang uses ExDoc now, so they have the exact same structure. However, we will need to either poll them or ask them to ping us once they publish a new version or ask them to push their docs to Hexdocs! I'd say we can postpone this to a follow up pull request. On the other hand, I believe Gleam does not have the data in the format we need (we created our |
|
@ruslandoga I'll try to review this sooner than later but I have a lot on my plate until October 15th, after that it's gonna be my number one priority to see this through. Sorry about that. |
ruslandoga
force-pushed
the
add-typesense
branch
from
ce7af3b
to
e18e317
Compare
|
👋 I think it's ready for review now :) Updates since the last review:
|
| search_data_js = | ||
| Enum.find_value(files, fn {path, content} -> | ||
| case Path.basename(path) do | ||
| "search_data-" <> _digest -> content |
There was a problem hiding this comment.
We cannot trust this data since it's user provided, can they do anything dangerous by providing something we don't expect? Maybe we should do some rudimentary validation?
There was a problem hiding this comment.
They can provide long strings like https://github.com/cloudpods-dev/docker-engine-api-elixir/blob/813cc557da483f623a8f484db04efc7e58db0376/lib/docker_engine_api/api/container.ex#L67, but Typesense seems to handle it fine. We can check for content size, maybe. I think if Typesense doesn't like the payload, it would simply reject it.
There was a problem hiding this comment.
I added a test that checks that invalid fields in search items (like type being a map instead of a string, or doc being a list) are rejected: 8d58e4f
Co-authored-by: Eric Meadows-Jönsson <[email protected]>
ruslandoga
force-pushed
the
add-typesense
branch
from
5d9f67d
to
bfc0539
Compare
| @@ -5,6 +5,9 @@ if config_env() == :prod do | |||
| port: System.fetch_env!("HEXDOCS_PORT"), | |||
| hexpm_url: System.fetch_env!("HEXDOCS_HEXPM_URL"), | |||
| hexpm_secret: System.fetch_env!("HEXDOCS_HEXPM_SECRET"), | |||
| typesense_url: System.fetch_env!("TYPESENSE_URL"), | |||
| typesense_api_key: System.fetch_env!("TYPESENSE_API_KEY"), | |||
| typesense_collection: System.fetch_env!("TYPESENSE_COLLECTION"), | |||
There was a problem hiding this comment.
I created hexdocs-test collection on Typesense Cloud, https://cloud.typesense.org/clusters/ent97o5sv4dzx2f0p/collections
I think we can use it during alpha/beta testing.
| @@ -5,6 +5,9 @@ if config_env() == :prod do | |||
| port: System.fetch_env!("HEXDOCS_PORT"), | |||
| hexpm_url: System.fetch_env!("HEXDOCS_HEXPM_URL"), | |||
| hexpm_secret: System.fetch_env!("HEXDOCS_HEXPM_SECRET"), | |||
| typesense_url: System.fetch_env!("TYPESENSE_URL"), | |||
| typesense_api_key: System.fetch_env!("TYPESENSE_API_KEY"), | |||
There was a problem hiding this comment.
This would be the "Admin API Key" for https://cloud.typesense.org/clusters/ent97o5sv4dzx2f0p cluster, it can be downloaded from the dashboard.
| @@ -5,6 +5,9 @@ if config_env() == :prod do | |||
| port: System.fetch_env!("HEXDOCS_PORT"), | |||
| hexpm_url: System.fetch_env!("HEXDOCS_HEXPM_URL"), | |||
| hexpm_secret: System.fetch_env!("HEXDOCS_HEXPM_SECRET"), | |||
| typesense_url: System.fetch_env!("TYPESENSE_URL"), | |||
There was a problem hiding this comment.
And this would be https://ent97o5sv4dzx2f0p.a1.typesense.net
|
👋 Just wanted to check if there’s anything else we'd need to address before it's merged? |
This PR that integrates Typesense into Hexdocs.
TODOs:
Logging excessively for now
Indexing in the same step for now
proglangThe format is the same: Add Typesense #44 (comment)
Commit reverted, approach needs discussion with Gleam team
CI results: ruslandoga#1