Baseline tools

[roodboi]

a few repo deps needed for the actions in here

adding this to the repo to help manage changes
https://github.com/apps/changeset-bot

adding the following env vars (plus ensuring package name availability)
NPM_TOKEN -> can create an npm account and generate a token to use for publishing
GITHUB_TOKEN -> a gh personal access token with pr perms for this repo ( used for automating auto PR opening for publishing)

(happy to go deeper on changesets or deploy/publishing flow in here - its using this https://github.com/changesets/changesets)

[jxnl]

holy shit can we seperate some of the PR

I think the changeset and preitter stuff can be merged right away, but theres so much js code i can't review it all hah

[nipunsadvilkar]

@jxnl Have setup prettier in this PR #14

[jxnl]

any reason you're not using zod-to-json-schema?

[jxnl]

MD_JSON
JSON
JSON_SCHEMA are different modes

and the parameters are different.

if mode == Mode.JSON:
   new_kwargs["response_format"] = {"type": "json_object"}

elif mode == Mode.JSON_SCHEMA:
    new_kwargs["response_format"] = {
        "type": "json_object",
        "schema": response_model.model_json_schema(),
    }

elif mode == Mode.MD_JSON:
    new_kwargs["messages"].append(
        {
            "role": "assistant",
            "content": "```json",
        },
    )
    new_kwargs["stop"] = "```"
    ```

[roodboi]

@jxnl re: zod-to-json-schema

There seemed like an opportunity to move to 0 deps and the conversion is relatively straight forward. Additionally, having the control within the lib to decide how to pass through refinement rules/describes/etc into the JSON description per parameter seemed like it would be pretty useful to control the quality of results.

[jxnl]

I think my goal is to have something instructor can plug into whatever is the most popular validation framework.

Elixir has ecto
Python has Pydantic
js with zod.

[jxnl]

I would prefer minimizing the code in the instructor library and transferring it to the zod+zodjson schema. This will make the implementation lighter and easier to understand. Many other libraries have been criticized for having excessive indirection, making it difficult to determine what is actually happening. I believe the params/parse approach is logical and cleaner than the Python implementation.

I bet therea long type of types that we can take advantage of and don't need to reinvent the wheel

[roodboi]

Yeah that's fair - no need to maintain control unless you need it -

also probably relevant but theres a zod alternative that has some traction, Its very similar, but I would imagine there might be some folks that might want to use itt (https://github.com/sinclairzx81/typebox)

Which is only to say, that keeping the schema mapping layer flexible might be a good idea

[jxnl]

easier to market when i can pick the most popular lib. the mission is to show how much prompt engineering can be done with your favourite validation library and then have them integrated with that.

for ruby i can imagine activerecord or dry-validation for example. anywhere when you get runtime validated structs, theres a place to prompt engineer

[jxnl]

added a enviroment OPENAI with a OPENAI_API_KEY