Replace Dune grammar and add more filenames

[huwaireb]

Description

Migrates to the official OCaml vscode extension's TextMate grammar for Dune. And adds dune and dune-workspace to the files to be recognized.

Checklist:

• I am adding a new extension to a language.

  • The new extension is used in hundreds of repositories on GitHub.com
    • Search results for each extension:
      • https://github.com/search?type=code&q=NOT+is%3Afork+path%3Adune+lang
      • https://github.com/search?type=code&q=NOT+is%3Afork+path%3Adune-workspace+lang
  • I have included a real-world usage sample for all extensions added in this PR:
    • Sample source(s):
      • https://github.com/reasonml/reason/blob/20eb82b7dccbdc8ad3d41c2d0b6dba006d3583c8/src/reason-parser/dune
    • Sample license(s): MIT
  • I have included a change to the heuristics to distinguish my language from others using the same extension.

• I am changing the source of a syntax highlighting grammar

  • Old: https://github.com/maelvls/vscode-dune
  • New: https://github.com/ocamllabs/vscode-ocaml-platform

[huwaireb]

There's one non-blocking issue with this update,

vscode-ocaml-platform has three textmate grammars for dune:

• source.dune
• source.dune-project
• source.dune-workspace

source.dune-project and source.dune-workspace include source.dune#general. However, they don't share source.dune#stanzas, as each has their own set of supported stanzas, e.g source.dune-project#stanzas. Given linguist offers no way to use multiple grammars per language, the only way is to merge all possible stanzas into a single tmLanguage.json grammar and put it inside another repo, or put it in vscode-ocaml-platform and make a new source that is source.dune-all for example.

cc @smorimoto

[lildude]

Please update the template to fill in the appropriate section for adding the other filenames, You need to fill out all that apply and this is not a simple grammar switch with these additions.

Please note, if any of these new files do not meet our usage requirements, this PR will be blocked until they do meet our requirements or they are removed from this PR. Accordingly, you might want to consider splitting this PR into two.

[lildude]

• source.dune

This will be the only grammar used and it will be applied to all files identified as Dune as per the tm_scope: source.dune in the language.yml. Multiple grammars can no apply to the same language. The same grammar can however apply to multiple languages.

[huwaireb]

Please update the template to fill in the appropriate section for adding the other filenames, You need to fill out all that apply and this is not a simple grammar switch with these additions.

I've updated the PR to reflect.

Please note, if any of these new files do not meet our usage requirements, this PR will be blocked until they do meet our requirements or they are removed from this PR. Accordingly, you might want to consider splitting this PR into two.

They meet the requirements, except for dune-workspace.dev which only has 75 results. I can remove that, but most use of dune-workspace will revolve around it being scoped like that.

[lildude]

They meet the requirements, except for dune-workspace.dev which only has 75 results. I can remove that, but most use of dune-workspace will revolve around it being scoped like that.

You'll need to remove this filename if you want this PR merged.

[huwaireb]

You'll need to remove this filename if you want this PR merged.

I've removed references to dune-workspace.dev

[lildude]

Looks like you might not have used the script/add-grammar command to replace the grammar.

[huwaireb]

Looks like you might not have used the script/add-grammar command to replace the grammar.

Mm, i'm pretty sure i had, but it was weird. It didn't generate some files, i had to replace them manually.

Looks like i missed updating gitmodules, one second.

[huwaireb]

Should be okay now? Just missed deleting the actual submodule.

[lildude]

Please use the script. It correctly orders things in various files that are automatically maintained by it. It also validates the grammar files.

This should do the trick:

script/add-grammar --replace vendor/grammars/vscode-ocaml-platform https://github.com/ocamllabs/vscode-ocaml-platform.git

[huwaireb]

Please use the script. It correctly orders things in various files that are automatically maintained by it. It also validates the grammar files.

This should do the trick:

script/add-grammar --replace vendor/grammars/vscode-ocaml-platform https://github.com/ocamllabs/vscode-ocaml-platform.git

That did not do the trick

[huwaireb]

[huwaireb]

Ah there were errors in upstream's dune grammar,

will sort it out

Not sure how I didn't get this error before, but sure.

[huwaireb]

@lildude, quick question relating to the previous non-blocking issue I mentioned.

If I have a new tmLanguage.json syntax file that includes other patterns as follows

{ 
"name": "dune-all",
"scopeName": "source.dune-all",
"fileTypes": ["dune", "dune-project", "dune-workspace"],
"patterns": [{ "include": "source.dune#general" }, { "include": "source.dune#stanzas" }, { "include": "source.dune-project#stanzas" }, { "include": "source.dune-workspace#stanzas" }],
}

would that work with linguist? These "sources" come from the same submodule.

It's just to reduce maintainership burden, otherwise we can have a script that merges all three into a single file.

[lildude]

would that work with linguist? These "sources" come from the same submodule.

Yes, I believe the syntax highlighter will successfully pull those in as we have other grammars that use includes without any reports of problem. @Alhadis is more of an expert about writing grammars so can confirm from his experience.

[Alhadis]

would that work with linguist? These "sources" come from the same submodule.

@huwaireb Yes. In fact, the largest grammar repository I maintain, language-etc, makes extensive use of include: "etc#…" directives to drastically cut down on repeated/mostly-identical pattern rules. Which is particularly pertinent for a repository that mostly contains grammars to highlight assorted config files and data formats. 😉