"should not compile" tests

[gildor478]

This feature request has been migrated from artifact #1375 on forge.ocamlcore.org. It was assigned to user102.

user157 posted on 2014-02-14 21:55:26:

It would be nice to have tests like "this .ml should not compile".
It's useful while developing modules with abstract/private types, to make sure that some internal information must be hidden ( = "trying to access internals, it doesn't compile -> test passed"), or other compile-time constraints are obeyed (like "trying to make some operation on data with phantom/gadt type while it should be impossible -> it should not compile at all").
Also it could be very useful for projects that generate OCaml code, to check that generated code fits some constraints, not only positive ("code must compile and give expected results"), but negative also.

I understand you may have no time to implement it, but please answer these questions:

1. does the whole idea fits OUnit ideology? (I doubt it, because OUnit works when something have compiled and already running.)
2. in case it does, what ideas/pointers can you give me to ease implementation/merging of this feature in OUnit?

user102 replied on 2014-02-14 23:23:38:

At this point you reach one of my project (as is in my personal TODO list):
ounit-build: build code, run ocamlbuild plugin, run syntax extension

I would say, ounit itself is not the perfect place but a related project (I was planning to call it ounit-build) would be perfect.

Idealy both of them should live in this project and will be closely related, but may follow different release cycle.

In fact, I have already done some piece of this in OASIS:
https://github.com/ocaml/oasis/blob/master/test/TestFullUtils.ml

In particular:
l130 check_file_style

l196 setup_test_directories (create temporary directory and setup bin/lib/ocaml_lib et al)

l411 assert_compile && l416 try_installed_library

All these are in the context of OASIS, which means that it needs some adaptation to refer to file generated in the build system of the project itself (OASIS copies projects and compiles them fully).

If you can/want to combine that with OASIS, you can just do a fake installation of the project in a temporary directory.