fix doc build on windows

[drammock]

A couple of contributors have reported doc build failures on Windows:

Traceback (most recent call last):
  File "C:\Users\user\anaconda3\envs\mnedev\Lib\shutil.py", line 847, in move
    os.rename(src, real_dst)
OSError: [WinError 87] The parameter is incorrect: 'C:\Users\user\Documents\GitHub\mne-python\doc\generated\mne-gui-addons:sphx_glr_auto_examples_evoked_ers_source_power.py.examples.new' -> 'C:\Users\user\Documents\GitHub\mne-python\doc\generated\mne-gui-addons:sphx_glr_auto_examples_evoked_ers_source_power.py.examples'

The direct problem (IIUC) seems to be that the file name mne-gui-addons:sphx_glr_auto_examples_evoked_ers_source_power.py.examples.new contains a colon, which is not allowed by the Windows filesystem.

This PR should hopefully make it possible again to build our docs on windows. Since I don't have a windows machine I'm hoping either @contsili or @CarinaFo can test this branch (or anyone else with access to a MNE dev environment on a windows machine).

The fix (🤞🏻) is to explicitly mark the intersphinx cross-reference as an external crossref so that Sphinx (or Sphinx-Gallery? not sure) won't generate that file with the colon in its filename in the first place. TBH I haven't dug into why this works, I just tried it on a hunch and it seems to have the desired effect: it does prevent that file-that-contains-a-colon from getting generated, no alternatively-named file exists in its place, and yet in the locally-built docs the crossref is correctly resolved to https://mne.tools/mne-gui-addons/auto_examples/evoked_ers_source_power.html#sphx-glr-auto-examples-evoked-ers-source-power-py

cc @lucyleeow, do you know whether it's Sphinx or Sphinx-gallery that generates that file, and if it's Sphinx Gallery, do you think it's worth adding a note to the sphinx-gallery docs about avoiding using crossref syntax that contains a colon?

[larsoner]

do you know whether it's Sphinx or Sphinx-gallery that generates that file, and if it's Sphinx Gallery, do you think it's worth adding a note to the sphinx-gallery docs about avoiding using crossref syntax that contains a colon?

Pretty sure it's sphinx-gallery. SG probably shouldn't create any examples.new files for any module that isn't set in doc_module, I think it would fix this problem.

[contsili]

@drammock thank you for looking into it.

I tested your branch and I now get a new error:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\user\anaconda3\envs\mnedev\Lib\site-packages\towncrier\__main__.py", line 6, in <module>
    cli()
  File "C:\Users\user\anaconda3\envs\mnedev\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\anaconda3\envs\mnedev\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\Users\user\anaconda3\envs\mnedev\Lib\site-packages\click\core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\anaconda3\envs\mnedev\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\anaconda3\envs\mnedev\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\anaconda3\envs\mnedev\Lib\site-packages\towncrier\build.py", line 123, in _main
    return __main(
           ^^^^^^^
  File "C:\Users\user\anaconda3\envs\mnedev\Lib\site-packages\towncrier\build.py", line 265, in __main
    click.echo(content)
  File "C:\Users\user\anaconda3\envs\mnedev\Lib\site-packages\click\utils.py", line 318, in echo
    file.write(out)  # type: ignore
    ^^^^^^^^^^^^^^^
  File "C:\Users\user\anaconda3\envs\mnedev\Lib\encodings\cp1253.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\xf6' in position 3720: character maps to <undefined> [docutils]

[contsili]

This new error comes after all the galleries are generated. More specifically, it comes after reading sources... [100%] sg_execution_times

[larsoner]

Looks to me like a likely bug in sphinxcontrib-towncrier and/or towncrier itself and/or our usage of it. Somewhere we need to be able to specify that our stubs are UTF-8 encoded so it doesn't use the system encoding (which is different by default on Windows!).

[drammock]

do you know whether it's Sphinx or Sphinx-gallery that generates that file, and if it's Sphinx Gallery, do you think it's worth adding a note to the sphinx-gallery docs about avoiding using crossref syntax that contains a colon?

Pretty sure it's sphinx-gallery. SG probably shouldn't create any examples.new files for any module that isn't set in doc_module, I think it would fix this problem.

I think that it's not that simple: it's a crossref to an SG-created gallery example not a cross-ref to an API element. So for generic :ref: or :doc:-type cross-references it wouldn't make sense for SG to check against doc_module right? Or maybe I'm misunderstanding...

[larsoner]

it's a crossref to an SG-created gallery example not a cross-ref to an API element.

Not sure if this matters too much. I think any :ref: that has a colon inside the backticks is likely to cause this problem on Windows. Like if we had a

:ref:`mne:mne.Epochs`

in an example I suspect it would probably cause the same issue. And what you have so far works around the issue by moving the colon out of the backticks, which is what I think SG uses to decide what to call the <whatever>.examples.new file it creates.

So it seems like the more general fix is to stop SG from creating .example.new files that would have colons in them the first place, at least on Windows (and if we do it there, we might as well do it more generally for consistency). The idea I had above was to avoid it for anything that isn't in doc_module, which determines what SG decides to try to make backreference files for automatically IIRC. However, perhaps a better idea that makes it more general is just to sanitize the colon (e.g., to an underscore) when creating the <whatever>.examples.new file, probably here:

https://github.com/sphinx-gallery/sphinx-gallery/blob/53f197a6894747d0cb060827ed1205387aad9c44/sphinx_gallery/backreferences.py#L377

[drammock]

ah, yeah, you're right. the mne:mne.Epochs example makes it super clear what the fix should be (sanitize the colon).

[contsili]

Looks to me like a likely bug in sphinxcontrib-towncrier and/or towncrier itself and/or our usage of it. Somewhere we need to be able to specify that our stubs are UTF-8 encoded so it doesn't use the system encoding (which is different by default on Windows!).

@larsoner you are right. To fix encoding issues when building the website, I simply run: set PYTHONIOENCODING=utf-8 before executing make html-noplot in the command line. The website builds fine afterwards.

To ensure all stubs are UTF-8 encoded, we could modify file.write(out) in click/utils.py (line 318) to force UTF-8 encoding when writing files. Alternatively, we could set it globally earlier in the code. Thoughts?

[CarinaFo]

Sorry for the late reply, moving continents is more involved than I thought.
I tested building the docs on my Windows machine, and it worked except for a few warnings. Is there anything specific I should/can test? I did not change the Python encoding before I ran make html.