-
Notifications
You must be signed in to change notification settings - Fork 525
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Engineering Design Interface (EDI) Contrib Package #2937
Open
codykarcher
wants to merge
87
commits into
Pyomo:main
Choose a base branch
from
codykarcher:edi_dev
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+7,542
−47
Open
Changes from 56 commits
Commits
Show all changes
87 commits
Select commit
Hold shift + click to select a range
b354844
initial commit for EDI
3377110
adding coloring to the readme
604b0a1
correcting the readme example
eed941f
fixing readme again
f5a3232
fixing readme again again
d2696b6
fixing readme again again again
eba2990
updating readme, I really should read...
f625ef2
updating some legal stuff
b8dd90e
fixing the readme
4603e40
adding Michaels help:
39ea960
working on unit tests
b818da7
Merge remote-tracking branch 'upstream/main' into edi_dev
fe2821f
minor fixes
550fb70
still writing tests
f3d306f
working on documentation
ab02aba
still documenting
4e3f1d1
formulation unit testing finished
d62b011
variable docs done i think
39a5211
fixing the unit tests
09064b4
updating with an init file
df6336b
trying a testing fix
b369e68
working on runtime constraint docs
e594986
Merge remote-tracking branch 'upstream/main' into edi_dev
3a7918b
black box constraint docs good enough
a1049a8
docs done I think
816a022
more testing
871014c
finalizing for PR
6c4ef32
adding black
a6cde71
typos in docs
f76f497
addressing concerns in PR
7542063
adding black, part 2
b75d75f
undoing black
e5fbc0e
fixing imports
10d6845
fixing headers and staging for PR
6df90fd
applying black 3
cc683cb
fixing the aircraft GP example
2acc63e
trying to fix the import error
fd0c45b
adding a conditional wrapper to the pynumero import
7a58723
forgot an import statement, fixing
e769553
trying another import fix
4ebe52a
forgot to add black
48abb03
still updating imports
ca94021
Merge remote-tracking branch 'upstream/main' into edi_dev
085e729
Merge remote-tracking branch 'upstream/main' into edi_dev
36b99f9
changed the init file, not sure what I did
4381d3b
Merge remote-tracking branch 'upstream/main' into edi_dev
98c3ef6
Merge branch 'Pyomo:main' into edi_dev
codykarcher 7635017
adding black to the init file
2d4a0ed
Merge branch 'Pyomo:main' into edi_dev
codykarcher 2c48406
Merge branch 'Pyomo:main' into edi_dev
codykarcher 29dd11b
fixing a unit test
871c8ed
Merge branch 'Pyomo:main' into edi_dev
codykarcher 13c58af
Merge branch 'main' into edi_dev
blnicho ba8c1f3
Merge branch 'Pyomo:main' into edi_dev
codykarcher ae6c1be
Merge branch 'Pyomo:main' into edi_dev
codykarcher f186d89
Merge branch 'Pyomo:main' into edi_dev
codykarcher 7acdac4
Merge branch 'Pyomo:main' into edi_dev
codykarcher f18548d
encountered sierra bug, caching work
1237116
updating the black box tests to pass
782696f
figuring out what Im doing
13bc460
Merge branch 'Pyomo:main' into edi_dev
codykarcher e07dce3
needed to pip upgrade black
1850267
adding a todo list
54d50eb
Merge branch 'Pyomo:main' into edi_dev
codykarcher 7def60b
finally updating EDI
28df353
fixing documentation and typos
e255302
had an outdated version of black
65f23dd
removing broken links from the readme
d3b8dcd
Merge branch 'main' into edi_dev
codykarcher ef135d1
actualy removing links this time
1fb0c8d
should be ready for reveiw?
3a2ea94
Merge branch 'Pyomo:main' into edi_dev
codykarcher ac487b8
fixing dependency issue
808483f
adding black
2342baa
updating a python 3.8 compatibilty issue
42d2357
Merge branch 'main' into edi_dev
mrmundt b101ab4
Merge branch 'Pyomo:main' into edi_dev
codykarcher 0ef183f
Merge branch 'Pyomo:main' into edi_dev
codykarcher fd4a171
Merge branch 'main' into edi_dev
blnicho 0691ed4
Updating location of EDI docs following the documentation reorg
blnicho 895fe84
NFC: removing file that was missed during the doc reorg
blnicho bd8420d
Updating year in copyright statements and making sure copyright state…
blnicho e73803c
Add missing newlines
blnicho 204021a
Merge branch 'main' into edi_dev
codykarcher 4d41f08
Merge branch 'main' into edi_dev
codykarcher 6207fe1
Merge branch 'main' into edi_dev
mrmundt ff1787f
Merge branch 'main' into edi_dev
mrmundt File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
37 changes: 37 additions & 0 deletions
37
doc/OnlineDocs/contributed_packages/edi/additionaltips.rst
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
Additional Tips | ||
--------------- | ||
|
||
* Developers may need to install the following additional packages: | ||
|
||
:: | ||
|
||
pip install pytest | ||
pip install pytest-cov | ||
pip install sphinx | ||
pip install sphinx_rtd_theme | ||
pip install sphinx_copybutton | ||
|
||
|
||
* If you wish to build the documentation locally, use: | ||
|
||
:: | ||
|
||
cd <path_to_pyomo>/pyomo/doc/OnlineDocs | ||
make html | ||
|
||
then open the file ``<path_to_pyomo/doc/OnlineDocs/_build/html/index.html>`` | ||
|
||
|
||
* Unit tests and coverage can be run locally using: | ||
|
||
:: | ||
|
||
cd <path_to_pyomo>/pyomo/pyomo/contrib/edi | ||
pytest --cov-report term-missing --cov=pyomo.contrib.edi -v ./tests/ | ||
|
||
or generating html output: | ||
|
||
:: | ||
|
||
cd <path_to_pyomo>/pyomo/pyomo/contrib/edi | ||
pytest --cov-report html --cov=pyomo.contrib.edi -v ./tests/ | ||
blnicho marked this conversation as resolved.
Show resolved
Hide resolved
|
115 changes: 115 additions & 0 deletions
115
doc/OnlineDocs/contributed_packages/edi/advancedruntimeconstraints.rst
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,115 @@ | ||
Advanced Runtime Constraints | ||
============================ | ||
|
||
|
||
Automated unpacking for multi-use black-boxes | ||
blnicho marked this conversation as resolved.
Show resolved
Hide resolved
|
||
--------------------------------------------- | ||
|
||
Coding a black-box model often represents a significant effort, and it is therefore desirable to have the black-box work in many situations. A common case is to have a black-box model with a scalar input variable perform vectorized operations, ie, take in a vector and return a vector. In the more general case, this can be thought of as passing in multiple run-cases as input to the black-box model. | ||
blnicho marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
The ``parseInputs()`` method enables this batch-like capability in a variety of forms: | ||
|
||
.. py:function:: BlackBoxFunctionModel.parseInputs(self, *args, **kwargs) | ||
|
||
Parses the inputs to a black-box model into a list of run-cases | ||
|
||
:param args: The self attribute in all python methods | ||
:type self: black-box model | ||
:param args: index passed arguments | ||
blnicho marked this conversation as resolved.
Show resolved
Hide resolved
|
||
:type args: list or tuple | ||
:param kwargs: keyword passed arguments | ||
:type kwargs: dict | ||
|
||
:return: runcases | ||
:rtype: list | ||
:return: returnMode | ||
:rtype: int | ||
:return: extras | ||
:rtype: list | ||
blnicho marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
The function definition is not particularly helpful, so let's dive in a bit. For the typical user, we recommend that the top of all ``BlackBox()`` methods appear as follows: | ||
|
||
.. literalinclude:: ../../../../pyomo/contrib/edi/tests/test_docSnippets.py | ||
:language: python | ||
:dedent: 12 | ||
:start-after: # BEGIN: AdvancedRTC_Snippet_02 | ||
:end-before: # END: AdvancedRTC_Snippet_02 | ||
|
||
|
||
Essentially, ``parseInputs()`` is a pre-processor that directly takes the inputs of the black-box. The ``parseInputs()`` method will check all of the inputs, ensure that size and units are correct, split into run cases as appropriate, and return a run-cases list that is ready to operate on. | ||
|
||
The ``runCases`` return (which can be named as any valid python name) is a list of dicts, where the keys to the dict are the names of the inputs declared in the ``__init__()`` method. Ex: ``runCases[0]['x']`` will give the ``'x'`` input (in units specified in the ``__init__()`` method) in the first run-case. | ||
blnicho marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
The ``returnMode`` return (which can be named as any valid python name) is an integer that indicates how the return should be processed. If ``returnMode`` is passed as a ``kwarg``, then the passed in value will be cast to this output. If it is not passed in, then ``returnMode`` will be either ``-1*self.availableDerivative``, indicating that there is only a single run case, or ``self.availableDerivative`` which indicated multiple run cases. A negative value for ``returnMode`` indicates that there will be one less layer of indexing in the output. | ||
|
||
The ``extras`` return (which can be named as any valid python name) is a dict that will include all of the additional passed in values not expected by the black-box model. The extra ``kwargs`` will appear as normal, and extra ``args`` get put in a list in ``extras['remainingArgs']``. If you pass in a ``kwarg`` with key name ``'remainingArgs'``, it will be overwritten. The extras dict is the place to find options that may affect the code (ex: tolerances, run modes, etc) that are not expected inputs to the black-box model. | ||
|
||
Note that when using run case input, the output will always take the following unpacking structure: | ||
|
||
:: | ||
|
||
output[<run_case_index>][0] = <list_of_outputs_for_specified_runcase> | ||
output[<run_case_index>][0][<index_of_output>] = <output_for_specified_runcase> | ||
|
||
output[<run_case_index>][1] = <list_of_jacobians_for_specified_runcase> | ||
output[<run_case_index>][1][<index_of_output>][<index_of_input>] = <d(output_of_specified_index)/d(input_of_specified_index)> | ||
|
||
|
||
There is **not** a shortcut for single scalar output black boxes as is the case when not using run cases, the final index to output 0 must be included. | ||
|
||
|
||
|
||
An Example | ||
++++++++++ | ||
|
||
There are many ways this functionality can be used, we provide an example here to get new users started | ||
|
||
.. literalinclude:: ../../../../pyomo/contrib/edi/tests/test_docSnippets.py | ||
:language: python | ||
:dedent: 8 | ||
:start-after: # BEGIN: AdvancedRTC_Snippet_01 | ||
:end-before: # END: AdvancedRTC_Snippet_01 | ||
|
||
|
||
Check outputs | ||
------------- | ||
|
||
There is a ``checkOutputs()`` method that is not supported in the current version. Contact the developers if you desire this functionality, but the following the practices described in this documentation should render the need for this moot. | ||
blnicho marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
|
||
Cases of non-scalar inputs or outputs | ||
------------------------------------- | ||
|
||
Indexing can get somewhat complicated when inputs and outputs are not scalars. Users should be warned this feature is supported, but not well tested, so please contact the developers if you encounter any unusual behavior. | ||
|
||
The following unpacking remains the same: | ||
|
||
:: | ||
|
||
output[0] = <list_of_outputs> | ||
output[0][<index_of_output>] = <output> | ||
|
||
output[1] = <list_of_jacobians> | ||
output[1][<index_of_output>][<index_of_input>] = <d(output_of_specified_index)/d(input_of_specified_index)> | ||
|
||
However, for outputs, the result will be an array with dimensions equal to the size of the output. For Jacobians, it breaks down as the following: | ||
|
||
:: | ||
|
||
jacobian_list_entry[(output_dim_1_ix, output_dim_2_ix, ..., input_dim_1_ix, input_dim_2_ix, ...)] = <scalar_d(output_of_specified_index)/d(input_of_specified_index)> | ||
|
||
For example, with an output that is 2x2 and an input that is also 2x2 | ||
|
||
:: | ||
|
||
output[1][<index_of_output>][<index_of_input>][(0,0,1,1)] | ||
|
||
is the derivative of ``output[0,0]`` with respect to ``input[1,1]`` | ||
|
||
|
||
|
||
Tips | ||
---- | ||
|
||
* A model summary can be printed by calling ``print(model_instance.summary)`` |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Most of this seems generic and not specific to EDI. Should this get moved to the "contribution" section of Pyomo's documentation?
There are also some build options that can be specified when
pip
installing Pyomo that will grab the packages needed to run tests or build documentation. Maybe we should point the users to those rather than manually listing packages? I'm mainly concerned that this list could become out of sync with the requirements specified insetup.py
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
setup.py
does not includepytest-cov
https://github.com/Pyomo/pyomo/blob/main/setup.py#L232
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Current documentation only lists:
but it is not clear if a developer would also have to run the following