Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Expansion of Encoders Implementation for Full Flights. #679

Merged
merged 3 commits into from
Dec 21, 2024

Conversation

phmbressan
Copy link
Collaborator

@phmbressan phmbressan commented Aug 30, 2024

Pull request type

  • Code changes (bugfix, features)

Checklist

  • Tests for the changes have been added (partially done)
  • Docs have been reviewed and added / updated (future PR)
  • Lint (black rocketpy/ tests/) has passed locally
  • All tests (pytest tests -m slow --runslow) have passed locally
  • CHANGELOG.md has been updated (if relevant)

Current behavior

Currently the _encoding module is just a stub and cannot still encode full simulations into a JSON like format.

New behavior

This PR expand this module bringing the following:

  • Custom Encoding for more types;
  • Customized behavior for some required classes with methods to_dict;
  • Support for Function encoding.

This PR also introduces decoding functionalities. The way this works is:

  • If the is a to_dict or from_dict method, they are used to perform the (de)encoding;
  • Otherwise, a default encoding of all vars() and constructor based decoding is done.

Breaking change

  • Yes
  • No

Additional information

There is much to discuss here on the implementation and maintainability side of things. Main discussion points:

  • Should we add a to_dict and (in the future) a from_dict method to some (or all) rocketpy classes?
    • My Opinion: I think adding the methods to everything might be the only way out. But there is some work and maintainability considerations.
  • The current status of the encoding can be fully deserialized?
    • My Opinion: All the information is there, but the typing should be improved to know which objects to decode (deserialize) into. For that reason, I kept the _encoders private.
  • Are there any other modules to make our work easier?
    • From my research, I have tested the following:
      • simplejson: brings some more types and speed into encoding, but does not help much with custom types;
      • jsonpickle: really great for the typing handler (already writes by default each object type) and is able to serialize. The problem is that is does not handle functions/lambdas and I did not find a good way to implement custom handlers for our use case;
      • pickle / dill: the most plug and play solution, however the output is not human readable and not generally compatible between different rocketpy versions.

Of course, this is what I understood upon researching and testing, feel free to make any comments or suggest other modules.

Remarks and Future Considerations

Here are some of the steps that could be improved:

  • Verify why the test for hybrid motor flight requires higher tolerances;
    • Corrected, no higher tolerances needed anymore.
  • Test if all of the atmospheric models of Environments can be decoded without requiring re-read of files or API;
  • Evaluate whether we should encode outputs or only the minimum required information to decode the class from its init;
    • Option for including outputs or not was added.
  • (Optional) Expand implementation to AirBrakes + Controllers and new classes.

Some important remarks:

  • The simple getting started simulation is essentially identical comparing the store & load, the only difference is the random lag and noise from Parachute class. I could not found a simple solution to reset those and always get the same results.

@phmbressan phmbressan added the Enhancement New feature or request, including adjustments in current codes label Aug 30, 2024
@phmbressan phmbressan added this to the Release v1.X.0 milestone Aug 30, 2024
@phmbressan phmbressan self-assigned this Aug 30, 2024
@phmbressan phmbressan requested a review from a team as a code owner August 30, 2024 21:32
Copy link

codecov bot commented Aug 30, 2024

Codecov Report

Attention: Patch coverage is 70.58824% with 115 lines in your changes missing coverage. Please review.

Project coverage is 76.34%. Comparing base (83aa20e) to head (c389519).
Report is 10 commits behind head on develop.

Files with missing lines Patch % Lines
rocketpy/rocket/rocket.py 43.58% 22 Missing ⚠️
rocketpy/environment/environment.py 59.61% 21 Missing ⚠️
rocketpy/_encoders.py 78.18% 12 Missing ⚠️
rocketpy/motors/tank_geometry.py 64.51% 11 Missing ⚠️
...ocketpy/rocket/aero_surface/fins/free_form_fins.py 33.33% 6 Missing ⚠️
rocketpy/rocket/parachute.py 71.42% 6 Missing ⚠️
rocketpy/motors/tank.py 87.80% 5 Missing ⚠️
...cketpy/rocket/aero_surface/fins/elliptical_fins.py 37.50% 5 Missing ⚠️
rocketpy/rocket/components.py 37.50% 5 Missing ⚠️
rocketpy/simulation/flight.py 37.50% 5 Missing ⚠️
... and 9 more
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #679      +/-   ##
===========================================
- Coverage    76.42%   76.34%   -0.08%     
===========================================
  Files           95       96       +1     
  Lines        11090    11460     +370     
===========================================
+ Hits          8475     8749     +274     
- Misses        2615     2711      +96     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@Gui-FernandesBR Gui-FernandesBR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing to complain about, good work.

requirements.txt Show resolved Hide resolved
rocketpy/mathutils/function.py Show resolved Hide resolved
rocketpy/rocket/components.py Outdated Show resolved Hide resolved
@phmbressan
Copy link
Collaborator Author

phmbressan commented Nov 12, 2024

I have just noticed that I made a mistake when merging CHANGELOG conflicts for this PR.

I will fix it before merging. This does not block reviewing.

  • Fix CHANGELOG.md merging errors.

@Gui-FernandesBR Gui-FernandesBR self-requested a review November 13, 2024 03:32
Copy link
Member

@MateusStano MateusStano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall comments:

  • to_dict and from_dict methods are missing docstrings in all classes

rocketpy/simulation/flight.py Outdated Show resolved Hide resolved
rocketpy/motors/tank_geometry.py Show resolved Hide resolved
rocketpy/_encoders.py Outdated Show resolved Hide resolved
@Gui-FernandesBR
Copy link
Member

Overall comments:

  • to_dict and from_dict methods are missing docstrings in all classes

I strongly advise to not include dosctrings yet.
We don't really know if these methods will really be used.

Also, they exists mainly for private reasons: we need them in order have the JSONEncoder working.

Btw the methods are so simple that they don't even need docstrings. But that's a discussion for future sessions...

Copy link
Member

@Gui-FernandesBR Gui-FernandesBR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last test we could run is: Create a Monte Carlo simulation with 100 simulations and verify if the output .txt files are still readable enough.

@Gui-FernandesBR Gui-FernandesBR linked an issue Nov 23, 2024 that may be closed by this pull request
@phmbressan
Copy link
Collaborator Author

One last test we could run is: Create a Monte Carlo simulation with 100 simulations and verify if the output .txt files are still readable enough.

Good point. I have added some customization options to MonteCarlo so that one can choose whether to include all Function data or keep the way it is currently on develop that is only the Function string.

MNT: Add encoding feature to CHANGELOG.

BUG: add dill to the requirements file.
ENH: extend encoding and decoding to Liquid and Hybrid.

MNT: correct decoding of liquid and hybrid motors.

STY: solve pylint remarks.

MNT: adapt encoding to new post merge attributes.

MNT: restore typo to correct values on flight test.

ENH: add option for including outputs on JSON export.

TST: add tests for motor encoding.

DOC: Improve docstrings of encoders signature.

MNT: Make no output encoding the default.

MNT: Standardize include outputs parameter.

DOC: Correct phrasing and typos of encoders docstring.

MNT: Correct json export environment naming.
Copy link
Member

@Gui-FernandesBR Gui-FernandesBR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@phmbressan just a few more conversations need to be solved.

The PR looks good btw.

DEV: fix CHANGELOG

MNT: reposition barometric height as env json output.

MNT: Allow for encoding customization of MonteCarlo.
Copy link
Member

@Gui-FernandesBR Gui-FernandesBR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, amazing work @phmbressan .

@GabrielBarberini should be interested in this feature. Please notice this is not a part of the public API!!

@phmbressan phmbressan merged commit cbbdb65 into develop Dec 21, 2024
8 of 10 checks passed
@phmbressan phmbressan deleted the enh/sim-encoding branch December 21, 2024 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request, including adjustments in current codes
Projects
Status: Closed
Development

Successfully merging this pull request may close these issues.

ENH: make rocketpy classes json serializable
3 participants