Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Gazebo materials #2269

Merged
merged 21 commits into from
Jan 12, 2024
Merged

Support for Gazebo materials #2269

merged 21 commits into from
Jan 12, 2024

Conversation

quarkytale
Copy link
Contributor

@quarkytale quarkytale commented Dec 14, 2023

🎉 New feature

Closes #10

Summary

Ported the OgreMaterialParser from the Ogre community, also used in gzweb to add minimal parsing capabilities of Gazebo materials. This would make migration from Gazebo Classic to Gazebo Sim a little bit easier.
Limitations:

  • Only parse solid colors
  • Custom material scripts are not supported

Task list:

  • Install gazebo.material at gz-sim/media/
  • Add warning with documentation page linked
  • Write integration test for the parser
  • Check for a solution to handle dependent materials

Test it

Run Gazebo from the source directory
gz sim src/sdformat/test/integration/model/double_pendulum.sdf -v 3

Checklist

  • Signed all commits for DCO
  • Added tests
  • Added example and/or tutorial
  • Updated documentation (as needed)
  • Updated migration guide (as needed)
  • Consider updating Python bindings (if the library has them)
  • codecheck passed (See contributing)
  • All tests passed (See test coverage)
  • While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

Signed-off-by: Dharini Dutia <[email protected]>
Signed-off-by: Dharini Dutia <[email protected]>
@github-actions github-actions bot added the 🎵 harmonic Gazebo Harmonic label Dec 14, 2023
Copy link
Contributor

@azeey azeey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works well, but there are a few changes we need to make before merging:

src/CMakeLists.txt Outdated Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.cc Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.hh Outdated Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.hh Outdated Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.hh Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.cc Outdated Show resolved Hide resolved
src/SdfEntityCreator.cc Outdated Show resolved Hide resolved
src/SdfEntityCreator.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.cc Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.hh Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.hh Outdated Show resolved Hide resolved
quarkytale and others added 3 commits December 20, 2023 00:33
Signed-off-by: Dharini Dutia <[email protected]>
Co-authored-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Dharini Dutia <[email protected]>
Signed-off-by: Dharini Dutia <[email protected]>
src/SdfEntityCreator.cc Outdated Show resolved Hide resolved
Signed-off-by: Dharini Dutia <[email protected]>
Copy link
Contributor

@mjcarroll mjcarroll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a style nit, but there is a good chunk of code here that uses ::iterator types rather than range-based for loops. I know that this is copied code so it's probably not a huge deal, but it looks inconsistent with the rest of the codebase.

src/rendering/MaterialParser/ConfigLoader.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.cc Outdated Show resolved Hide resolved
src/CMakeLists.txt Outdated Show resolved Hide resolved
Signed-off-by: Dharini Dutia <[email protected]>
src/rendering/MaterialParser/ConfigLoader.cc Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.cc Outdated Show resolved Hide resolved
src/SdfEntityCreator.cc Outdated Show resolved Hide resolved
src/SdfEntityCreator.cc Outdated Show resolved Hide resolved
src/SdfEntityCreator.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.hh Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.hh Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.hh Outdated Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.hh Outdated Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.hh Outdated Show resolved Hide resolved
Signed-off-by: Dharini Dutia <[email protected]>
Copy link

codecov bot commented Dec 27, 2023

Codecov Report

Attention: 45 lines in your changes are missing coverage. Please review.

Comparison is base (f8b1a46) 65.61% compared to head (5455d1e) 65.80%.
Report is 6 commits behind head on gz-sim8.

Files Patch % Lines
src/rendering/MaterialParser/ConfigLoader.cc 78.67% 45 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           gz-sim8    #2269      +/-   ##
===========================================
+ Coverage    65.61%   65.80%   +0.19%     
===========================================
  Files          324      327       +3     
  Lines        30925    31212     +287     
===========================================
+ Hits         20292    20540     +248     
- Misses       10633    10672      +39     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Dharini Dutia <[email protected]>
@quarkytale quarkytale force-pushed the quarkytale/material branch from 0bb3523 to f2e2f5f Compare January 2, 2024 08:45
@quarkytale quarkytale marked this pull request as ready for review January 2, 2024 09:41
@quarkytale quarkytale requested a review from mabelzhang as a code owner January 2, 2024 09:41
@quarkytale quarkytale force-pushed the quarkytale/material branch from ad1de96 to 7b08ebf Compare January 2, 2024 17:49
@iche033
Copy link
Contributor

iche033 commented Jan 3, 2024

Tested different Gazebo/* material names and seems to work:

pendulum_material

I also tried Gazebo/Invalid - It rendered the link black which is probably expected but maybe add a warning that the material name is not recognized?

if (!_visual->Material()->ScriptUri().empty())
{
gzwarn << "Gazebo does not support Ogre material scripts. See " <<
"https://gazebosim.org/api/sim/8/migrationsdf.html#:~:text=Materials " <<
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does the #:~:text=Materials part work? It takes me to the migration page but not the Materials section.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its using text fragments, and mostly browser dependent. For me, it highlights and scrolls to the Material section but scrolls back up the page. Do you think it would be useful?

Copy link
Contributor

@iche033 iche033 Jan 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah ok I thought it would jump to that section. Yes I see it highlighted. We can leave the text fragment here.

@quarkytale quarkytale force-pushed the quarkytale/material branch from 26565b2 to 7621715 Compare January 4, 2024 02:22
src/SdfEntityCreator.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/ConfigLoader.cc Show resolved Hide resolved
tutorials/migration_sdf.md Outdated Show resolved Hide resolved
Signed-off-by: Dharini Dutia <[email protected]>
src/SdfEntityCreator.cc Outdated Show resolved Hide resolved
src/rendering/MaterialParser/MaterialParser.cc Outdated Show resolved Hide resolved
@quarkytale quarkytale force-pushed the quarkytale/material branch from 60f05db to eacd18f Compare January 6, 2024 02:17
Signed-off-by: Dharini Dutia <[email protected]>
@quarkytale quarkytale force-pushed the quarkytale/material branch from eacd18f to 404ffef Compare January 8, 2024 20:00
ConfigNode * ambientNode = passNode->findChild("ambient");
if (ambientNode) {
std::vector<float> ambientValues;
ambientNode->getValuesInFloat(ambientValues);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't guarantee that getValuesInFloat will return a vector of 3 (at least). For example, if the user modifies the gazebo.material file and forgets to put the third value for a color, we'd end up with a vector of size 2, but then we'd try to access ambientValues[2] which would be undefined behavior. So I think we should add a check here and other places getValuesInFloat is used in this file. Alternatively, we can have a fuction in ConfigNode that returns a color instead of a vector.

Copy link
Contributor Author

@quarkytale quarkytale Jan 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of gz/math dependency in ConfigLoader and placing extra checks in MaterialParser, I'm thinking of updating getValuesInFloat to check if the vector has 3 or more values, which is common for all colors in the material file. If not then throwing an error saying "Bad material file", what do you think about that?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do that, my preference would be to create another getColorValues function or something similar so getValuesInFloat can remain a generic function that gets a list of floats for non-colors. Either way, by "throwing an error", you mean printing an error message, I definitely agree. But if you mean throw an exception, I would say we should avoid throwing exceptions since that's our general practice in the Gazebo codebase.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added another function with the size check. Yeah that's right just printing an error message, not throwing an exception.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I see the size check has been added, but it would still allow out of bounds access on the vector since the code continues to assign the colors after getColorValues whether that fails or not. I would suggest something like the following:

            std::vector<float> ambientValues;
            ambientNode->getColorValues(ambientValues);
            values->ambient.emplace();
            for (std::size_t i=0; i < ambientValues.size(); ++i)
            {
                   values->ambient[i] = ambientValues[i];
            }

However, it looks like there's a bug in Color::operator[] since it doesn't return a reference. So another way would be to modify the getColorValues function to resize the vector.

  inline void getColorValues(std::vector<float> & colorValues)
  {
    getValuesInFloat(colorValues);
    if (colorValues.size() < 3) 
   {
      gzerr << "Bad material file." << std::endl;
      colorValues.resize(3);
    }
  }

ConfigNode * ambientNode = passNode->findChild("ambient");
if (ambientNode) {
std::vector<float> ambientValues;
ambientNode->getValuesInFloat(ambientValues);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I see the size check has been added, but it would still allow out of bounds access on the vector since the code continues to assign the colors after getColorValues whether that fails or not. I would suggest something like the following:

            std::vector<float> ambientValues;
            ambientNode->getColorValues(ambientValues);
            values->ambient.emplace();
            for (std::size_t i=0; i < ambientValues.size(); ++i)
            {
                   values->ambient[i] = ambientValues[i];
            }

However, it looks like there's a bug in Color::operator[] since it doesn't return a reference. So another way would be to modify the getColorValues function to resize the vector.

  inline void getColorValues(std::vector<float> & colorValues)
  {
    getValuesInFloat(colorValues);
    if (colorValues.size() < 3) 
   {
      gzerr << "Bad material file." << std::endl;
      colorValues.resize(3);
    }
  }

src/rendering/MaterialParser/ConfigLoader.hh Outdated Show resolved Hide resolved
Signed-off-by: Dharini Dutia <[email protected]>
Signed-off-by: Dharini Dutia <[email protected]>
Copy link
Contributor

@azeey azeey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a couple more minor comments. Thanks for iterating!

There are also a few style issues, but for the sake of expediency, we'll address them later.

tutorials/migration_sdf.md Outdated Show resolved Hide resolved
tutorials/migration_sdf.md Outdated Show resolved Hide resolved
Signed-off-by: Dharini Dutia <[email protected]>
@azeey azeey dismissed ahcorde’s stale review January 12, 2024 00:00

Feedback has been addressed

@quarkytale quarkytale merged commit c6aaaf2 into gz-sim8 Jan 12, 2024
8 of 10 checks passed
@quarkytale quarkytale deleted the quarkytale/material branch January 12, 2024 01:31
@quarkytale quarkytale restored the quarkytale/material branch February 1, 2024 19:48
quarkytale added a commit that referenced this pull request Feb 14, 2024
* test script tag

Signed-off-by: Dharini Dutia <[email protected]>

* add dark grey before creating component

Signed-off-by: Dharini Dutia <[email protected]>

* material parser

Signed-off-by: Dharini Dutia <[email protected]>

* linters

Signed-off-by: Dharini Dutia <[email protected]>

* Update src/rendering/MaterialParser/MaterialParser.cc

Co-authored-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Dharini Dutia <[email protected]>
Signed-off-by: Dharini Dutia <[email protected]>

* fix configLoader, material struct and feedback

Signed-off-by: Dharini Dutia <[email protected]>

* default color and todos

Signed-off-by: Dharini Dutia <[email protected]>

* install/load one file, range based loop, hardcode dependent solid colors

Signed-off-by: Dharini Dutia <[email protected]>

* fix install_dir property

Signed-off-by: Dharini Dutia <[email protected]>

* credits, initializing cleanup

Signed-off-by: Dharini Dutia <[email protected]>

* eof

Signed-off-by: Dharini Dutia <[email protected]>

* reformat

Signed-off-by: Dharini Dutia <[email protected]>

* add integration test

Signed-off-by: Dharini Dutia <[email protected]>

* migration note

Signed-off-by: Dharini Dutia <[email protected]>

* intends, default case, invalid color

Signed-off-by: Dharini Dutia <[email protected]>

* optional materialValues, typo

Signed-off-by: Dharini Dutia <[email protected]>

* feedback

Signed-off-by: Dharini Dutia <[email protected]>

* size check

Signed-off-by: Dharini Dutia <[email protected]>

* get color values

Signed-off-by: Dharini Dutia <[email protected]>

* migration doc update

Signed-off-by: Dharini Dutia <[email protected]>

---------

Signed-off-by: Dharini Dutia <[email protected]>
Signed-off-by: Dharini Dutia <[email protected]>
Co-authored-by: Alejandro Hernández Cordero <[email protected]>
azeey pushed a commit that referenced this pull request Mar 26, 2024
* test script tag

Signed-off-by: Dharini Dutia <[email protected]>

* add dark grey before creating component

Signed-off-by: Dharini Dutia <[email protected]>

* material parser

Signed-off-by: Dharini Dutia <[email protected]>

* linters

Signed-off-by: Dharini Dutia <[email protected]>

* Update src/rendering/MaterialParser/MaterialParser.cc

Co-authored-by: Alejandro Hernández Cordero <[email protected]>
Signed-off-by: Dharini Dutia <[email protected]>
Signed-off-by: Dharini Dutia <[email protected]>

* fix configLoader, material struct and feedback

Signed-off-by: Dharini Dutia <[email protected]>

* default color and todos

Signed-off-by: Dharini Dutia <[email protected]>

* install/load one file, range based loop, hardcode dependent solid colors

Signed-off-by: Dharini Dutia <[email protected]>

* fix install_dir property

Signed-off-by: Dharini Dutia <[email protected]>

* credits, initializing cleanup

Signed-off-by: Dharini Dutia <[email protected]>

* eof

Signed-off-by: Dharini Dutia <[email protected]>

* reformat

Signed-off-by: Dharini Dutia <[email protected]>

* add integration test

Signed-off-by: Dharini Dutia <[email protected]>

* migration note

Signed-off-by: Dharini Dutia <[email protected]>

* intends, default case, invalid color

Signed-off-by: Dharini Dutia <[email protected]>

* optional materialValues, typo

Signed-off-by: Dharini Dutia <[email protected]>

* feedback

Signed-off-by: Dharini Dutia <[email protected]>

* size check

Signed-off-by: Dharini Dutia <[email protected]>

* get color values

Signed-off-by: Dharini Dutia <[email protected]>

* migration doc update

Signed-off-by: Dharini Dutia <[email protected]>

---------

Signed-off-by: Dharini Dutia <[email protected]>
Signed-off-by: Dharini Dutia <[email protected]>
Co-authored-by: Alejandro Hernández Cordero <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🎵 harmonic Gazebo Harmonic
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Convert models with material scripts
5 participants