Path Excludes not working when workspaces are introduced in the code repository based on NPM, PNPM, Yarn2 #8940
Comments
porsche-rishisaxena
added
bug
@porsche-rishisaxena have you configured ORT to do so? (If yes, this could be added to the reproduce step after #1.) |
|
@fviernau we did this step in our .ort.yml file which you can see in the sample code repository as following; |
|
@fviernau I have added the second step in the problem statement description what @porsche-ozgursedefoglu mentioned where we use skip_excluded: true. Please refer to the sample repo we created for reproducing: https://github.com/porsche-ozgursedefoglu/npm-workspaces/blob/main/.ort.yml |
sschuberth
added
analyzer
fviernau
added a commit
that referenced
this issue
Oct 7, 2024
Previously, the submodules of any `Yarn` or `Pnpm` workspaces were represented as packages. This was inconsistent, because ORT normally represents any definition files found in the analyzed sources as a `Project`, not as a `Package`. Besides being inconsistent, the `Package` representation renders the `ort.yml` features unusable. For example, workspace submodules could not be excluded via path excludes. Furthermore, the previous implementation represented the workspace root project (in case of Pnpm) as both, as a `Package` and as a `Project`. Finally, the previous Package representation of a submodule did not have any reference from any project scope. As a consequnce, any (license) policy rules which operates only on non-excluded dependencies would have disregarded the submodules and their transitive dependencies, potentially leading to an incorrect underreporting of rule violations. Extend the `NpmModuleInfo` class by the flag `isProject` and make use of it in the `NpmDependencyHandler` for creating the packages and determining the linkage type. This change guarantees that `parsePackage()` is no more called for `Project`s, but only for `Package`s which is why the project-specific logic is dropped from `parsePackage()`. For projects the dedicated `parseProjects()` is now consistently used instead. As in the new representation there are no more unreferenced packages, the dependency handler does take care of creating all packages. So, the logic which calls `graph.addPackage()` for each module returned by `parseInstalledModules()` became unnecessary and is dropped. Note: This commit fixes multiple things at once, because it seemed too complicated to fix each issue separately due to various chicken-egg like problems. Fixes #9196, fixes #8940. Signed-off-by: Frank Viernau <[email protected]>
fviernau
added a commit
that referenced
this issue
Oct 8, 2024
Previously, the submodules of any `Yarn` or `Pnpm` workspaces were represented as packages. This was inconsistent, because ORT normally represents any definition files found in the analyzed sources as a `Project`, not as a `Package`. Besides being inconsistent, the `Package` representation renders the `ort.yml` features unusable. For example, workspace submodules could not be excluded via path excludes. Furthermore, the previous implementation represented the workspace root project (in case of Pnpm) as both, as a `Package` and as a `Project`. Finally, the previous Package representation of a submodule did not have any reference from any project scope. As a consequnce, any (license) policy rules which operates only on non-excluded dependencies would have disregarded the submodules and their transitive dependencies, potentially leading to an incorrect underreporting of rule violations. Extend the `NpmModuleInfo` class by the flag `isProject` and make use of it in the `NpmDependencyHandler` for creating the packages and determining the linkage type. This change guarantees that `parsePackage()` is no more called for `Project`s, but only for `Package`s which is why the project-specific logic is dropped from `parsePackage()`. For projects the dedicated `parseProjects()` is now consistently used instead. As in the new representation there are no more unreferenced packages, the dependency handler does take care of creating all packages. So, the logic which calls `graph.addPackage()` for each module returned by `parseInstalledModules()` became unnecessary and is dropped. Note: This commit fixes multiple things at once, because it seemed too complicated to fix each issue separately due to various chicken-egg like problems. Fixes #9196, fixes #8940. Signed-off-by: Frank Viernau <[email protected]>
fviernau
added a commit
that referenced
this issue
Oct 8, 2024
Previously, the submodules of any `Yarn` or `Pnpm` workspaces were represented as packages. This was inconsistent, because ORT normally represents any definition files found in the analyzed sources as a `Project`, not as a `Package`. Besides being inconsistent, the `Package` representation renders the `ort.yml` features unusable. For example, workspace submodules could not be excluded via path excludes. Furthermore, the previous implementation represented the workspace root project (in case of Pnpm) as both, as a `Package` and as a `Project`. Finally, the previous Package representation of a submodule did not have any reference from any project scope. As a consequnce, any (license) policy rules which operates only on non-excluded dependencies would have disregarded the submodules and their transitive dependencies, potentially leading to an incorrect underreporting of rule violations. Extend the `NpmModuleInfo` class by the flag `isProject` and make use of it in the `NpmDependencyHandler` for creating the packages and determining the linkage type. This change guarantees that `parsePackage()` is no more called for `Project`s, but only for `Package`s which is why the project-specific logic is dropped from `parsePackage()`. For projects the dedicated `parseProjects()` is now consistently used instead. As in the new representation there are no more unreferenced packages, the dependency handler does take care of creating all packages. So, the logic which calls `graph.addPackage()` for each module returned by `parseInstalledModules()` became unnecessary and is dropped. Note: This commit fixes multiple things at once, because it seemed too complicated to fix each issue separately due to various chicken-egg like problems. Fixes #9196, fixes #8940. Signed-off-by: Frank Viernau <[email protected]>
sschuberth
pushed a commit
that referenced
this issue
Oct 8, 2024
Previously, the submodules of any `Yarn` or `Pnpm` workspaces were represented as packages. This was inconsistent, because ORT normally represents any definition files found in the analyzed sources as a `Project`, not as a `Package`. Besides being inconsistent, the `Package` representation renders the `ort.yml` features unusable. For example, workspace submodules could not be excluded via path excludes. Furthermore, the previous implementation represented the workspace root project (in case of Pnpm) as both, as a `Package` and as a `Project`. Finally, the previous Package representation of a submodule did not have any reference from any project scope. As a consequnce, any (license) policy rules which operates only on non-excluded dependencies would have disregarded the submodules and their transitive dependencies, potentially leading to an incorrect underreporting of rule violations. Extend the `NpmModuleInfo` class by the flag `isProject` and make use of it in the `NpmDependencyHandler` for creating the packages and determining the linkage type. This change guarantees that `parsePackage()` is no more called for `Project`s, but only for `Package`s which is why the project-specific logic is dropped from `parsePackage()`. For projects the dedicated `parseProjects()` is now consistently used instead. As in the new representation there are no more unreferenced packages, the dependency handler does take care of creating all packages. So, the logic which calls `graph.addPackage()` for each module returned by `parseInstalledModules()` became unnecessary and is dropped. Note: This commit fixes multiple things at once, because it seemed too complicated to fix each issue separately due to various chicken-egg like problems. Fixes #9196, fixes #8940. Signed-off-by: Frank Viernau <[email protected]>
fviernau
added a commit
that referenced
this issue
Oct 8, 2024
Previously, the submodules of any `Yarn` or `Pnpm` workspaces were represented as packages. This was inconsistent, because ORT normally represents any definition files found in the analyzed sources as a `Project`, not as a `Package`. Besides being inconsistent, the `Package` representation renders the `ort.yml` features unusable. For example, workspace submodules could not be excluded via path excludes. Furthermore, the previous implementation represented the workspace root project (in case of Pnpm) as both, as a `Package` and as a `Project`. Finally, the previous Package representation of a submodule did not have any reference from any project scope. As a consequnce, any (license) policy rules which operates only on non-excluded dependencies would have disregarded the submodules and their transitive dependencies, potentially leading to an incorrect underreporting of rule violations. Extend the `NpmModuleInfo` class by the flag `isProject` and make use of it in the `NpmDependencyHandler` for creating the packages and determining the linkage type. This change guarantees that `parsePackage()` is no more called for `Project`s, but only for `Package`s which is why the project-specific logic is dropped from `parsePackage()`. For projects the dedicated `parseProjects()` is now consistently used instead. As in the new representation there are no more unreferenced packages, the dependency handler does take care of creating all packages. So, the logic which calls `graph.addPackage()` for each module returned by `parseInstalledModules()` became unnecessary and is dropped. Note: This commit fixes multiple things at once, because it seemed too complicated to fix each issue separately due to various chicken-egg like problems. Fixes #9196, fixes #8940. Signed-off-by: Frank Viernau <[email protected]>
|
@porsche-rishisaxena it would be great if you could confirm on your end that #9247 fixes your issue. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
Path Excludes not working when workspaces are introduced in the code repository based on NPM, PNPM, Yarn2.
To Reproduce
Steps to reproduce the behavior:
docker run -v $PWD:/project
ort:latest
-P ort.analyzer.allowDynamicVersions=true --debug analyze
-i /project/
-o /project/ort/
Expected behavior
After Path Exclude applied in .ort.yml, the paths are still visible in the reports which should not be the case and the below statement is violated:
"ORT can be configured to skip them completely during the analysis phase. The affected elements are then not processed any further and do not occur in generated reports."
The text was updated successfully, but these errors were encountered: