Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(README): update the Gradle section #295

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 98 additions & 17 deletions README.md
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functional Findings:

  • optional: encapsulate the code in a separate gradle file (e.g. dash.gradle) and just apply the script where needed (rootProject, subprojects, both...) - it's cleaner than "polluting" the build.gradle with different contexts and different tasks. However it might get a bit finicky with the plugin you apply, as you can't use the plugin-block in other gradle scripts than settings.gradle and build.gradle
  • when executing dashDependencies not on the rootProject but on a subproject the results will contain an entry like this: project :app. These entries should be excluded from the dependencies and therefore not be part of the dependencies.txt
  • right now the dashDependencies task is only executed on the rootProject, however each subproject will have it's own dependency tree and configurations. Basically you need to create and execute your task for each subproject, similar like this:
subprojects {
    tasks.register('dashDependencies') {
        description = "Output all project dependencies as a flat list and save an intermediate file 'deps.txt'."
        group = 'License'

        doLast {
            def deps = []
            project.configurations.each { conf ->
                if (conf.canBeResolved && conf.getName() != 'archives' && conf.getName() != 'default') {
                    deps.addAll(conf.incoming.resolutionResult.allDependencies
                            .findAll({ it instanceof ResolvedDependencyResult })
                            .collect { ResolvedDependencyResult dep ->
                                "${dep.selected}"
                            })
                }
            }
            def sorted = deps.unique().sort()
            mkdir "$rootDir/build/oss/${project.name}"
            file("$rootDir/build/oss/${project.name}/dependencies.txt").write(sorted.join('\n'))
        }
    }
}

Execute ./gradlew dashDependenciesand check the results for each submodule here: $rootDir/build/oss/. All of them should be clarified using the dash license tool. It's better to scan to much than scanning to few :)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have something similar in place here (we don't scan with gradle, we just provide the dependencies.txt for the CI):

Besides the few "project"-entries the recognition of both our scripts is on the same level (if executed on all subprojects). However I prefer your way of collecting the dependencies, because it's less error prone than using the regular expressions to parse them.

Maybe you can pick the best of both our worlds so we get something that is top-notch :)

Once its more fine-tuned we will definitely use it in favor of our dash.sh in our project.

Original file line number Diff line number Diff line change
Expand Up @@ -388,27 +388,108 @@ In the case where the license information for content is not already known, this

### Example: Gradle

Find all of the potentially problematic third party libraries from a Gradle build.
Before you begin, it is recommended to add the following two lines to your `.gitignore`:

```
deps.txt
dasj.jar
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
dasj.jar
dash.jar

typo

```

To use Dash directly within Gradle, add the following code to your `build.gradle`.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rootProject build.gradle or app / lib build.gradle? I guess you mean the rootProject build.gradle here as you also call it with without a project here


NOTE: you'll need to input values for the `<an-up-to-date-version>` and the `<your-project-here>` variables from the snippet below.

```groovy
plugins {
// used to download the 'dash.jar' for license checks
// docs: https://github.com/michel-kraemer/gradle-download-task
id "de.undercouch.download" version "<an-up-to-date-version>"
}

repositories {
maven {
// Used to resolve Dash License Tool
// Dash has a maven plugin, BUT is not resolvable through mavenCentral()
url = uri("https://repo.eclipse.org/content/repositories/dash-licenses/")
}
}
Comment on lines +409 to +415
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really needed?
You don't resolve the Dash License Tool using a maven dependency but by downloading a file via url


// uses the 'download' plugin
// docs: https://plugins.gradle.org/plugin/de.undercouch.download
tasks.register('dashDownload', Download) {
description = 'Download the Dash License Tool standalone jar'
group = 'License'
src 'https://repo.eclipse.org/service/local/artifact/maven/redirect?r=dash-licenses&g=org.eclipse.dash&a=org.eclipse.dash.licenses&v=LATEST'
dest layout.projectDirectory.file('dash.jar')
// will not replace an existing file. If you know you need a new version
// then manually delete the file yourself, or run `dashClean`
overwrite false
}

// This task is primarily used by CIs
tasks.register('dashClean') {
description = "Clean all files used by the 'License' group"
group = 'License'
logger.lifecycle("Removing 'dash.jar'")
file('dash.jar').delete()
logger.lifecycle("Removing 'deps.txt'")
file('deps.txt').delete()
Comment on lines +433 to +436
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
logger.lifecycle("Removing 'dash.jar'")
file('dash.jar').delete()
logger.lifecycle("Removing 'deps.txt'")
file('deps.txt').delete()
doLast {
logger.lifecycle("Removing 'dash.jar'")
file('dash.jar').delete()
logger.lifecycle("Removing 'deps.txt'")
file('deps.txt').delete()
}

Should be wrapped inside a doLast-block so it is not executed during configuration phase

}

// Usage: in the root of the project: `./gradlew -q dashDependencies`
// The `-q` option is important if you want to use the output in a pipe.
tasks.register('dashDependencies') { dashDependencies ->
description = "Output all project dependencies as a flat list and save an intermediate file 'deps.txt'."
group = 'License'
dashDependencies.dependsOn('dashDownload')
doLast {
def deps = []
project.configurations.each { conf ->
// resolving 'archives' or 'default' is deprecated
if (conf.canBeResolved && conf.getName() != 'archives' && conf.getName() != 'default') {
deps.addAll(conf.incoming.resolutionResult.allDependencies
// the 'allDependencies' method return a 'DependencyResult'
// we're only interested in the 'ResolvedDependencyResult' sub-interface
// docs: https://docs.gradle.org/current/javadoc/org/gradle/api/artifacts/result/ResolutionResult.html#allDependencies-groovy.lang.Closure-
// docs: https://docs.gradle.org/current/javadoc/org/gradle/api/artifacts/result/DependencyResult.html
// docs: https://docs.gradle.org/current/javadoc/org/gradle/api/artifacts/result/ResolvedDependencyResult.html
.findAll({ it instanceof ResolvedDependencyResult })
.collect { ResolvedDependencyResult dep ->
"${dep.selected}"
})
}
}

def sorted = deps.unique().sort()
filtered.each { logger.quiet("{}", it) }
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
filtered.each { logger.quiet("{}", it) }
sorted.each { logger.quiet("{}", it) }

typo?

file("deps.txt").write(sorted.join('\n'))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional: can we rename the file to dependencies.txt pretty please? :) If yes, keep in mind to rename all occurrences in all files

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a different location would be preferable, e.g. $rootDir/build/oss

Advantage: It would automatically be cleaned with a ./gradlew clean

}
}

tasks.register('dashLicenseCheck', JavaExec) { dashLicenseCheck ->
description = "Run the Dash License Tool and save the summary in the 'DEPENDENCIES' file"
group = 'License'
dashLicenseCheck.dependsOn('dashDownload')
dashLicenseCheck.dependsOn('dashDependencies')
doFirst {
classpath = files('dash.jar')
// docs: https://eclipse-tractusx.github.io/docs/release/trg-7/trg-7-04
args('-project', '<your-project-here>', '-summary', 'DEPENDENCIES', 'deps.txt')
}
doLast {
logger.lifecycle("Removing 'deps.txt' now.")
file('deps.txt').delete()
}
}
```

Note that we have mixed success with this use of Gradle as it is very dependent on the specific nature of the build. Please verify that Gradle is correctly identifying your dependencies by invoking `./gradlew dependencies` before trying this.
Find all of the potentially problematic third party libraries from a Gradle build.

```bash
$ ./gradlew dashLicenseCheck
```
$ ./gradlew dependencies \
| grep -Poh "(?<=\-\-\- ).*" \
| grep -Pv "\([c\*]\)" \
| perl -pe 's/([\w\.\-]+):([\w\.\-]+):(?:[\w\.\-]+ -> )?([\w\.\-]+).*$/$1:$2:$3/gmi;t' \
| sort -u \
| java -jar org.eclipse.dash.licenses-<version>.jar -
```

Steps:

1. Use the Gradle `dependencies` command to generate a dependency list;
2. Extract the lines that contain references to content;
3. Remove the lines that are dependency constraints `(c)` or are duplicates `(*)`;
4. Normalise the GAV to the `groupid`, `artifactid` and resolved `version` (e.g., when the version is "1.8.20 -> 1.9.0", map that to "1.9.0");
5. Sort and remove duplicates; and
6. Invoke the tool.
You can use the `$ ./gradlew dashClean` command to remove your `dash.jar` and receive an updated copy through the execution of the `$ ./gradlew dashLicenseCheck` command.

### Example: Yarn

Expand Down