Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RFC: Buildpacks config #191

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft

RFC: Buildpacks config #191

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b2059b7
WIP: Initial idea of buildpacks config
jromero Nov 18, 2021
6cd8b76
Restructure 'images' map
jromero Nov 18, 2021
e9f36ba
Add motivations, definitions, and more details
jromero Nov 29, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
314 changes: 314 additions & 0 deletions text/0000-buildpacks-config.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,314 @@
# Meta
[meta]: #meta
- Name: (fill in the feature name: My Feature)
- Start Date: (fill in today's date: YYYY-MM-DD)
- Author(s): (Github usernames)
- Status: Draft <!-- Acceptable values: Draft, Approved, On Hold, Superseded -->
- RFC Pull Request: (leave blank)
- CNB Pull Request: (leave blank)
- CNB Issue: (leave blank)
- Supersedes:
- [RFC#19](https://github.com/buildpacks/rfcs/blob/main/text/0019-project-descriptor.md)
- [RFC#54](https://github.com/buildpacks/rfcs/blob/main/text/0054-project-descriptor-schema.md)
- [RFC#54](https://github.com/buildpacks/rfcs/blob/main/text/0054-project-descriptor-schema.md)
- [RFC#80](https://github.com/buildpacks/rfcs/blob/main/text/0080-builder-key-in-project-descriptor.md)
- [RFC#84](https://github.com/buildpacks/rfcs/blob/main/text/0084-project-descriptor-domains.md)

# Summary
[summary]: #summary

A configuration file intended for end-users to influence how a buildpack build occurs.

# Definitions
[definitions]: #definitions

- **Application API**: A new specification detail the contract and schema between the application, it's configuration, and the `lifecycle` components.
- **Platform API**: Existing specification detailing contract between the `platform` and `lifecycle`.
- **platform config**: A new file that provides a Platform API specific configuration to the platform from an Application API configuration.
- **project.toml (Project Descriptor)**: An existing configuration file created by this project with higher asperations of configuring not just buildpacks but other tools and platforms. See "Supersedes" meta issues above.

# Motivation
[motivation]: #motivation

The current use of the _project descriptor_ has a few issues:

### Cross-Platform Support

The current specification treats the _project descriptor_ as an "extension" specification meaning that it may only be optionally supported by platforms to be compliant.

From a user's point-of-view, it's unexpected to have their application built one way locally via `pack` (which supports `project.toml`) and another way using another platform such as `Tekton` (which doesn't support `project.toml`) where both platforms natively support Cloud Native Buildpacks. See reported [issue][issue-tekton-33].

### Obscure

Due to the generic intent of `project.toml`, it is not inherently clear that a project is configured to be built by Cloud Native Buildpacks using a `project.toml`. Compare this to recognizing a platform or system specific configuration file such as `.github/`, `Dockerfile`, `travis.yml`, etc.

```text
.
├── .git/
├── .github/
├── src/
├── .gitignore
├── .gitpod.yml
├── README.md
├── codecov.yml
├── golangci.yaml
├── Dockerfile
└── project.toml
```

### Complex Syntax

In trying to support multiple use cases in a single file, the syntax become slightly more complex by the use of ["reverse domain namespacing"][reverse-domain-namespacing].

```toml
[_]
api = "0.2"
id = "<string>" # machine readable
name = "<string>" # human readable
version = "<string>"
authors = ["<string>"]
documentation-url = "<url>"
source-url = "<url>"

[[_.licenses]]
type = "<string>"
uri = "<uri>"

[io.buildpacks]
api = "0.1"

[io.buildpacks.build]
include = ["<string>"]
exclude = ["<string>"]

[[io.buildpacks.build.buildpacks]]
id = "<string>"
version = "<string>"
uri = "<string>"

[[io.buildpacks.build.env]]
name = "<string>"
value = "<string>"
```

[reverse-domain-namespacing]: https://github.com/buildpacks/rfcs/blob/main/text/0084-project-descriptor-domains.md
[issue-tekton-33]: https://github.com/buildpacks/tekton-integration/issues/33

# What it is
[what-it-is]: #what-it-is

The proposed solution to the issues listed in the [motivation](#motivation) is to create a **Cloud Native Buildpacks specific file** that **MUST be supported by platforms**.

## File

`buildpacks.<ext>` where `<ext>` corrolates to a supported format.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is way too easily confused with buildpack.toml

Copy link
Member Author

@jromero jromero Nov 29, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree but do we expect app developers to be using buildpack.toml? To me, they are separated enough in context that I personally don't get troubled.

Alternatively, maybe something like Buildpacks, with no extension, similar to Dockerfile.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree but do we expect app developers to be using buildpack.toml?

If we don't, I think that makes the problem worse, because they won't even notice when they need to pay attention to the differences.


By using the term "buildpacks" in the file name, it becomes immediately apparent that this project has custom configuration for Cloud Native Buildpacks.

### Supported Formats

By supporting multiple file formats, we lower the barrier to entry and allow for the flexibility desired by some app developers.

Proposed supported extensions (in order of precedence):

- `toml` => TOML
- `yaml`, `yml` => YAML

#### TOML
Copy link
Member

@jkutner jkutner Nov 22, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a dot-file, i would expect this file structure to map directly to the pack commands. So a table for each command, with

[build]
builder = ""
clear-cache = true

  [[build.buildpacks]]
  id = "

and so on...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about the build table but given that all of the properties only apply to build I didn't see the need to add a layer of depth that the users need to repeatedly add.


```toml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't look any less complex to me than project.toml

schema="0.1"

[images]
names=[
"cnbs/sample-app",
"cnbs/sample-app:v1",
"grc.io/cnbs/sample-app:v1"
]

[images.builder]
name="cnbs/sample:builder"
trusted=true

[images.previous]
name="cnbs/sample-app"

[images.run]
name = "cnbs/sample-stack-run:bionic"
mirrors = [
"grc.io/cnbs/sample-stack-run:bionic",
]

[[env]] # build-time env vars
name="ENV_1"
value="ENV_2"
operation="append" # default=override

[[buildpacks]]
id = "samples/hello-world"
version = "0.0.1"

[[buildpacks]]
id = "samples/java-maven"
version = "0.0.1"

###
# Contents Configuration
###

[source]
workspace = "/workspace"
# exclude = [] # mutially exclusive with 'include'
include = [
"cmd/",
"go.mod",
"go.sum",
"*.go"
]

##
# Caching
# See https://github.com/buildpacks/rfcs/blob/main/text/0091-pack-cache-options.md
##

[cache]
format="image"
name="cnbs/sample-app-cache:build"

## Alternatives:
#
# [cache]
# format="volume"
# name=""
#
# [cache]
# format="bind"
# source="./cache"

###
# Process Specific Configuration
###

[process]
default = "web"

[[process.web.env]] # runtime env vars
name="PATH"
value="ENV_2"
operation="append" # default=override
```

#### YAML

```yaml
schema: "0.1"
images:
names:
- cnbs/sample-app
- 'cnbs/sample-app:v1'
- 'grc.io/cnbs/sample-app:v1'
builder:
name: 'cnbs/sample:builder'
trusted: true
previous:
name: cnbs/sample-app
run:
name: 'cnbs/sample-stack-run:bionic'
mirrors:
- 'grc.io/cnbs/sample-stack-run:bionic'
env:
- name: ENV_1
value: ENV_2
operation: append
buildpacks:
- id: samples/hello-world
version: 0.0.1
- id: samples/java-maven
version: 0.0.1
source:
workspace: /workspace
include:
- cmd/
- go.mod
- go.sum
- '*.go'
cache:
format: image
name: 'cnbs/sample-app-cache:build'
process:
default: web
web:
env:
- name: PATH
value: ENV_2
operation: append
```

# How it Works
[how-it-works]: #how-it-works

### Configuration Properties

To prevent unnecessary complexity for end-users (app developers), the `buildpacks.<ext>` file would only support properties associated with Cloud Native Buildpacks. See full list of properties in the [file](#file) section.

### Cross-Platform Support
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't appear to improve on the status quo (project.toml). The buildpacks.* would. need to be converted into another format to become cross-platform supportable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does in that Platforms become required to support buildpacks.*. That said, we could do the same to project.toml. This RFC tries to paint a new picture on a CNB specific config file that becomes required for platforms to support.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what are the advantages of requiring this file over project.toml?


Support for this configuration would be part of a _new_ specification (`Application API`).

Building upon [RFC PR#182][rfc-pr-182], the `buildpacks.<ext>` file would be part of a contract between the `application` -> `converter` -> `platform`. Platforms will be required to support the `buildpacks.<ext>` files through changes to the `Platform API` and use of an intermediate file named _platform config_ (`platform.toml`). The `platform.toml` will be a Platform API version specific configuration file which contains only properties the platform can handle. Any unsupported features should yield a warning. Platforms can additional opt-out of features but would also be requested to yield warnings or errors in such cases.

```text
Application API
┌────────────────────────────────────────────────────────────────┐
│ │

┌───────┐
│ │ buildpacks.* ┌───────────┐
┌────────┐ │ p ├───────────────────► │
│ │ buildpacks.* │ l │ │ converter │
│ source ├───────────────► a │ │ │
│ │ │ t │ └──┬────────┘
└────────┘ │ f │ │
│ o ◄──────────────────────┘
│ r │ platform.toml
│ m │
│ │
└───────┘

│ │
└───────────────────────────────────────┘
Platform API
```


[rfc-pr-182]: https://github.com/buildpacks/rfcs/pull/182

# Drawbacks
[drawbacks]: #drawbacks

Why should we *not* do this?
Copy link
Member

@jkutner jkutner Nov 29, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • buildpacks.* is easily confused with buildpack.toml
  • buildpacks* is not inherently portable. We still need to implement a mechanism and more spec to convert it to something portable
  • yet-another-config-file: I won't have a place to put config that's common across tools (license, include/exclude, version, image names, etc)
  • doesn't map well to pack commands/flags


# Alternatives
[alternatives]: #alternatives

- What other designs have been considered?
- Why is this proposal the best?
- What is the impact of not doing this?

# Prior Art
[prior-art]: #prior-art

Discuss prior art, both the good and bad.

# Unresolved Questions
[unresolved-questions]: #unresolved-questions

- What parts of the design do you expect to be resolved before this gets merged?
- What parts of the design do you expect to be resolved through implementation of the feature?
- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC?

# Spec. Changes (OPTIONAL)
[spec-changes]: #spec-changes
Does this RFC entail any proposed changes to the core specifications or extensions? If so, please document changes here.
Examples of a spec. change might be new lifecycle flags, new `buildpack.toml` fields, new fields in the buildpackage label, etc.
This section is not intended to be binding, but as discussion of an RFC unfolds, if spec changes are necessary, they should be documented here.