Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build docs, helper scripts #192

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions docs/build.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Building a go binary that includes golang-fips patches

## Env variables

`GOEXPERIMENT=strictfipsruntime` helps ensure a fips compliant binary is built. Note that this is functionally equivalent of

```
GO_BUILDTAGS+="goexperiment.strictfipsruntime"
```

## Native build - Method 1

The easiest way to do is to simply clone this repo, apply patches and from the internal go/src directory run ./make.bash. See a more comprehensive script at `scripts/build1.sh`

```
git clone https://github.com/golang-fips/go.git golang-fips
cd golang-fips
./scripts/full-initialize-repo.sh go$GOLANG_VER
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default version described in go/config/versions.json is the only explicitly compatible upstream version at a given time. It would be preferable to recommend calling this script without the version-override argument and just use the default configuration.

cd go/src
./make.bash --no-clean
```

## Native build - Method 2

Another way is to directly apply fips patches on downloaded go and golang-fips binaries. See full script at `scripts/build2.sh`

```
wget https://github.com/golang/go/archive/refs/tags/go1.22.2.tar.gz
wget https://github.com/golang-fips/go/archive/refs/tags/go1.22.2-1-openssl-fips.tar.gz
tar -xf go1.22.2.tar.gz
tar -xf go1.22.2-1-openssl-fips.tar.gz
cd go-go1.22.2
for patch in ../go-go1.22.2-1-openssl-fips/patches/*.patch; do
patch -p1 < "${patch}"
done
cd src
./make.bash --no-clean
```

## Container Build

So far the two methods above described steps to build golang natively. To build golang inside a container, copy the sample Dockerfile located at scripts/Dockerfile.sample to a new directory, modify it to add distro specific variables and run docker/podman on it.

```
mkdir build-context
cp scripts/Dockerfile.sample scripts/Dockerfile
# modify it to make it distro specific
podman/docker build .
```
1 change: 1 addition & 0 deletions docs/deps.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Dependencies
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fill in with actual documentation, I think we just include it directly in this PR.

1 change: 1 addition & 0 deletions docs/openssl.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# OpenSSL considerations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fill in with actual documentation, I think we just include it directly in this PR.

1 change: 1 addition & 0 deletions docs/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Overview here
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fill in with actual documentation, I think we just include it directly in this PR.

1 change: 1 addition & 0 deletions docs/test-conf.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Test configuration
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fill in with actual documentation, I think we just include it directly in this PR.

43 changes: 43 additions & 0 deletions scripts/Dockerfile.sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
FROM <distro-specific-base-image>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what benefit we get from having this Dockerfile in here. In fact, if it's just a sample and not meant to be runnable directly and used via any kind of CI or anything I can see it getting stagnant over time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do u think it would help if we add distro specific files under a new scripts/distro folder and have that add over time.

distro/rh/Dockerfile
distro/centos/Dockerfile 

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think container images are better suited for downstream maintainers to handle. We don't intend to provide an upstream golang-fips container image, so we don't need to include a Dockerfile.


ARG GOLANG_VER=1.21.9
# turn on FIPS and CGO_ENABLED by default
ARG FIPS=1
ARG GODEBUG=0
ARG CGO_ENABLED=1

# certain env need proxy to download from github
ENV http_proxy=""
ENV https_proxy=""
ENV no_proxy=""

ENV GOLANG_FIPS=$FIPS
ENV GODEBUG=$GODEBUG
ENV CGO_ENABLED=1

# distro specific package manager
# need bootstrap golang to build go, will be removed later
RUN dnf/apt install -y git wget golang && \
dnf/apt groupinstall -y "Development Tools"

RUN go version
RUN echo "GOROOT_BOOTSTRAP = / by default so will use installed go as compiler"
RUN git config --global user.email "<add email here>"
RUN git config --global user.name "<add name here>"

RUN git clone https://github.com/golang-fips/go.git golang-fips && cd golang-fips && \
GOLANG_VER_SHORT=${GOLANG_VER%.*} && \
git fetch origin go${GOLANG_VER_SHORT}-fips-release && git checkout go${GOLANG_VER_SHORT}-fips-release && \
./scripts/full-initialize-repo.sh go$GOLANG_VER

RUN cd golang-fips/go && \
cd src && ./make.bash --no-clean

# rm bootstrap golang
RUN dnf -y erase golang

ENV PATH=/golang-fips/go/bin:$PATH:/usr/local/go/bin
ENV GOPATH=$HOME/go
ENV PATH=$PATH:$GOPATH/bin

RUN go version
11 changes: 11 additions & 0 deletions scripts/build1.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/usr/bin/env bash
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improved documentation would be preferred over adding another script, as this is already attainable with the current build scripts.

# this script builds a go binary after applying openssl fips patches.

set -x
export GOLANG_VER=1.22.2
[[ -d go ]] || ./scripts/full-initialize-repo.sh go$GOLANG_VER
cd go/src
export GO_BUILDTAGS+="goexperiment.strictfipsruntime"
export CGO_ENABLED=1
./make.bash --no-clean
{ set +x ; } 2>/dev/null
49 changes: 49 additions & 0 deletions scripts/build2.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
#! /usr/bin/env bash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure the benefit of adding this script, it's just recreating what the other scripts already do, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scripts/build1.sh -> clones golang-fips and run full-initialize-repo.sh

scripts/build2.sh -> clone golang binaries and apply patches one by one.

In cases when working with downstream distros, users already have a go source tar e.g. centos go tar file (different from upstream vanilla go source that build1.sh uses), and they just care about applying patches on them, one by one. In those cases, second script helps. Atleast that was my intention of adding the second script. I had to use the second one when building golang-fips on RH golang source imported from OpenELA.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see why someone might find these scripts useful, but they appear rather configuration and workflow specific. Whereas the existing builds scripts are workflow agnostic and already capable of producing a Go toolchain with the same configuration. I would suggest to instead that we document a more generic disconnected build approach using the patches, and avoid adding additional scripts.

# this script builds a go binary after applying openssl fips patches one by one.
#
export GOLANG_VER=1.22.2
export GOLANG_REL=1

# variables to store tar file names
GO_TAR_NAME=go$GOLANG_VER
PATCH_TAR_NAME=go$GOLANG_VER-$GOLANG_REL-openssl-fips
GO_SRC_TAR=$GO_TAR_NAME.tar.gz
PATCH_TAR=$PATCH_TAR_NAME.tar.gz

# after untar, the dir names are stored in PATCH_DIR and SRC_DIR
SRC_DIR=go-$GO_TAR_NAME
PATCH_DIR=go-$PATCH_TAR_NAME
TOP_DIR=/tmp/golang-fips/build2
# If the go src dir ends up in a weird state where in patches are failing to apply, you might want to remove the $TOP_DIR (rm -rf /tmp/golang-fips/build2) to start with a clean slate.
PATCH_PATH=$TOP_DIR/$PATCH_DIR/patches

# this file stores state of last applied patch, that way patches are not reapplied
PATCH_STATE=$TOP_DIR/state
set -x
[[ -d $TOP_DIR ]] || mkdir -p $TOP_DIR
cd $TOP_DIR

[[ -f $GO_SRC_TAR ]] || wget -q --show-progress https://github.com/golang/go/archive/refs/tags/$GO_SRC_TAR
[[ -f $PATCH_TAR ]] || wget -q --show-progress https://github.com/golang-fips/go/archive/refs/tags/$PATCH_TAR
[[ -d $SRC_DIR ]] || tar -xf $GO_SRC_TAR
[[ -d $PATCH_DIR ]] || tar -xf $PATCH_TAR

cd $SRC_DIR
{ set +x; } 2>/dev/null
if [[ -f $PATCH_STATE && "$(cat $PATCH_STATE)" == "complete" ]]; then
echo "patches already applied. skipping"
else
for patch in $PATCH_PATH/*.patch; do
patch -p1 < "${patch}"
[[ $? -eq 0 ]] || { echo "incomplete" > $PATCH_STATE; break; exit 1; }
done
fi

echo "complete" > $PATCH_STATE
# patches have been supplied successfully, simply build like above.
set -x
cd src
export GO_BUILDTAGS+="goexperiment.strictfipsruntime,!no_openssl"
export CGO_ENABLED=1
./make.bash --no-clean
{ set +x; } 2>/dev/null