-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
build docs, helper scripts #192
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
# Building a go binary that includes golang-fips patches | ||
|
||
## Env variables | ||
|
||
`GOEXPERIMENT=strictfipsruntime` helps ensure a fips compliant binary is built. Note that this is functionally equivalent of | ||
|
||
``` | ||
GO_BUILDTAGS+="goexperiment.strictfipsruntime" | ||
``` | ||
|
||
## Native build - Method 1 | ||
|
||
The easiest way to do is to simply clone this repo, apply patches and from the internal go/src directory run ./make.bash. See a more comprehensive script at `scripts/build1.sh` | ||
|
||
``` | ||
git clone https://github.com/golang-fips/go.git golang-fips | ||
cd golang-fips | ||
./scripts/full-initialize-repo.sh go$GOLANG_VER | ||
cd go/src | ||
./make.bash --no-clean | ||
``` | ||
|
||
## Native build - Method 2 | ||
|
||
Another way is to directly apply fips patches on downloaded go and golang-fips binaries. See full script at `scripts/build2.sh` | ||
|
||
``` | ||
wget https://github.com/golang/go/archive/refs/tags/go1.22.2.tar.gz | ||
wget https://github.com/golang-fips/go/archive/refs/tags/go1.22.2-1-openssl-fips.tar.gz | ||
tar -xf go1.22.2.tar.gz | ||
tar -xf go1.22.2-1-openssl-fips.tar.gz | ||
cd go-go1.22.2 | ||
for patch in ../go-go1.22.2-1-openssl-fips/patches/*.patch; do | ||
patch -p1 < "${patch}" | ||
done | ||
cd src | ||
./make.bash --no-clean | ||
``` | ||
|
||
## Container Build | ||
|
||
So far the two methods above described steps to build golang natively. To build golang inside a container, copy the sample Dockerfile located at scripts/Dockerfile.sample to a new directory, modify it to add distro specific variables and run docker/podman on it. | ||
|
||
``` | ||
mkdir build-context | ||
cp scripts/Dockerfile.sample scripts/Dockerfile | ||
# modify it to make it distro specific | ||
podman/docker build . | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# Dependencies | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fill in with actual documentation, I think we just include it directly in this PR. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# OpenSSL considerations | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fill in with actual documentation, I think we just include it directly in this PR. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# Overview here | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fill in with actual documentation, I think we just include it directly in this PR. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# Test configuration | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fill in with actual documentation, I think we just include it directly in this PR. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
FROM <distro-specific-base-image> | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure what benefit we get from having this Dockerfile in here. In fact, if it's just a sample and not meant to be runnable directly and used via any kind of CI or anything I can see it getting stagnant over time. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do u think it would help if we add distro specific files under a new scripts/distro folder and have that add over time.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think container images are better suited for downstream maintainers to handle. We don't intend to provide an upstream golang-fips container image, so we don't need to include a Dockerfile. |
||
|
||
ARG GOLANG_VER=1.21.9 | ||
# turn on FIPS and CGO_ENABLED by default | ||
ARG FIPS=1 | ||
ARG GODEBUG=0 | ||
ARG CGO_ENABLED=1 | ||
|
||
# certain env need proxy to download from github | ||
ENV http_proxy="" | ||
ENV https_proxy="" | ||
ENV no_proxy="" | ||
|
||
ENV GOLANG_FIPS=$FIPS | ||
ENV GODEBUG=$GODEBUG | ||
ENV CGO_ENABLED=1 | ||
|
||
# distro specific package manager | ||
# need bootstrap golang to build go, will be removed later | ||
RUN dnf/apt install -y git wget golang && \ | ||
dnf/apt groupinstall -y "Development Tools" | ||
|
||
RUN go version | ||
RUN echo "GOROOT_BOOTSTRAP = / by default so will use installed go as compiler" | ||
RUN git config --global user.email "<add email here>" | ||
RUN git config --global user.name "<add name here>" | ||
|
||
RUN git clone https://github.com/golang-fips/go.git golang-fips && cd golang-fips && \ | ||
GOLANG_VER_SHORT=${GOLANG_VER%.*} && \ | ||
git fetch origin go${GOLANG_VER_SHORT}-fips-release && git checkout go${GOLANG_VER_SHORT}-fips-release && \ | ||
./scripts/full-initialize-repo.sh go$GOLANG_VER | ||
|
||
RUN cd golang-fips/go && \ | ||
cd src && ./make.bash --no-clean | ||
|
||
# rm bootstrap golang | ||
RUN dnf -y erase golang | ||
|
||
ENV PATH=/golang-fips/go/bin:$PATH:/usr/local/go/bin | ||
ENV GOPATH=$HOME/go | ||
ENV PATH=$PATH:$GOPATH/bin | ||
|
||
RUN go version |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
#!/usr/bin/env bash | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Improved documentation would be preferred over adding another script, as this is already attainable with the current build scripts. |
||
# this script builds a go binary after applying openssl fips patches. | ||
|
||
set -x | ||
export GOLANG_VER=1.22.2 | ||
[[ -d go ]] || ./scripts/full-initialize-repo.sh go$GOLANG_VER | ||
cd go/src | ||
export GO_BUILDTAGS+="goexperiment.strictfipsruntime" | ||
export CGO_ENABLED=1 | ||
./make.bash --no-clean | ||
{ set +x ; } 2>/dev/null |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
#! /usr/bin/env bash | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure the benefit of adding this script, it's just recreating what the other scripts already do, right? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. scripts/build1.sh -> clones golang-fips and run full-initialize-repo.sh scripts/build2.sh -> clone golang binaries and apply patches one by one. In cases when working with downstream distros, users already have a go source tar e.g. centos go tar file (different from upstream vanilla go source that build1.sh uses), and they just care about applying patches on them, one by one. In those cases, second script helps. Atleast that was my intention of adding the second script. I had to use the second one when building golang-fips on RH golang source imported from OpenELA. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see why someone might find these scripts useful, but they appear rather configuration and workflow specific. Whereas the existing builds scripts are workflow agnostic and already capable of producing a Go toolchain with the same configuration. I would suggest to instead that we document a more generic |
||
# this script builds a go binary after applying openssl fips patches one by one. | ||
# | ||
export GOLANG_VER=1.22.2 | ||
export GOLANG_REL=1 | ||
|
||
# variables to store tar file names | ||
GO_TAR_NAME=go$GOLANG_VER | ||
PATCH_TAR_NAME=go$GOLANG_VER-$GOLANG_REL-openssl-fips | ||
GO_SRC_TAR=$GO_TAR_NAME.tar.gz | ||
PATCH_TAR=$PATCH_TAR_NAME.tar.gz | ||
|
||
# after untar, the dir names are stored in PATCH_DIR and SRC_DIR | ||
SRC_DIR=go-$GO_TAR_NAME | ||
PATCH_DIR=go-$PATCH_TAR_NAME | ||
TOP_DIR=/tmp/golang-fips/build2 | ||
# If the go src dir ends up in a weird state where in patches are failing to apply, you might want to remove the $TOP_DIR (rm -rf /tmp/golang-fips/build2) to start with a clean slate. | ||
PATCH_PATH=$TOP_DIR/$PATCH_DIR/patches | ||
|
||
# this file stores state of last applied patch, that way patches are not reapplied | ||
PATCH_STATE=$TOP_DIR/state | ||
set -x | ||
[[ -d $TOP_DIR ]] || mkdir -p $TOP_DIR | ||
cd $TOP_DIR | ||
|
||
[[ -f $GO_SRC_TAR ]] || wget -q --show-progress https://github.com/golang/go/archive/refs/tags/$GO_SRC_TAR | ||
[[ -f $PATCH_TAR ]] || wget -q --show-progress https://github.com/golang-fips/go/archive/refs/tags/$PATCH_TAR | ||
[[ -d $SRC_DIR ]] || tar -xf $GO_SRC_TAR | ||
[[ -d $PATCH_DIR ]] || tar -xf $PATCH_TAR | ||
|
||
cd $SRC_DIR | ||
{ set +x; } 2>/dev/null | ||
if [[ -f $PATCH_STATE && "$(cat $PATCH_STATE)" == "complete" ]]; then | ||
echo "patches already applied. skipping" | ||
else | ||
for patch in $PATCH_PATH/*.patch; do | ||
patch -p1 < "${patch}" | ||
[[ $? -eq 0 ]] || { echo "incomplete" > $PATCH_STATE; break; exit 1; } | ||
done | ||
fi | ||
|
||
echo "complete" > $PATCH_STATE | ||
# patches have been supplied successfully, simply build like above. | ||
set -x | ||
cd src | ||
export GO_BUILDTAGS+="goexperiment.strictfipsruntime,!no_openssl" | ||
export CGO_ENABLED=1 | ||
./make.bash --no-clean | ||
{ set +x; } 2>/dev/null |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The default version described in go/config/versions.json is the only explicitly compatible upstream version at a given time. It would be preferable to recommend calling this script without the version-override argument and just use the default configuration.