Skip to content

Commit

Permalink
Implement git refs (#79)
Browse files Browse the repository at this point in the history 
This closes #73.

This PR does not remove `--commit`, `--branch`, or `--tag` options, because that would be a breaking change. That could be an option in the future, as long as we know the impact of that on everyone's workflows.
  • Loading branch information
@pete0emerson
pete0emerson authored Jan 13, 2021
1 parent 0b7d5da commit 37e8f10
Show file tree
Hide file tree
Showing 5 changed files with 186 additions and 96 deletions.
17 changes: 9 additions & 8 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ authentication. Fetch makes it possible to handle all of these cases with a one-

#### Features

- Download from a specific git tag, branch, or commit SHA.
- Download from any git reference, such as a specific git tag, branch, or commit SHA.
- Download a single file, a subset of files, or all files from the repo.
- Download one or more binary assets from a specific release that match a regular expression.
- Verify the SHA256 or SHA512 checksum of a binary asset.
Expand Down Expand Up @@ -74,6 +74,7 @@ fetch [OPTIONS] <local-download-path>
The supported options are:

- `--repo` (**Required**): The fully qualified URL of the GitHub repo to download from (e.g. https://github.com/foo/bar).
- `--ref` (**Optional**): The git reference to download. If specified, will override `--commit`, `--branch`, and `--tag`.
- `--tag` (**Optional**): The git tag to download. Can be a specific tag or a [Tag Constraint
Expression](#tag-constraint-expressions).
- `--branch` (**Optional**): The git branch from which to download; the latest commit in the branch will be used. If
Expand Down Expand Up @@ -137,7 +138,7 @@ fetch --repo="https://github.com/foo/bar" --tag="~>0.1.5" --source-path="/module
Download all files in `/modules/foo` from a GitHub release where the tag is exactly `0.1.5`, and save them to `/tmp`:

```
fetch --repo="https://github.com/foo/bar" --tag="0.1.5" --source-path="/modules/foo" /tmp
fetch --repo="https://github.com/foo/bar" --ref="0.1.5" --source-path="/modules/foo" /tmp
```

#### Usage Example 3
Expand All @@ -147,39 +148,39 @@ Download all files from a private GitHub repo using the GitHUb oAuth Token `123`
```
GITHUB_OAUTH_TOKEN=123
fetch --repo="https://github.com/foo/bar" --tag="0.1.5" /tmp
fetch --repo="https://github.com/foo/bar" --ref="0.1.5" /tmp
```

#### Usage Example 4

Download all files from the latest commit on the `sample-branch` branch, and save them to `/tmp`:

```
fetch --repo="https://github.com/foo/bar" --branch="sample-branch" /tmp/josh1
fetch --repo="https://github.com/foo/bar" --ref="sample-branch" /tmp/josh1
```

#### Usage Example 5

Download all files from the git commit `f32a08313e30f116a1f5617b8b68c11f1c1dbb61`, and save them to `/tmp`:

```
fetch --repo="https://github.com/foo/bar" --commit="f32a08313e30f116a1f5617b8b68c11f1c1dbb61" /tmp/josh1
fetch --repo="https://github.com/foo/bar" --ref="f32a08313e30f116a1f5617b8b68c11f1c1dbb61" /tmp
```

#### Usage Example 6

Download the release asset `foo.exe` from a GitHub release where the tag is exactly `0.1.5`, and save it to `/tmp`:

```
fetch --repo="https://github.com/foo/bar" --tag="0.1.5" --release-asset="foo.exe" /tmp
fetch --repo="https://github.com/foo/bar" --ref="0.1.5" --release-asset="foo.exe" /tmp
```

#### Usage Example 7

Download the release asset `foo.exe` from a GitHub release hosted on a GitHub Enterprise instance running at `ghe.mycompany.com` where the tag is exactly `0.1.5`, and save it to `/tmp`:

```
fetch --repo="https://ghe.mycompany.com/foo/bar" --tag="0.1.5" --release-asset="foo.exe" /tmp
fetch --repo="https://ghe.mycompany.com/foo/bar" --ref="0.1.5" --release-asset="foo.exe" /tmp
```

## License
Expand All @@ -190,4 +191,4 @@ This code is released under the MIT License. See [LICENSE.txt](/LICENSE.txt).

- Introduce code verification using something like GPG signatures or published checksums
- Explicitly test for exotic repo and org names
- Apply stricter parsing for repo-filter command-line arg
- Apply stricter parsing for repo-filter command-line arg
4 changes: 3 additions & 1 deletion file.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,9 @@ func MakeGitHubZipFileRequest(gitHubCommit GitHubCommit, gitHubToken string, ins

// This represents either a commit, branch, or git tag
var gitRef string
if gitHubCommit.CommitSha != "" {
if gitHubCommit.GitRef != "" {
gitRef = gitHubCommit.GitRef
} else if gitHubCommit.CommitSha != "" {
gitRef = gitHubCommit.CommitSha
} else if gitHubCommit.BranchName != "" {
gitRef = gitHubCommit.BranchName
Expand Down
228 changes: 146 additions & 82 deletions file_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,44 +44,56 @@ func TestDownloadGitTagZipFile(t *testing.T) {
}

for _, tc := range cases {
gitHubCommit := GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
gitHubCommits := []GitHubCommit{
// Test as a GitTag
GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
},
GitTag: tc.gitTag,
},
// Test as a GitRef
GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
},
GitRef: tc.gitTag,
},
GitTag: tc.gitTag,
}
for _, gitHubCommit := range gitHubCommits {
zipFilePath, err := downloadGithubZipFile(gitHubCommit, tc.githubToken, tc.instance)

zipFilePath, err := downloadGithubZipFile(gitHubCommit, tc.githubToken, tc.instance)

defer os.RemoveAll(zipFilePath)
defer os.RemoveAll(zipFilePath)

// We don't have a running instance of GitHub Enterprise against which to validate tests as we do for GitHub public,
// so this test will only validate that fetch attempted to download from the expected URL. The download itself
// will fail.
// We don't have a running instance of GitHub Enterprise against which to validate tests as we do for GitHub public,
// so this test will only validate that fetch attempted to download from the expected URL. The download itself
// will fail.

githubEnterpriseDownloadUrl := fmt.Sprintf("https://%s/repos/%s/%s/zipball/%s", tc.instance.ApiUrl, tc.repoOwner, tc.repoName, tc.gitTag)
githubEnterpriseDownloadUrl := fmt.Sprintf("https://%s/repos/%s/%s/zipball/%s", tc.instance.ApiUrl, tc.repoOwner, tc.repoName, tc.gitTag)

// TODO: The awkwardness of this test makes it clear that a better structure for this program would be to refactor
// the downloadGithubZipFile() function to a function called downloadGithubFile() that would accept a URL as a
// param. We could then test explicitly that the URL is as expected, which would make GitHub Enterprise test cases
// simpler to handle.
// TODO: The awkwardness of this test makes it clear that a better structure for this program would be to refactor
// the downloadGithubZipFile() function to a function called downloadGithubFile() that would accept a URL as a
// param. We could then test explicitly that the URL is as expected, which would make GitHub Enterprise test cases
// simpler to handle.

if err != nil && strings.Contains(err.Error(), "no such host") {
if strings.Contains(err.Error(), githubEnterpriseDownloadUrl) {
t.Logf("Found expected download URL %s. Download itself failed as expected because no GitHub Enterprise instance exists at the given URL.", githubEnterpriseDownloadUrl)
return
} else {
t.Fatalf("Attempted to download from URL other than the expected download URL of %s. Full error: %s", githubEnterpriseDownloadUrl, err.Error())
if err != nil && strings.Contains(err.Error(), "no such host") {
if strings.Contains(err.Error(), githubEnterpriseDownloadUrl) {
t.Logf("Found expected download URL %s. Download itself failed as expected because no GitHub Enterprise instance exists at the given URL.", githubEnterpriseDownloadUrl)
return
} else {
t.Fatalf("Attempted to download from URL other than the expected download URL of %s. Full error: %s", githubEnterpriseDownloadUrl, err.Error())
}
}
}

if err != nil {
t.Fatalf("Failed to download file: %s", err)
}
if err != nil {
t.Fatalf("Failed to download file: %s", err)
}

if _, err := os.Stat(zipFilePath); os.IsNotExist(err) {
t.Fatalf("Downloaded file doesn't exist at the expected path of %s", zipFilePath)
if _, err := os.Stat(zipFilePath); os.IsNotExist(err) {
t.Fatalf("Downloaded file doesn't exist at the expected path of %s", zipFilePath)
}
}
}
}
Expand All @@ -106,22 +118,32 @@ func TestDownloadGitBranchZipFile(t *testing.T) {
}

for _, tc := range cases {
gitHubCommit := GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
gitHubCommits := []GitHubCommit{
GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
},
BranchName: tc.branchName,
},
GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
},
GitRef: tc.branchName,
},
BranchName: tc.branchName,
}

zipFilePath, err := downloadGithubZipFile(gitHubCommit, tc.githubToken, tc.instance)
defer os.RemoveAll(zipFilePath)
if err != nil {
t.Fatalf("Failed to download file: %s", err)
}
for _, gitHubCommit := range gitHubCommits {
zipFilePath, err := downloadGithubZipFile(gitHubCommit, tc.githubToken, tc.instance)
defer os.RemoveAll(zipFilePath)
if err != nil {
t.Fatalf("Failed to download file: %s", err)
}

if _, err := os.Stat(zipFilePath); os.IsNotExist(err) {
t.Fatalf("Downloaded file doesn't exist at the expected path of %s", zipFilePath)
if _, err := os.Stat(zipFilePath); os.IsNotExist(err) {
t.Fatalf("Downloaded file doesn't exist at the expected path of %s", zipFilePath)
}
}
}
}
Expand All @@ -145,18 +167,28 @@ func TestDownloadBadGitBranchZipFile(t *testing.T) {
}

for _, tc := range cases {
gitHubCommit := GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
gitHubCommits := []GitHubCommit{
GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
},
BranchName: tc.branchName,
},
GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
},
GitRef: tc.branchName,
},
BranchName: tc.branchName,
}

zipFilePath, err := downloadGithubZipFile(gitHubCommit, tc.githubToken, tc.instance)
defer os.RemoveAll(zipFilePath)
if err == nil {
t.Fatalf("Expected that attempt to download repo %s/%s for branch \"%s\" would fail, but received no error.", tc.repoOwner, tc.repoName, tc.branchName)
for _, gitHubCommit := range gitHubCommits {
zipFilePath, err := downloadGithubZipFile(gitHubCommit, tc.githubToken, tc.instance)
defer os.RemoveAll(zipFilePath)
if err == nil {
t.Fatalf("Expected that attempt to download repo %s/%s for branch \"%s\" would fail, but received no error.", tc.repoOwner, tc.repoName, tc.branchName)
}
}
}
}
Expand All @@ -183,22 +215,32 @@ func TestDownloadGitCommitFile(t *testing.T) {
}

for _, tc := range cases {
gitHubCommit := GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
GitHubCommits := []GitHubCommit{
GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
},
CommitSha: tc.commitSha,
},
GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
},
GitRef: tc.commitSha,
},
CommitSha: tc.commitSha,
}

zipFilePath, err := downloadGithubZipFile(gitHubCommit, tc.githubToken, tc.instance)
defer os.RemoveAll(zipFilePath)
if err != nil {
t.Fatalf("Failed to download file: %s", err)
}
for _, gitHubCommit := range GitHubCommits {
zipFilePath, err := downloadGithubZipFile(gitHubCommit, tc.githubToken, tc.instance)
defer os.RemoveAll(zipFilePath)
if err != nil {
t.Fatalf("Failed to download file: %s", err)
}

if _, err := os.Stat(zipFilePath); os.IsNotExist(err) {
t.Fatalf("Downloaded file doesn't exist at the expected path of %s", zipFilePath)
if _, err := os.Stat(zipFilePath); os.IsNotExist(err) {
t.Fatalf("Downloaded file doesn't exist at the expected path of %s", zipFilePath)
}
}
}
}
Expand Down Expand Up @@ -227,18 +269,29 @@ func TestDownloadBadGitCommitFile(t *testing.T) {
}

for _, tc := range cases {
gitHubCommit := GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,

gitHubCommits := []GitHubCommit{
GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
},
CommitSha: tc.commitSha,
},
GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
},
GitRef: tc.commitSha,
},
CommitSha: tc.commitSha,
}

zipFilePath, err := downloadGithubZipFile(gitHubCommit, tc.githubToken, tc.instance)
defer os.RemoveAll(zipFilePath)
if err == nil {
t.Fatalf("Expected that attempt to download repo %s/%s at commmit sha \"%s\" would fail, but received no error.", tc.repoOwner, tc.repoName, tc.commitSha)
for _, gitHubCommit := range gitHubCommits {
zipFilePath, err := downloadGithubZipFile(gitHubCommit, tc.githubToken, tc.instance)
defer os.RemoveAll(zipFilePath)
if err == nil {
t.Fatalf("Expected that attempt to download repo %s/%s at commmit sha \"%s\" would fail, but received no error.", tc.repoOwner, tc.repoName, tc.commitSha)
}
}
}
}
Expand All @@ -262,17 +315,28 @@ func TestDownloadZipFileWithBadRepoValues(t *testing.T) {
}

for _, tc := range cases {
gitHubCommit := GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
gitHubCommits := []GitHubCommit{
GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
},
GitTag: tc.gitTag,
},
GitHubCommit{
Repo: GitHubRepo{
Owner: tc.repoOwner,
Name: tc.repoName,
},
GitRef: tc.gitTag,
},
GitTag: tc.gitTag,
}
for _, gitHubCommit := range gitHubCommits {

_, err := downloadGithubZipFile(gitHubCommit, tc.githubToken, tc.instance)
if err == nil && err.errorCode != 500 {
t.Fatalf("Expected error for bad repo values: %s/%s:%s", tc.repoOwner, tc.repoName, tc.gitTag)
_, err := downloadGithubZipFile(gitHubCommit, tc.githubToken, tc.instance)
if err == nil && err.errorCode != 500 {
t.Fatalf("Expected error for bad repo values: %s/%s:%s", tc.repoOwner, tc.repoName, tc.gitTag)
}
}
}
}
Expand Down
1 change: 1 addition & 0 deletions github.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ type GitHubInstance struct {
// - Example: BranchName alone is specified; use BranchName
type GitHubCommit struct {
Repo GitHubRepo // The GitHub repo where this release lives
GitRef string // The git reference
GitTag string // The specific git tag for this release
BranchName string // If specified, indicates that this commit should be the latest commit on the given branch
CommitSha string // If specified, indicates that this commit should be exactly this Git Commit SHA.
Expand Down
Loading

0 comments on commit 37e8f10

Please sign in to comment.