Add files/folders to a specific commit in history (initial or second commit)

[tbnorris]

I have a task to move some existing repos to GitHub and we are wanting to add somethings like .github/CODEOWNERS and workflow kind of data to the repo from the initial commit as we migrate them.
We are pulling in a mirror repo, purging some Gerrit related refs and .lfsconfig files using git-filter-repo, and changing the repo's url before we push it.

We are setting up some workflow and github actions, but want to have so we could trigger bug fixes on the history of commits so were trying to add the .github folder (that is already setup for our new repo for new work products) to our initial commit and have it propagate to all branches and all tags so if we need to do work on fixing an older release (via tag number) the new release/feature/fix branch will already have the actions and code owner setup as well as the tip of the main branch as if the repo was created in GitHub with all the actions, codeowners, and workflow there from the start.

It feels like git-filter-repo has that power, but I cannot seem to make it work.

We have tried
something like

$ git checkout -b withgitignore $firstcommithash
$ git add .gitignore
$ git commit --amend
$ for branch in branch1 branch2 branch3 ... ; do
      git checkout $branch
      git rebase withgitignore
   done

but it requires us to do it with each branch and pretty much requires user interaction which we can't do.
I have found this: https://github.com/newren/git-filter-repo/blob/main/contrib/filter-repo-demos/insert-beginning
but am completely lost reading how I can "oneliner" this code.
It seems like it should be something like this;

$ RELATIVE_TO_PROJECT_ROOT_PATHNAME="../template_repo/.github" # also tried with an end slash and with /CODEOWNERS
$ FILENAME=".github"
# also tried the above two lines with an end slash (.github/) and with .github/CODEOWNERS and it just says it can't find whatever "FILENAME" is set to
$ git filter-repo --force --commit-callback "if not commit.parents: commit.file_changes.append(FileChange(b'M', $RELATIVE_TO_PROJECT_ROOT_PATHNAME, b'$(git hash-object -w $FILENAME)', b'100644'))"
fatal: could not open '.github' for reading: No such file or directory
Traceback (most recent call last):
  File "C:\Users\xzlg3r\bin\git-filter-repo", line 4005, in <module>
    main()
  File "C:\Users\xzlg3r\bin\git-filter-repo", line 4002, in main
    filter.run()
  File "C:\Users\xzlg3r\bin\git-filter-repo", line 3937, in run
    self._parser.run(self._input, self._output)
  File "C:\Users\xzlg3r\bin\git-filter-repo", line 1409, in run
    self._parse_commit()
  File "C:\Users\xzlg3r\bin\git-filter-repo", line 1260, in _parse_commit
    self._commit_callback(commit, aux_info)
  File "C:\Users\xzlg3r\bin\git-filter-repo", line 3485, in _tweak_commit
    self._insert_into_stream(commit)
  File "C:\Users\xzlg3r\bin\git-filter-repo", line 3912, in _insert_into_stream
    self._parser.insert(obj)
  File "C:\Users\xzlg3r\bin\git-filter-repo", line 1387, in insert
    obj.dump(self._output)
  File "C:\Users\xzlg3r\bin\git-filter-repo", line 721, in dump
    change.dump(file_)
  File "C:\Users\xzlg3r\bin\git-filter-repo", line 618, in dump
    quoted_filename = PathQuoting.enquote(self.filename)
  File "C:\Users\xzlg3r\bin\git-filter-repo", line 191, in enquote
    if unquoted_string.startswith(b'"') or b'\n' in unquoted_string:
TypeError: startswith first arg must be str or a tuple of str, not bytes
fatal: stream ends early
fast-import: dumping crash report to .git/fast_import_crash_28148

Any help would be welcome, I am a bit over my head. I am pretty good at bash, C/C++, Java, C#, but just learning Python so reading the code I am making some kind of headway, but getting lost.

[newren]

A few issues here:

1. git hash-object will treat the argument to it as a file unless you pass the -t option. And if you pass the -t option with tree, you need to file represent a tree object, not an actual directory.
2. fast-import (which filter-repo uses to write the new history) doesn't accept trees/directories as inputs, meaning you cannot use them and instead have to list all the individual files. If the only file is .github/CODEOWNERS, you'd need to change your command to add that file. If you have multiple files within your .github directory, you'll need to have multiple calls to commit.file_changes.append, one for each file.
3. It looks like you had your $RELATIVE_TO_PROJECT_ROOT_PATHNAME and $FILENAME in mixed up places. git hash-object isn't going to be able to find the file without the path to it, and anything passed to FileChange needs to know the name of the file as it'll be recorded in the repository, not where your temporary file is relative to your project root write now.

Hope that helps.

[tbnorris]

Sorry for the long delay. You have helped me a ton with your response.
It is a little unfortunate I can't add a whole folder in one command, but I understand why.
I managed to get

$ RELATIVE_PATHNAME="../template_repo/.github/CODEOWNERS"
$ FILENAME=".github/CODEOWNERS"
$ git filter-repo --force --commit-callback "if not commit.parents: commit.file_changes.append(FileChange(b'M', b'$FILENAME', b'$(git hash-object -w "$RELATIVE_PATHNAME" --no-filters)', b'100644'))"

After understanding your response, and realizing what I needed to run git hash-object I started getting a hash object made that was giving me an message about the line endings being change from crlf to lf then the hash. So, I appended --no-filters which made the hash start working correctly. I then realized I needed to specify an encoding with the b'$FILENAME' for FileChange.

I am guessing from here, I can add two files by doing something close to:

$ RELATIVE_PATHNAME1="../template_repo/.github/CODEOWNERS"
$ RELATIVE_PATHNAME2="../template_repo/.github/pull_request_template.md"
$ FILENAME1=".github/CODEOWNERS"
$ FILENAME2=".github/pull_request_template.md"
$ git filter-repo --force --commit-callback "if not commit.parents: commit.file_changes.append(FileChange(b'M', b'$FILENAME1', b'$(git hash-object -w "$RELATIVE_PATHNAME1" --no-filters)', b'100644')) commit.file_changes.append(FileChange(b'M', b'$FILENAME2', b'$(git hash-object -w "$RELATIVE_PATHNAME2" --no-filters)', b'100644'))"

Or I will just have to call the second line twice once per file.