Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Autogenerate Telegraf docs from influxdata/telegraf repo. #5656

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
.DS_Store
*~
external
public
.*.swp
node_modules
Expand All @@ -14,4 +15,5 @@ node_modules
.idea
**/config.toml
package-lock.json
tmp
tmp
content/telegraf/v1/input-plugins
1 change: 1 addition & 0 deletions build-telegraf/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Build Telegraf documentation from https://github.com/influxdata/telegraf
112 changes: 112 additions & 0 deletions build-telegraf/plugins.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
#!/bin/bash

# Determine the root project directory
ROOT_DIR=$(git rev-parse --show-toplevel)

# Clone an external repository
git clone https://github.com/influxdata/telegraf.git "$ROOT_DIR/external/telegraf"

# Update the external repository
cd "$ROOT_DIR/external/telegraf"
git pull origin master

# Function to create a Hugo page
create_page() {
local plugin_dir=$1
local dest_dir=$2
local dest_file="$dest_dir/_index.md"
local parent=$3

# Check if the plugin directory exists
if [ -d "$plugin_dir" ]; then
mkdir -p "$dest_dir"
# Use a heredoc to write the frontmatter to the destination file
cat <<EOF > "$dest_file"
---
description: "Telegraf plugin for collecting metrics from $plugin_dir"
menu:
telegraf_v1_ref:
parent: $parent
name: $(basename "$plugin_dir")
tags: [$(basename "$plugin_dir")]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing that would be nice to account for here is introduced and deprecated versions. Were you thinking we'd do that by injecting frontmatter or adding that frontmatter to the readmes?

---

EOF
else
echo "Plugin directory not found: $plugin_dir"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We currently have 20 external plugins that are listed on the plugins page but are not part of the Telegraf repo. How do we want to handle these? @srebhan, do you have any thoughts?

fi
}

# Function to copy README.md content to _index.md
copy_readme() {
local plugin_dir=$1
local src_readme="$plugin_dir/README.md"
local dest_dir=$2
local dest_file="$dest_dir/_index.md"

# If src_readme doesn't exist, then return an error
if [ ! -f "$src_readme" ]; then
echo "README.md not found in $plugin_dir"
return 1
fi
# If $dest_file doesn't exist, then return an error
if [ ! -f "$dest_file" ]; then
echo "$dest_file not found"
return 1
fi

# Copy the README.md content to _index.md
cat $src_readme >> $dest_file
echo "Copied $src_readme to $dest_file"
}

# Semaphore function to limit concurrent processes
semaphore() {
local max_concurrent=$1
while [ "$(jobs -r | wc -l)" -ge "$max_concurrent" ]; do
sleep 0.1
done
}

# Limit the number of concurrent processes
MAX_CONCURRENT=10

build() {
# For each plugin in external/telegraf/plugins/inputs,
# copy the README.md file to content.
local input_plugins_dir="$ROOT_DIR/content/telegraf/v1/input-plugins"
cat <<EOF > "$input_plugins_dir/_index.md"
---
title: "Telegraf Input Plugins"
description: "Telegraf input plugins collect metrics from the system, services, and third-party APIs."
menu:
telegraf_v1_ref:
name: Input Plugins
identifier: input-plugins
weight: 10
tags: [input-plugins]
---

EOF

for plugin_dir in "$ROOT_DIR/external/telegraf/plugins/inputs"/*; do
if [ -d "$plugin_dir" ]; then
# If the directory name is "all", then skip it
if [ "$(basename "$plugin_dir")" == "all" ]; then
continue
fi
local plugin_name=$(basename "$plugin_dir")
local dest_dir=$input_plugins_dir/$plugin_name
semaphore "$MAX_CONCURRENT"
(
create_page "$plugin_dir" "$dest_dir" "Input Plugins"
copy_readme "$plugin_dir" "$dest_dir"
) &
fi
done

# Wait for all background jobs to finish
wait
}

build
1 change: 1 addition & 0 deletions layouts/shortcodes/telegraf/plugins.html
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,6 @@ <h3 id="{{ $type }}-{{ .id }}">{{ .name }}</h3>
<a class="btn github-link" href="{{ if .link }}{{ .link }}{{ else }}https://github.com/influxdata/telegraf/blob/release-{{ $latestTelegrafVersion }}/plugins/{{ $type }}s/{{ .id }}/README.md{{ end }}" target="_blank">
<span class="icon-github"></span> <span class="hide">View</span>
</a>
<div>{{ .Site.GetPage "/telegraf/v1/input-plugins/apache/_index.md" }}</div>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume you'll iterate over the page data to do something here?

</div>
{{ end }}