Skip to content

Commit

Permalink
Implement data sanitation workaround
Browse files Browse the repository at this point in the history
Implement a workaround in dealing with potentially unsanitary response
data with NetBox.

* Add a list of confirmed troublesome structs
* Add checks at struct generation and mark all fields of unsanitary
structs as Option
  • Loading branch information
ByteOtter committed Sep 5, 2024
1 parent 8178cee commit 99c9f38
Show file tree
Hide file tree
Showing 6 changed files with 88 additions and 7 deletions.
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,3 +121,11 @@ thanix $YOUR_API_YAML --output thanix_client/
schema file your want to generate a client for.
- The `--output` parameter is optional and refers to the path where thanix' output should be put. If omitted, it will
create a `output` directory in your current wokring directory.
- The `--workaround` flag can be set to allow Thanix to create a **strongly opinionated** version of `thanix_client`. This is
primarily used to avoid serialization errors when handling API object responses which we have confirmed to diverge from the
values expected according to the schema.

> [!Note]
> The `--workaround` flag is only useful when creating a client for [`NetBox`](https://netbox.dev). In other cases it might produce
> a broken or unsafe API client by weakening response data validation.
6 changes: 4 additions & 2 deletions src/bindgen.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
//! Generate bindings, parse YAML and create output files.
use crate::pathgen;
use crate::structgen;
use openapiv3::Schema;
Expand All @@ -11,7 +13,7 @@ use std::{
};

/// Generate Rust bindings from an OpenAPI schema.
pub fn gen(input_path: impl AsRef<Path>, output_path: impl AsRef<Path>) {
pub fn gen(input_path: impl AsRef<Path>, output_path: impl AsRef<Path>, workaround_mode: bool) {
// Parse the schema.
let input = fs::read_to_string(input_path).unwrap();
let api: OpenAPI = serde_yaml::from_str(&input).unwrap();
Expand All @@ -32,7 +34,7 @@ pub fn gen(input_path: impl AsRef<Path>, output_path: impl AsRef<Path>) {
_ => continue,
};
// Generate struct and write it to file.
if let Some(structure) = structgen::gen(name, s) {
if let Some(structure) = structgen::gen(name, s, workaround_mode) {
types_file.write_all(structure.as_bytes()).unwrap();
}
}
Expand Down
9 changes: 8 additions & 1 deletion src/main.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
mod bindgen;
mod pathgen;
mod structgen;
mod util;

use std::path::PathBuf;

Expand All @@ -14,6 +15,12 @@ struct Args {
output: PathBuf,
/// Path to a YAML schema file.
input: Option<String>,
/// Enable Workaround mode.
/// Creates opinionated NetBox API client.
/// Can help with unsanitary response data crashing deserialization by making API object fields optional, even though
/// the YAML might state otherwise.
#[arg(short, long, action = clap::ArgAction::SetTrue)]
workaround: bool,
}

fn main() {
Expand All @@ -28,7 +35,7 @@ fn main() {
);

match args.input {
Some(file) => bindgen::gen(file, args.output),
Some(file) => bindgen::gen(file, args.output, args.workaround),
None => println!("Error: You need to provide a YAML schema to generate from."),
}
}
4 changes: 3 additions & 1 deletion src/pathgen.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
//! Generate API request functions.
use crate::bindgen::{self, make_comment};
use check_keyword::CheckKeyword;
use convert_case::{Case, Casing};
Expand Down Expand Up @@ -240,7 +242,7 @@ fn gen_fn(name: &str, op_type: &str, op: &Operation) -> String {
result += &fn_response_name;
result += "::Other(r#response)) }\n\t}\n}\n";

return result;
result
}

fn make_fn_name_from_path(input: &str) -> String {
Expand Down
44 changes: 41 additions & 3 deletions src/structgen.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,33 @@
use crate::bindgen;
//! Generate structs from API objects.
use crate::{bindgen, util::is_unsanitary};
use check_keyword::CheckKeyword;
use openapiv3::{ReferenceOr, Schema, SchemaKind, Type};

pub fn gen(name: &str, schema: &Schema) -> Option<String> {
/// Generate the structs to be used as API request payloads.
///
/// If `workaround_mode` is enabled, will check if the current struct matches with the names listed
/// in `unsanitary_data` and make all fields of these structs optional.
/// This can help when normally generated API clients crash with serialization issues due to
/// NetBox's response data having some fileds set to `null`, despite the YAML stating that they are
/// not nullable.
///
/// > [!Note]
/// > The workaround mentioned above is *only* valid and useful when creating an API client with
/// NetBox.
/// > Using the `--workaround` flag with any other use case is **not advised** because it weakens
/// data validation.
///
/// # Parameters
///
/// * `name: &str` - The name of the struct to generate.
/// * `schema: &Schema` - The schema this struct follows.
/// * `workaround_mode: bool` - Whether `--workaround` flag has been set or not.
///
/// # Returns
///
/// * `Option<String>` - The string represnetation of the given struct.
pub fn gen(name: &str, schema: &Schema, workaround_mode: bool) -> Option<String> {
let typ = match &schema.schema_kind {
SchemaKind::Type(x) => x,
_ => return None,
Expand Down Expand Up @@ -34,7 +59,20 @@ pub fn gen(name: &str, schema: &Schema) -> Option<String> {
result += "\t";
result += &format!("pub {}", &prop_name.clone().into_safe());
result += ": ";
result += &type_name;

// HACK
// Turn all fields in a Response struct (except te id) into an Option to prevent unsanitary
// response data from crashing serialization.
if workaround_mode {
if is_unsanitary(name)
&& !type_name.contains("Option<")
&& !result.ends_with("\tpub id:")
{
result += &format!("Option<{}>", type_name);
}
} else {
result += &type_name;
}
result += ",\n";
}
result += "}\n";
Expand Down
24 changes: 24 additions & 0 deletions src/util.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
//! Various supporting functionality.
/// Names of NetBox API objects which can cause the client to crash at response data serialization.
///
/// This workaround is necessary as a lot of the time, some database entries can have fieds set to
/// `null` even though the API schema states that they are not nullable.
/// This leads to problems when creating an API client crate just by using the YAML schema as the
/// client then expects all response data to be correct, otherwise `serde` cannot build the
/// required structs.
///
/// To work around this, the `--workaround` flag was added to Thanix, which will check at struct
/// generation, whether the struct is part of this **manually maintained list of troublemakers**
///
/// > [!Note]
/// > This list is maintained manually by the Nazara Team, as there is currently no real way to
/// automate this.
/// > If you have problems and need something to be added to it, please open a bug in our [issues
/// section](https://github.com/The-Nazara-Project/Thanix/issues/).
static UNSANITARY_OBJECTS: &[&str] = &["interface"];

/// Check if a given struct's name contains any entry from the `UNSANITARY_OBJECTS` list.
pub fn is_unsanitary(name: &str) -> bool {
UNSANITARY_OBJECTS.iter().any(|&word| name.contains(word))
}

0 comments on commit 99c9f38

Please sign in to comment.