diff --git a/Gemfile b/Gemfile new file mode 100644 index 0000000..d5810e3 --- /dev/null +++ b/Gemfile @@ -0,0 +1,7 @@ +source 'https://rubygems.org' + +gem "jekyll", "~> 4.3.3" # installed by `gem jekyll` +# gem "webrick" # required when using Ruby >= 3 and Jekyll <= 4.2.2 + +gem "just-the-docs", "0.8.1" # pinned to the current release +# gem "just-the-docs" # always download the latest release diff --git a/README.md b/README.md index 317b894..8ad39d9 100644 --- a/README.md +++ b/README.md @@ -2,71 +2,16 @@ # Metafacture Documentation -Metafacture is a toolkit for processing semi-structured data with a focus on library metadata. It provides a versatile set of tools for reading, writing and transforming data. Metafacture can be used as a stand-alone application via CLI or as a Java library in other applications. There is also a playground where you can test workflows. +Metafacture is a toolkit for processing semi-structured data with a focus on library metadata +For the documentation see: http://metafacture.github.io/metafacture-documentation/ -[Have a look here for a quick intro into metafacture.](./MF-in-5-min.md) +This repo is the central place for the documentation about Metafacture that we render using the jekyll template [Just the docs](https://github.com/just-the-docs/just-the-docs). -This is the central place for the documentation about Metafacture. - -Metafacture comprises three main parts: **Framework**, **Flux** and one of the **Transformation-Modules Fix and Morph**. It can be extended with modules. +The content pages can be found [here](/docs/) > [!NOTE] > With regard to the Transformation-Modules this documentation focusses on Fix instead of MORPH. If you want to find out more about MORPH. Have a look at [the old documentation](https://github.com/metafacture/metafacture-core/wiki/Metamorph-User-Guide) and the german cookbook by [Swissbib](https://swissbib.gitlab.io/metamorph-doku/). -Our goal with this repo is to collaboratively create comprehensive documentation on Metafacture in the [issue tracker](https://github.com/culturegraph/metafacture-documentation/issues?q=). Feel free to open issues not only for bugs or enhancements, but also questions about Metafacture usage, or to share your experiences. We hope that over time, in that way we can create useful tutorials, how-tos, and collect good practices for using Metafacture. - -__________________ - -Deciding which parts are relevant to you depends on the way you are using Metafacture: - -## Using Metafacture via playground or CLI - -> [!NOTE] -> No Java-Code is necessary!!! - -While working with the playground or the command line you only need [Flux](#flux) and the transformation module [Fix](#fix). -Have a look here for [Getting started](/Getting-Started.md). - -## Framework for Java integration/development - -If you plan to use Metafacture as a Java library or if you wish to add commands to Flux. You should get familar with the [Framework](#framework). - -__________________ - -## FLUX - -Flux is a scripting language to easily build and run processing pipelines. No Java programming is necessary - it's used as a command line. To use Flux you may download the binary distribution of Metafacture. - -For more information on how to use Flux, see the [Flux User Guide](/Flux-User-Guide.md). - -See [here a list for all available FLUX-Commands](/flux-commands.md). - -__________________ - -## FIX - -Metafix is a domain specific language for metadata transformation based on Catmandu FIX. The FIX object performing the transformation is used as part of a processing pipeline. - -If you are using **Metafacture with CLI or Playground** and therefore the Flux scripting language to build and run pipelines, use the `fix` command in your FLUX-Pipeline. - -If you are using **Metafacture as a Java library**, just create a Metafix object and add it to your pipeline (see also the [Framework User Guide](#framework)). - -The transformation itself is declared in a fix-object which can be a file. For more information on how to declare transformations see [Metafix User Guide](/Fix-User-Guide.md). - -See [here a list for all available FIX functions and a cookbook for using fix](/Fix-function-and-Cookbook.md). - -> [!NOTE] -> PS: There is also the transformation modul MORPH. Have a look at[ the old documentation](https://github.com/metafacture/metafacture-core/wiki/Metamorph-User-Guide) and the german cookbook by [Swissbib](https://swissbib.gitlab.io/metamorph-doku/). - -__________________ - -## Framework - -> [!NOTE] ->Relevant for developers - -The framework includes the interfaces and abstract classes which form the foundation of the data processing pipelines. This part of Metafacture is only relevant for you if you plan to use Metafacture as a Java library or if you wish to add pipe elements to Flux. - -For more information see the [Framework User Guide](/Framework-User-Guide.md). +Our goal with this repo is to collaboratively create comprehensive documentation on Metafacture in the [issue tracker](https://github.com/metafacture/metafacture-documentation/issues?q=). Feel free to open issues not only for bugs or enhancements, but also questions about Metafacture usage, or to share your experiences. We hope that over time, in that way we can create useful tutorials, how-tos, and collect good practices for using Metafacture. diff --git a/_config.yml b/_config.yml new file mode 100644 index 0000000..40a01cd --- /dev/null +++ b/_config.yml @@ -0,0 +1,56 @@ +title: Metafacture Documentation +description: This is the central place for the documentation about Metafacture. +theme: just-the-docs + +url: https://metafacture.github.io/metafacture-documentation + +aux_links: + Metafacture Documentation on Github: https://github.com/metafacture/metafacture-documentation + +# External navigation links +nav_external_links: + - title: LICENSE + url: https://github.com/metafacture/metafacture-core/blob/master/LICENSE + +# Set a path/url to a logo that will be displayed instead of the title +logo: "/assets/images/metafacture_small.png" +favicon_ico: "/assets/images/favicon.ico" + +# Footer content +# appears at the bottom of every page's main content + +# Back to top link +back_to_top: true +back_to_top_text: "Back to top" + +footer_content: 'Metafacture Documentation is maintained by the Open infrastructure team of hbz.' + +# Footer last edited timestamp +last_edit_timestamp: true # show or hide edit time - page must have `last_modified_date` defined in the frontmatter +last_edit_time_format: "%b %e %Y at %I:%M %p" # uses ruby's time format: https://ruby-doc.org/stdlib-2.7.0/libdoc/time/rdoc/Time.html + +# Custom colors, see _sass/color_schemes/metafacture.scss +color_scheme: metafacture + +# see https://just-the-docs.github.io/just-the-docs/docs/search/#search-granularity +search: + heading_level: 6 + previews: 6 + +# Callouts, see https://just-the-docs.com/docs/configuration/#callouts +callouts_level: quiet # or loud +callouts: + highlight: + color: yellow + important: + title: Important + color: blue + new: + title: New + color: green + note: + title: Note + color: purple + warning: + title: Warning + color: red diff --git a/_sass/color_schemes/metafacture.scss b/_sass/color_schemes/metafacture.scss new file mode 100644 index 0000000..d0444d9 --- /dev/null +++ b/_sass/color_schemes/metafacture.scss @@ -0,0 +1 @@ +$link-color: #4183c4; \ No newline at end of file diff --git a/_sass/custom/custom.scss b/_sass/custom/custom.scss new file mode 100644 index 0000000..23ec7bb --- /dev/null +++ b/_sass/custom/custom.scss @@ -0,0 +1,4 @@ +h4 {font-size: 1rem !important;} +h5 { + font-size: 1.5rem !important; +} \ No newline at end of file diff --git a/assets/images/favicon.ico b/assets/images/favicon.ico new file mode 100644 index 0000000..4fd4115 Binary files /dev/null and b/assets/images/favicon.ico differ diff --git a/assets/images/metafacture_small.png b/assets/images/metafacture_small.png new file mode 100644 index 0000000..c0a990d Binary files /dev/null and b/assets/images/metafacture_small.png differ diff --git a/Approaching a transformation with metafacture.md b/docs/Approaching a transformation with metafacture.md similarity index 92% rename from Approaching a transformation with metafacture.md rename to docs/Approaching a transformation with metafacture.md index cbc326a..524ab4f 100644 --- a/Approaching a transformation with metafacture.md +++ b/docs/Approaching a transformation with metafacture.md @@ -1,3 +1,9 @@ +--- +layout: default +title: Approaching a transformation with metafacture +nav_order: 4 +--- + Every approach to transform metadata with metafacture is quite similiar: - You need to know the type and source of the input and the type and destination of the output: diff --git a/Documentation-Maintainer-Guide.md b/docs/Documentation-Maintainer-Guide.md similarity index 69% rename from Documentation-Maintainer-Guide.md rename to docs/Documentation-Maintainer-Guide.md index 735cd63..b17d62b 100644 --- a/Documentation-Maintainer-Guide.md +++ b/docs/Documentation-Maintainer-Guide.md @@ -1,7 +1,14 @@ +--- +layout: default +title: Maintainer Guide +nav_order: 8 +--- -## how to change flux-commands.md +# Maintainer Guide -The entries in flux-commands.md describe the usage of commands used by flux. +## how to change [docs/flux/flux-commands.md](https://github.com/metafacture/metafacture-documentation/blob/28-use-jekyll-theme/docs/flux/flux-commands.md) + +The entries in [docs/flux/flux-commands.md](https://github.com/metafacture/metafacture-documentation/blob/28-use-jekyll-theme/docs/flux/flux-commands.md) describe the usage of commands used by flux. flux-commands.md is fully automatically generated. To make this happen one has to fill in the proper annotations in the correponding java classes. E.g. @@ -30,7 +37,7 @@ The option's name is produced by cutting away the "set" from the methods name, l "BatchSize" which is then lowercased. The parameter of this option is generated from the parameter type of the method - here an "int"eger. -## how to publish flux-commands.md +## how to publish [docs/flux/flux-commands.md](https://github.com/metafacture/metafacture-documentation/blob/28-use-jekyll-theme/docs/flux/flux-commands.md) If you have updated some of these annotations, say "description", and these changes are merged into the master branch, generate a new flux-commands.md like this: @@ -43,6 +50,7 @@ $ flux.sh > flux-commands.md ``` Open the generated flux-commands.md and remove some boilerplate at the beginning of the -file manually. Save it, copy it here, commit and push. +file manually. Add the naviagtion part of the page, save it, copy it [here](https://github.com/metafacture/metafacture-documentation/blob/28-use-jekyll-theme/docs/flux/flux-commands.md), commit and push. + The [publishing process will be automated with an github action](https://github.com/metafacture/metafacture-core/issues/368). diff --git a/Getting-Started.md b/docs/Getting-Started.md similarity index 84% rename from Getting-Started.md rename to docs/Getting-Started.md index 6052620..466c94f 100644 --- a/Getting-Started.md +++ b/docs/Getting-Started.md @@ -1,5 +1,8 @@ -![logo](https://github.com/culturegraph/metafacture-core/wiki/img/metafacture_small.png) - +--- +layout: default +title: Getting Started +nav_order: 2 +--- # Getting started! @@ -7,11 +10,11 @@ The easiest way to get started with Metafacture is the Playground. Take a look at the [first example](https://metafacture.org/playground/?example=encode-xml) and run it by pressing the !["Process"](https://metafacture.org/img/process.png) button. Check out the other examples (first button, !["Load Examples"](https://metafacture.org/img/load-exmples.png)) for different input sources, transformations, and output formats. -For commands available in the Flux, see [the Flux commands documentation](/flux-commands.md). +For commands available in the Flux, see [the Flux commands documentation](flux/flux-commands.html). -For functions and usage of the Fix, see [the Fix functions and cookbook](/Fix-functions-and-cookbook). +For functions and usage of the Fix, see [the Fix functions and cookbook](fix/Fix-functions-and-Cookbook.html). -For next steps get familar with [FLUX](/Flux-User-Guide.md) and [FIX](/Fix-User-Guide.md). And try out some Metafacture workflows. +For next steps get familar with [FLUX](flux/Flux-User-Guide.html) and [FIX](fix/Fix-User-Guide.html). And try out some Metafacture workflows. ## Command line @@ -22,9 +25,9 @@ To use Metafacture as a command-line tool, download the latest metafix-runner fr To get started, you can export a workflow from the Playground (last button, !["Export Workflow"](https://metafacture.org/img/export.png)). -To set up IDE support for editing your Flux and Fix files, see [the IDE extensions page](https://metafacture.org/ide-extensions/index.html). +To set up IDE support for editing your Flux and Fix files, see [the IDE extensions page](https://metafacture.org/ide-extensions.html). -For next steps get familar with [FLUX](/Flux-User-Guide.md) and [FIX](/Fix-User-Guide.md). And try out some Metafacture workflows. +For next steps get familar with [FLUX](flux/Flux-User-Guide.html) and [FIX](fix/Fix-User-Guide.html). And try out some Metafacture workflows. ## Using Metafacture as a Java library @@ -55,4 +58,4 @@ To use Fix you would declare `metafix` instead of `metafacture-io` as in the exa Occasionally, we publish snapshot builds on [Sonatype OSS Repository](https://oss.sonatype.org/index.html#nexus-search;gav~org.metafacture~~~~~kw,versionexpand). The version number is derived from the branch name. Snapshot builds from the master branch always have the version `master-SNAPSHOT`. We also provide sometimes pre releases as github packages. -If you plan to use Metafacture as a Java library or if you wish to add commands to Flux you should get familar with the [Framework](/Framework-User-Guide.md). +If you plan to use Metafacture as a Java library or if you wish to add commands to Flux you should get familar with the [Framework](java-integration/Framework-User-Guide.html). diff --git a/MF-in-5-min.md b/docs/MF-in-5-min.md similarity index 94% rename from MF-in-5-min.md rename to docs/MF-in-5-min.md index 843171c..38c2e0d 100644 --- a/MF-in-5-min.md +++ b/docs/MF-in-5-min.md @@ -1,3 +1,9 @@ +--- +layout: default +title: 5 min Intro into MF +nav_order: 3 +--- + # Introduction - A quick 5 minute introduction to Metafacture ## HELLO WORLD @@ -87,7 +93,7 @@ $ cat yaml2json.flux ## FIX LANGUAGE -Many data conversions need a mapping from one field to another field plus optional conversions of the data inside these fields. Metafacture provides the transformation module `fix` that uses the Catmandu-inspired Fix language to assist in these mappings. A full list Fix functions is available at https://github.com/metafacture/metafacture-documentation/blob/master/Fix-function-and-Cookbook.md#functions. +Many data conversions need a mapping from one field to another field plus optional conversions of the data inside these fields. Metafacture provides the transformation module `fix` that uses the Catmandu-inspired Fix language to assist in these mappings. See the [full list of Fix functions](fix/Fix-functions-and-Cookbook.html#functions). Fixes can be provided inline as text argument of the fix module in the Flux script, or as a pointer to a Fix script. A Fix script groups one or more fixes in a file. @@ -173,6 +179,6 @@ The 245 field with its subfields of each MARC record is mapped to the title fiel The `retain` Fix function keeps only the title field in the output. (In contrast to Catmandu there are no special marc or pica fixes since the internal handling of records and elements is more generic. Also the internal serialization of MARC is not as complex as in Catmandu.) -Now you should be ready to [get started](./Getting-Started.md). +Now you should be ready to [get started](Getting-Started.html). -(Note: This mini introduction to Metafacture is inspired by the mini introduction to Catmandu here: https://metacpan.org/dist/Catmandu/view/lib/Catmandu/Introduction.pod) +(Note: This mini introduction to Metafacture is inspired by the mini [introduction to Catmandu](https://metacpan.org/dist/Catmandu/view/lib/Catmandu/Introduction.pod)) diff --git a/Fix-User-Guide.md b/docs/fix/Fix-User-Guide.md similarity index 97% rename from Fix-User-Guide.md rename to docs/fix/Fix-User-Guide.md index 415d52c..2c188e1 100644 --- a/Fix-User-Guide.md +++ b/docs/fix/Fix-User-Guide.md @@ -1,11 +1,16 @@ -![logo](https://github.com/culturegraph/metafacture-core/wiki/img/metafacture_small.png) +--- +layout: default +title: Fix User Guide +parent: Fix +nav_order: 1 +--- # Fix User Guide This document provides an introduction to the Metafacture Fix language (short: Metafix or Fix). The Fix language for Metafacture is introduced as an alternative to configuring data transformations with Metamorph. Inspired by Catmandu Fix, Metafix processes metadata not as a continuous data stream but as discrete records. ## Part of a metafacture worflow -Metafacture Fix is a transformation module that can be used in a [Flux Workflow](/Flux-User-Guide.md), for this you have to use this in your pipeline: +Metafacture Fix is a transformation module that can be used in a [Flux Workflow](../flux/Flux-User-Guide.html), for this you have to use this in your pipeline: Flux-Example: ```PERL @@ -85,7 +90,7 @@ do Bind(params,…) end ``` -Find here a [list of all functions, selectors, binds and conditionals](/Fix-function-and-Cookbook.md). +Find here a [list of all functions, selectors, binds and conditionals](Fix-functions-and-Cookbook.html). ## Addressing Pieces of Data: FIX-Path and the record structure in FIX diff --git a/Fix-function-and-Cookbook.md b/docs/fix/Fix-functions-and-Cookbook.md similarity index 73% rename from Fix-function-and-Cookbook.md rename to docs/fix/Fix-functions-and-Cookbook.md index 490246c..240fd49 100644 --- a/Fix-function-and-Cookbook.md +++ b/docs/fix/Fix-functions-and-Cookbook.md @@ -1,4 +1,11 @@ -This page is a replication of the passage of the Fix Readme.md. Status: [Fix Release 1.0.0](https://github.com/metafacture/metafacture-fix/releases/tag/1.0.0) +--- +layout: default +title: Fix Functions and Cookbook +parent: Fix +nav_order: 2 +--- + +This page is a replication of the passage [Fix Functions and Cookbook](https://github.com/metafacture/metafacture-fix?tab=readme-ov-file#functions-and-cookbook) of the Fix Readme.md. Status: [Fix Release 1.0.0](https://github.com/metafacture/metafacture-fix/releases/tag/1.0.0) ## Functions and cookbook @@ -58,6 +65,8 @@ Options: include(""[, ...]) ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+include+{") + ##### `nothing` Does nothing. It is used for benchmarking in Catmandu. @@ -68,6 +77,8 @@ nothing() [Example in Playground](https://metafacture.org/playground/?example=nothing) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+nothing+{") + ##### `put_filemap` Defines an external map for [lookup](#lookup) from a file or a URL. Maps with more than 2 columns are supported but are reduced to a defined key and a value column. @@ -95,6 +106,8 @@ Options: - `key_column`: Defines the column to be used for keys. Uses zero index. (Default: `0`) - `value_column`: Defines the column to be used for values. Uses zero index. (Default: `1`) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+put_filemap+{") + ##### `put_map` Defines an internal map for [lookup](#lookup) from key/value pairs. @@ -109,6 +122,8 @@ put_map("", [Example in Playground](https://metafacture.org/playground/?example=put_map) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+put_map+{") + ##### `put_rdfmap` Defines an external RDF map for lookup from a file or an HTTP(S) resource. @@ -122,6 +137,8 @@ put_rdfmap("", "", target: "", select_lang [Example in Playground](https://metafacture.org/playground/?example=put_rdfmap) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+put_rdfmap+{") + ##### `put_var` Defines a single global variable that can be referenced with `$[]`. @@ -132,6 +149,8 @@ put_var("", "") [Example in Playground](https://metafacture.org/playground/?example=put_var) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+put_var+{") + ##### `put_vars` Defines multiple global variables that can be referenced with `$[]`. @@ -145,6 +164,8 @@ put_vars( [Example in Playground](https://metafacture.org/playground/?example=put_vars) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+put_vars+{") + #### Record-level functions ##### `add_field` @@ -157,6 +178,8 @@ add_field("", "") [Example in Playground](https://metafacture.org/playground/?example=add_field) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+add_field+{") + ##### `array` Converts a hash/object into an array. @@ -172,6 +195,8 @@ array("foo") # {"name":"value"} => ["name", "value"] ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+array+{") + ##### `call_macro` Calls a named macro, i.e. a list of statements that have been previously defined with the [`do put_macro`](#do-put_macro) bind. @@ -193,6 +218,8 @@ call_macro(""[, ...]) [Example in Playground](https://metafacture.org/playground/?example=call_macro) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+call_macro+{") + ##### `copy_field` Copies a field from an existing field. @@ -203,6 +230,7 @@ copy_field("", "") [Example in Playground](https://metafacture.org/playground/?example=copy_field) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+copy_field+{") ##### `format` @@ -214,6 +242,8 @@ Replaces the value with a formatted (`sprintf`-like) version. format("", "") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+format+{") + ##### `hash` Converts an array into a hash/object. @@ -228,6 +258,8 @@ hash("foo") # ["name", "value"] => {"name":"value"} ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+hash+{") + ##### `move_field` Moves a field from an existing field. Can be used to rename a field. @@ -238,6 +270,7 @@ move_field("", "") [Example in Playground](https://metafacture.org/playground/?example=move_field) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+move_field+{") ##### `parse_text` @@ -249,6 +282,8 @@ Parses a text into an array or hash of values. parse_text("", "") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+parse_text+{") + ##### `paste` Joins multiple field values into a new field. Can be combined with additional literal strings. @@ -272,6 +307,8 @@ paste("my.string", "~Hi", "a", "~how are you?") [Example in Playground](https://metafacture.org/playground/?example=paste) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+paste+{") + ##### `print_record` Prints the current record as JSON either to standard output or to a file. @@ -305,6 +342,8 @@ print_record(destination: "record-%2$s.json", id: "001", pretty: "true") print_record(destination: "record-%03d.json.gz", header: "After transformation: ") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+print_record+{") + ##### `random` Creates (or replaces) a field with a random number (less than the specified maximum). @@ -313,6 +352,8 @@ Creates (or replaces) a field with a random number (less than the specified maxi random("", "") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+random+{") + ##### `remove_field` Removes a field. @@ -323,6 +364,8 @@ remove_field("") [Example in Playground](https://metafacture.org/playground/?example=remove_field) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+remove_field+{") + ##### `rename` Replaces a regular expression pattern in subfield names of a field. Does not change the name of the source field itself. @@ -333,6 +376,8 @@ rename("", "", "") [Example in Playground](https://metafacture.org/playground/?example=rename) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+rename+{") + ##### `retain` Deletes all fields except the ones listed (incl. subfields). @@ -343,6 +388,8 @@ retain(""[, ...]) [Example in Playground](https://metafacture.org/playground/?example=retain) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+retain+{") + ##### `set_array` Creates a new array (with optional values). @@ -354,6 +401,8 @@ set_array("", ""[, ...]) [Example in Playground](https://metafacture.org/playground/?example=set_array) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+set_array+{") + ##### `set_field` Creates (or replaces) a field with a defined value. @@ -362,6 +411,8 @@ Creates (or replaces) a field with a defined value. set_field("", "") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+set_field+{") + ##### `set_hash` Creates a new hash (with optional values). @@ -371,6 +422,8 @@ set_hash("") set_hash("", "subfieldName": ""[, ...]) ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+set_hash+{") + ##### `timestamp` Creates (or replaces) a field with the current timestamp. @@ -385,6 +438,8 @@ Options: timestamp(""[, format: ""][, timezone: ""][, language: ""]) ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+timestamp+{") + ##### `vacuum` Deletes empty fields, arrays and objects. @@ -395,6 +450,8 @@ vacuum() [Example in Playground](https://metafacture.org/playground/?example=vacuum) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+vacuum+{") + #### Field-level functions ##### `append` @@ -407,6 +464,8 @@ append("", "") [Example in Playground](https://metafacture.org/playground/?example=append) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+append+{") + ##### `capitalize` Upcases the first character in a field value. @@ -417,6 +476,8 @@ capitalize("") [Example in Playground](https://metafacture.org/playground/?example=capitalize) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+capitalize+{") + ##### `count` Counts the number of elements in an array or a hash and replaces the field value with this number. @@ -425,6 +486,8 @@ Counts the number of elements in an array or a hash and replaces the field value count("") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+count+{") + ##### `downcase` Downcases all characters in a field value. @@ -435,6 +498,8 @@ downcase("") [Example in Playground](https://metafacture.org/playground/?example=downcase) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+downcase+{") + ##### `filter` Only keeps field values that match the regular expression pattern. Works only with array of strings/repeated fields. @@ -443,6 +508,8 @@ Only keeps field values that match the regular expression pattern. Works only wi filter("", "") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+filter+{") + ##### `flatten` Flattens a nested array field. @@ -453,6 +520,8 @@ flatten("") [Example in Playground](https://metafacture.org/playground/?example=flatten) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+flatten+{") + ##### `from_json` Replaces the string with its JSON deserialization. @@ -465,6 +534,8 @@ Options: from_json(""[, error_string: ""]) ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+from_json+{") + ##### `index` Returns the index position of a substring in a field and replaces the field value with this number. @@ -473,6 +544,8 @@ Returns the index position of a substring in a field and replaces the field valu index("", "") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+index+{") + ##### `isbn` Extracts an ISBN and replaces the field value with the normalized ISBN; optionally converts and/or validates the ISBN. @@ -489,6 +562,8 @@ isbn(""[, to: ""][, verify_check_digit: ""][, [Example in Playground](https://metafacture.org/playground/?example=isbn) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+isbn+{") + ##### `join_field` Joins an array of strings into a single string. @@ -499,6 +574,8 @@ join_field("", "") [Example in Playground](https://metafacture.org/playground/?example=join_field) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+join_field+{") + ##### `lookup` Looks up matching values in a map and replaces the field value with this match. [External files](#put_filemap), [internal maps](#put_map) as well as [RDF resources](#put_rdfmap) can be used. @@ -558,6 +635,8 @@ lookup("path.to.field", "map-name", __default: "NA") lookup("path.to.field", "map-name", print_unknown: "true", destination: "unknown.txt") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+lookup+{") + ##### `prepend` Adds a string at the beginning of a field value. @@ -568,6 +647,8 @@ prepend("", "") [Example in Playground](https://metafacture.org/playground/?example=prepend) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+prepend+{") + ##### `replace_all` Replaces a regular expression pattern in field values with a replacement string. Regexp capturing is possible; refer to capturing groups by number (`$`) or name (`${}`). @@ -578,6 +659,8 @@ replace_all("", "", "") [Example in Playground](https://metafacture.org/playground/?example=replace_all) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+replace_all+{") + ##### `reverse` Reverses the character order of a string or the element order of an array. @@ -586,6 +669,8 @@ Reverses the character order of a string or the element order of an array. reverse("") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+reverse+{") + ##### `sort_field` Sorts strings in an array. Alphabetically and A-Z by default. Optional numerical and reverse sorting. @@ -598,6 +683,8 @@ sort_field("", numeric: "true") [Example in Playground](https://metafacture.org/playground/?example=sort_field) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+sort_field+{") + ##### `split_field` Splits a string into an array and replaces the field value with this array. @@ -608,6 +695,8 @@ split_field("", "") [Example in Playground](https://metafacture.org/playground/?example=split_field) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+split_field+{") + ##### `substring` Replaces a string with its substring as defined by the start position (offset) and length. @@ -616,6 +705,8 @@ Replaces a string with its substring as defined by the start position (offset) a substring("", "", "") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+substring+{") + ##### `sum` Sums numbers in an array and replaces the field value with this number. @@ -624,6 +715,8 @@ Sums numbers in an array and replaces the field value with this number. sum("") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+sum+{") + ##### `to_json` Replaces the value with its JSON serialization. @@ -637,6 +730,8 @@ Options: to_json(""[, pretty: ""][, error_string: ""]) ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+to_json+{") + ##### `trim` Deletes whitespace at the beginning and the end of a field value. @@ -647,6 +742,8 @@ trim("") [Example in Playground](https://metafacture.org/playground/?example=trim) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+trim+{") + ##### `uniq` Deletes duplicate values in an array. @@ -657,6 +754,7 @@ uniq("") [Example in Playground](https://metafacture.org/playground/?example=uniq) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+uniq+{") ##### `upcase` @@ -668,6 +766,8 @@ upcase("") [Example in Playground](https://metafacture.org/playground/?example=upcase) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+upcase+{") + ##### `uri_encode` Encodes a field value as URI. Aka percent-encoding. @@ -687,6 +787,8 @@ E.g.: uri_encode("path.to.field", plus_for_space:"false", safe_chars:"") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+uri_encode+{") + ### Selectors #### `reject` @@ -701,12 +803,16 @@ end [Example in Playground](https://metafacture.org/playground/?example=reject) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixMethod.java+"+reject+{") + ### Binds #### `do list` Iterates over each element of an array. In contrast to Catmandu, it can also iterate over a single object or string. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixBind.java+"+list+{") + ```perl do list(path: "") ... @@ -731,6 +837,8 @@ end Iterates over each _named_ element of an array (like [`do list`](#do-list) with a variable name). If multiple arrays are given, iterates over the _corresponding_ elements from each array (i.e., all elements with the same array index, skipping elements whose arrays have already been exhausted). +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixBind.java+"+list_as+{") + ```perl do list_as(element_1: ""[, ...]) ... @@ -762,6 +870,8 @@ end [Example in Playground](https://metafacture.org/playground/?example=do_once) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixBind.java+"+once+{") + In order to execute multiple blocks only once, tag them with unique identifiers: ```perl @@ -800,6 +910,8 @@ call_macro(""[, ...]) [Example in Playground](https://metafacture.org/playground/?example=do_put_macro) +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixBind.java+"+put_macro+{") + ### Conditionals Conditionals start with `if` in case of affirming the condition or `unless` rejecting the condition. @@ -836,36 +948,52 @@ end Executes the functions if/unless the field contains the value. If it is an array or a hash all field values must contain the string. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+all_contain+{") + ##### `any_contain` Executes the functions if/unless the field contains the value. If it is an array or a hash one or more field values must contain the string. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+any_contain+{") + ##### `none_contain` Executes the functions if/unless the field does not contain the value. If it is an array or a hash none of the field values may contain the string. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+none_contain+{") + ##### `str_contain` Executes the functions if/unless the first string contains the second string. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+str_contain+{") + #### `equal` ##### `all_equal` Executes the functions if/unless the field value equals the string. If it is an array or a hash all field values must equal the string. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+all_equal+{") + ##### `any_equal` Executes the functions if/unless the field value equals the string. If it is an array or a hash one or more field values must equal the string. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+any_equal+{") + ##### `none_equal` Executes the functions if/unless the field value does not equal the string. If it is an array or a hash none of the field values may equal the string. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+none_equal+{") + ##### `str_equal` Executes the functions if/unless the first string equals the second string. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+str_equal+{") + #### `exists` Executes the functions if/unless the field exists. @@ -873,6 +1001,7 @@ Executes the functions if/unless the field exists. ```perl if exists("") ``` +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+exists+{") #### `in` @@ -880,6 +1009,8 @@ Executes the functions if/unless the field value [is contained in](https://perld _Also aliased as [`is_contained_in`](#is_contained_in)._ +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+in+{") + #### `is_contained_in` _Alias for [`in`](#in)._ @@ -888,22 +1019,32 @@ _Alias for [`in`](#in)._ Executes the functions if/unless the field value is an array. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+is_array+{") + #### `is_empty` Executes the functions if/unless the field value is empty. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+is_empty+{") + #### `is_false` Executes the functions if/unless the field value equals `false` or `0`. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+is_false+{") + #### `is_hash` _Alias for [`is_object`](#is_object)._ +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+is_hash+{") + #### `is_number` Executes the functions if/unless the field value is a number. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+is_number+{") + #### `is_object` Executes the functions if/unless the field value is a hash (object). @@ -914,28 +1055,37 @@ _Also aliased as [`is_hash`](#is_hash)._ Executes the functions if/unless the field value is a string (and not a number). +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+is_string+{") + #### `is_true` Executes the functions if/unless the field value equals `true` or `1`. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+is_true+{") + + #### `match` ##### `all_match` Executes the functions if/unless the field value matches the regular expression pattern. If it is an array or a hash all field values must match the regular expression pattern. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+all_match+{") + ##### `any_match` Executes the functions if/unless the field value matches the regular expression pattern. If it is an array or a hash one or more field values must match the regular expression pattern. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+any_match+{") + ##### `none_match` Executes the functions if/unless the field value does not match the regular expression pattern. If it is an array or a hash none of the field values may match the regular expression pattern. +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+none_match+{") + ##### `str_match` Executes the functions if/unless the string matches the regular expression pattern. -## Xtext - -This repo has been originally set up with [Xtext](https://www.eclipse.org/Xtext/) 2.17.0 and Eclipse for Java 2019-03, following [https://www.eclipse.org/Xtext/documentation/104_jvmdomainmodel.html](https://www.eclipse.org/Xtext/documentation/104_jvmdomainmodel.html). +[Java Code](https://github.com/search?type=code&q=repo:metafacture/metafacture-fix+path:FixConditional.java+"+str_match+{") diff --git a/docs/fix/fix.md b/docs/fix/fix.md new file mode 100644 index 0000000..2932393 --- /dev/null +++ b/docs/fix/fix.md @@ -0,0 +1,19 @@ +--- +layout: default +title: Fix +nav_order: 6 +has_children: true +--- + +Metafix is a domain specific language for metadata transformation based on Catmandu FIX. The FIX object performing the transformation is used as part of a processing pipeline. + +If you are using **Metafacture with CLI or Playground** and therefore the Flux scripting language to build and run pipelines, use the `fix` command in your FLUX-Pipeline. + +If you are using **Metafacture as a Java library**, create a Metafix object and add it to your pipeline (see also the [Framework User Guide](../java-integration/Framework-User-Guide.html)). + +The transformation itself is declared in a fix-object which can be a file. For more information on how to declare transformations see [Metafix User Guide](Fix-User-Guide.html). + +See [here a list for all available FIX functions and a cookbook for using fix](Fix-functions-and-Cookbook.html). + +{: .note } +PS: There is also the transformation module MORPH. Have a look at [the old documentation](https://github.com/metafacture/metafacture-core/wiki/Metamorph-User-Guide) and the German cookbook by [Swissbib](https://swissbib.gitlab.io/metamorph-doku/). \ No newline at end of file diff --git a/Flux-User-Guide.md b/docs/flux/Flux-User-Guide.md similarity index 86% rename from Flux-User-Guide.md rename to docs/flux/Flux-User-Guide.md index eb13f3f..e681a3b 100644 --- a/Flux-User-Guide.md +++ b/docs/flux/Flux-User-Guide.md @@ -1,4 +1,9 @@ -![logo](https://github.com/culturegraph/metafacture-core/wiki/img/metafacture_small.png) +--- +layout: default +title: Flux User Guide +parent: Flux +nav_order: 1 +--- # Flux User Guide @@ -46,6 +51,7 @@ This sets the variable `var1` to the value 'value1' and `var2` to the value 'val ## Writing Flux files The following snippet shows a simple flux file: + ```c //declare variables default file = FLUX_DIR + "10.marc21"; @@ -60,8 +66,9 @@ file | write("stdout") ; ``` + In the first section [variables](#variables) are declared, in the second, we [define the flow](#flow-definitions). -A flow is a combination of different [FLUX commands. Here is a list to all available Flux-Commands.](https://github.com/metafacture/metafacture-documentation/blob/master/flux-commands.md) +A flow is a combination of different FLUX commands. [Here is a list of all available Flux-Commands.](flux-commands.html) Linebreaks are optional, but help concerning readability. One can add comments with `//`. Semicolons `;` mark the end of a variable assignment or flow definition. @@ -85,7 +92,7 @@ The syntax for defining flows takes its cues from bash pipes. Commands are conca Some commands take a constructor argument. It is provided within brackets: `command("arg")`. Furthermore, some commands have named options. These are set as follows `command(optionname="arg1",annotheroption="arg2")` or with constructor argument: `command("arg",option="arg2")`. -To learn about the available options of a command, execute Flux without arguments - it will list all available commands, including options. Or simply have a look at the [list of available FLUX commands.](https://github.com/metafacture/metafacture-documentation/blob/master/flux-commands.md) +To learn about the available options of a command, execute Flux without arguments - it will list all available commands, including options. Or simply have a look at the [list of available FLUX commands.](flux-commands.html) To some commands the entire environment can be given as an argument. This is done with the `*` character: `fix("tranformation.fix", *)`. In this case Metafix gains access to all variable assignments made in Flux. @@ -93,7 +100,7 @@ To some commands the entire environment can be given as an argument. This is don Note that unlike shell pipes, the data flowing between Flux commands is _typed_. This means that only commands with matching signatures can be combined. Commands expect a certain input and provide a certain output like: `StreamReceiver, `Object`, `Reader` and others. -To lookup the signatures, again: execute Flux without arguments or see: [[Metafix-User-Guide#parameters-to-metafix-definitions]]). It will list all available commands, including signatures. Or simply have a look at the [list of available FLUX commands.](https://github.com/metafacture/metafacture-documentation/blob/master/flux-commands.md) +To lookup the signatures, again: execute Flux without arguments or see: [[Metafix-User-Guide#parameters-to-metafix-definitions]]). It will list all available commands, including signatures. Or simply have a look at the [list of available FLUX commands.](flux-commands.html) ### Variables Variables are always Strings and can be concatenated with the `+` operator. Escape sequences follow the Java String conventions: `\n`=line break, `\t`=tab, `\\`=\, `\u0024`=unicode character, etc. @@ -108,16 +115,16 @@ Flux supports single line C/Java-style comments: `//comment`. ## Overview of the commands and some examples -1. Have a look at the [List of available FLUX commands](https://github.com/metafacture/metafacture-documentation/blob/master/flux-commands.md) or execute the flux without arguments to get a short help text along with a list of all registered commands. This is the list of FLUX commands mentioned already above. -2. There are several example flux files along with sample data in the folder `examples/`: https://github.com/metafacture/metafacture-core/tree/master/metafacture-runner/src/main/dist/examples +1. Have a look at the [list of available FLUX commands](flux-commands.html) or execute the flux without arguments to get a short help text along with a list of all registered commands. This is the list of FLUX commands mentioned already above. +2. There are several example flux files along with sample data in the repo folder `examples/`: [https://github.com/metafacture/metafacture-core/tree/master/metafacture-runner/src/main/dist/examples](https://github.com/metafacture/metafacture-core/tree/master/metafacture-runner/src/main/dist/examples) _________________________ -# For developers: +## For developers: -> [!NOTE] -> Coding in JAVA. +{: .note } +Coding in JAVA. -## Adding new Commands +### Adding new Commands Add your class and a descriptive flux shortcut to `flux-commands.properties`. This file acts as a lookup table for flux commands. Use the proper file, i.e. the one residing in the same module where your newly created class resides. If you have e.g. created a class in the module `metafacture-biblio`, you add the flux-command to https://github.com/metafacture/metafacture-core/blob/master/metafacture-biblio/src/main/resources/flux-commands.properties. Recompile. That's all to add a command. diff --git a/flux-commands.md b/docs/flux/flux-commands.md similarity index 99% rename from flux-commands.md rename to docs/flux/flux-commands.md index 1e52834..66f26e5 100644 --- a/flux-commands.md +++ b/docs/flux/flux-commands.md @@ -1,3 +1,10 @@ +--- +layout: default +title: Flux Commands +parent: Flux +nav_order: 2 +--- + Available flux commands ======================= diff --git a/docs/flux/flux.md b/docs/flux/flux.md new file mode 100644 index 0000000..9e5993e --- /dev/null +++ b/docs/flux/flux.md @@ -0,0 +1,14 @@ +--- +layout: default +title: Flux +nav_order: 5 +has_children: true +--- + +# FLUX + +Flux is a scripting language to easily build and run processing pipelines. No Java programming is necessary - it's used on the command line. To use Flux you may download the binary distribution of Metafacture. + +For more information on how to use Flux, see the [Flux User Guide](Flux-User-Guide.html). + +See [here a list for all available FLUX-Commands](flux-commands.html). \ No newline at end of file diff --git a/Framework-User-Guide.md b/docs/java-integration/Framework-User-Guide.md similarity index 92% rename from Framework-User-Guide.md rename to docs/java-integration/Framework-User-Guide.md index 7c06e5a..18b8102 100644 --- a/Framework-User-Guide.md +++ b/docs/java-integration/Framework-User-Guide.md @@ -1,9 +1,14 @@ -![logo](https://github.com/culturegraph/metafacture-core/wiki/img/metafacture_small.png) +--- +layout: default +title: Framework User Guide +parent: Java Integration +nav_order: 1 +--- # Framework User Guide -> [!NOTE] ->Relevant for Java developers. For using metafacture without Java Code see the [FLUX user guide](/Flux-User-Guide.md). +{: .note } +Relevant for Java developers. For using metafacture without Java Code see the [FLUX user guide](../flux/Flux-User-Guide.html). This page explains how to create a Metafacture objects and how to assemble them to form a processing pipeline. We use as an example a simple pipeline containing a Metafix instance. diff --git a/docs/java-integration/java-integration.md b/docs/java-integration/java-integration.md new file mode 100644 index 0000000..a98103c --- /dev/null +++ b/docs/java-integration/java-integration.md @@ -0,0 +1,15 @@ +--- +layout: default +title: Java Integration +nav_order: 7 +has_children: true +--- + +## Framework for Java integration/development + +{: .note } +Relevant for developers + +The framework includes the interfaces and abstract classes which form the foundation of the data processing pipelines. This part of Metafacture is only relevant for you if you plan to use Metafacture as a Java library or if you wish to add pipe elements to Flux. + +For more information see the [Framework User Guide](Framework-User-Guide.html). \ No newline at end of file diff --git a/index.md b/index.md new file mode 100644 index 0000000..91855cf --- /dev/null +++ b/index.md @@ -0,0 +1,40 @@ +--- +layout: default +title: Home +nav_order: 1 +description: "This is the central place for the documentation about Metafacture." +permalink: / +--- + +# Metafacture Documentation + +Metafacture is a toolkit for processing semi-structured data with a focus on library metadata. It provides a versatile set of tools for reading, writing and transforming data. Metafacture can be used as a stand-alone application via CLI or as a Java library in other applications. There is also a playground where you can test workflows. + +[Have a look here for a quick intro into metafacture.](./docs/MF-in-5-min.html) + +This is the central place for the documentation about Metafacture. + +Metafacture comprises three main parts: **Framework**, **Flux** and one of the **Transformation-Modules Fix and Morph**. It can be extended with modules. + +{: .note } +With regard to the Transformation-Modules this documentation focusses on Fix instead of MORPH. If you want to find out more about MORPH. Have a look at [the old documentation](https://github.com/metafacture/metafacture-core/wiki/Metamorph-User-Guide) and the german cookbook by [Swissbib](https://swissbib.gitlab.io/metamorph-doku/). + + +Our goal with this repo is to collaboratively create comprehensive documentation on Metafacture in the [issue tracker](https://github.com/metafacture/metafacture-documentation/issues?q=). Feel free to open issues not only for bugs or enhancements, but also questions about Metafacture usage, or to share your experiences. We hope that over time, in that way we can create useful tutorials, how-tos, and collect good practices for using Metafacture. + +__________________ + +Deciding which parts are relevant to you depends on the way you are using Metafacture: + +## Using Metafacture via playground or CLI + +{: .note } +No Java-Code is necessary!!! + +While working with the playground or the command line you only need [Flux](docs/flux/flux.html) and the transformation module [Fix](docs/fix/fix.html). +Have a look here for [Getting started](docs/Getting-Started.html). + + +## Framework for Java integration/development + +If you plan to use Metafacture as a Java library or if you wish to add commands to Flux. You should get familar with the [Framework](docs/java-integration/Framework-User-Guide.html).