Releases: metafacture/metafacture-core
Metafacture Core 4.0.0
This is the last release of metafacture-core as a single Maven artifact. The release concentrates on reorganising the sources to prepare for the split-up of metafacture-core in the next release.
Important: This release contains a number of breaking changes and updating from metafacture-core-3.5.0 is not trivial. The most notable changes are the update to Java 8 and the reorganisation of the package structure which changed the qualified names of all Metafacture modules and moved many of the other classes to new packages. Please refer to the list of modules per package at the end of the release notes to find the new module locations. All other relocated and renamed classes are listed below.
Changes
- New features
- Bug fixes
- Removed features
- Moved and renamed items
- Changed behaviour
- Other improvements
New features
Flux
Metamorph
- Fix #256: Support sameEntity in none and all. Add support for the
sameEntity
attribute tonone
andall
statements. The attribute does not make sense inany
statements (see 93bc48d) - The
RestMap
which looks up values by doing a REST request works now (thanks @philboeselager, see 704d4ad) - The new helper class
InlineMorph
simplifies embedding Metamorph scripts directly in Java. It was introduced to help writing test cases for Metamorph functions and collectors (see 9b6e7f1)
Metafacture modules
- Added
ForwardingStreamPipe
as a base class for modules which only need to intercept some events but forward all others unmodified (see 4412623) - Added
NullFilter
which replacesnull
values with a replacement string or discards them (see c282c74) - Added
JsonToElasticsearchBulk
module to create Elasticsearch bulk import data from JSON (thanks @fsteeg and @blackwinter, see 9d85194, 056fe60) - Added
AlephMabXmlHandler
for the widely used Mab-Xml derivative created by Aleph exports (thanks @dr0i, see 482af42) - Added
AseqDecoder
. A decoder for aseq data (thanks @larsgsvensson, see f77b1ae) - Added
XmlElementSplitter
. The module splits an xml document at acertain element (thanks @dr0i, see 5517beb) - Added
XmlFilenameWriter
: The module extracts a file name from an xml document and saves the document to the extracted file. It's possible to store the file uncompressed and as bz2 (thanks @dr0i, see 40d290e) - Added
XmlTee
. A tee implementation for XML event streams. Allows to forward a stream to more than one downstream module (thanks @dr0i, see 3fb3b4d) PojoEncoder
: Added support for populating maps in POJOs (thanks @thomasseidel, see e23bc13)IdChangePipe
(nowRecordIdChanger
): Added corresponding getters for setters (see e8300a8)Utf8Normalizer
(nowUnicodeNormalizer
): Made normalisation form configurable (see 314a641)PicaDecoder
: Added record ids for level 1 & 2 records. Local system records (level 1) and holding records (level 2) do not store their record id in field003@ $0
but in field107F $0
or203@ $0
(the latter may include an occurrence specification). These ids are now emitted as record ids in start-record events when level 1 or level 2 records are processed (see c955f4d, a9529dd)Marc21Encoder
: The record identifier field (001) can now be automatically created from the record identifier of the start record event. This is configured by thesetGenerateRecordId(boolean)
parameter (see 6d04d69)
Other
- Added a framework reading and writing ISO 2709:2008 records (see 3b24df5)
ResourceUtil
: AddedreadAll(InputStream, Charset)
andreadAll(Reader)
to read a full stream into a string (see 9a70936)XmlUtil
: Addedescape(String)
method for escaping strings for xml output (see 9fb4154)
Bug fixes
Flux
- The flux grammar supported octal escape sequence but failed to convert them into characters after parsing (see b4da5bf)
Metamorph
- Fix #255: Metamorph emits null as entity name (see 8adafef)
- Fix #257: Do not reset entity if reset is false (see a680a28)
- Fix #265:
split
andswitch-name-value
functions emitted wrong source (see a7f6785) - Fixed resource leak in Metamorph file-maps (see d519e8c)
Metamorph-Test
- Fix #6: Fix test names for Metamorph Test in Intellij. Intellij did not show the name of the xml files containing the tests but only the string "xml". This was caused by Intellij interpreting the test class name as a fully qualified java class name and attempting to extract the class name from it. By making sure that the Metamorph Test names do not contain any dots this problem can be avoided (see 033e6d0)
Metafacture modules
StreamUnicodeNormalizer
no longer fails onnull
values in literals but simply forwards them (see 12e3420)Marc21Encoder
produced invalid MARC 21 records if the record data contained unicode codepoints which required more than one byte in UTF-8 encoding (see 6d04d69)
Removed features
Flux
- Removed
generic-xml
Flux command. Usedecode-xml
followed byhandle-generic-xml
instead (see 53edfdb) - Removed
MorphVisualizer
. The tool was outdated and not well maintained. There is currently no replacement for it (see 73efd63)
Metamorph
- Fix #226: Remove miss-spelled options from
occurrence
function. Theoccurrence
function in Metamorph no longer supportslessThen
andmoreThen
in itsonly
attribute. InsteadlessThan
andmoreThan
must be used (see 800108e) - The constructors of the collector helper classes expected a reference to the
Metamorph
object. The parameter has been removed as collectors should not access theMetamorph
object (see f76f39e) CollectFactory
,FunctionFactory
andMapsFactory
have been made package-private in themetamorph
package. The are considered to be an internal part of Metamorph (see c805690)
Metamorph-Test
TestConfigurationException
is removed and replaced with JUnit'sInitializationError
(see 35dfaf5)MetamorphTestCase
,MetamorphTestLoader
andMetamorphTestRunner
are made package-private as the are not required for using Metamorph-Test (see 35dfaf5)
Metafacture modules
- Removed
CGEntityDecoder
,CGEntityEncoder
,CGTextDecoder
,CGEntityReader
and the helper classCGEntity
. UseFormetaDecoder
andFormetaEncoder
instead. To convert data cg-entity or cg-text format to Formeta use metafacture-3.5.0 which contains support for both formats (see ac1c71d) - Removed
Bzip2Opener
andGzipOpener
. The genericFileOpener
automatically recognises and handles compressed files (see c3f24eb) - Removed
RecordBounderyRemover
. The functionality provided by this module is also provided byStreamEventDiscarder
(see 305c8d4) - Removed
ObjectExceptionLogger
in favour ofObjectExceptionCatcher
which provides the same functionality (see 919349b) - Removed
RecordBatcher
. Use implementations ofAbstractBatcher
instead (see 52da508) - Removed
StreamFormatter
. TheFormetaEncoder
andStreamLogger
modules provide very similar functionality and should be used instead (see 2f6bc2a) - Removed
WrappingStreamPipe
. Modules with nested pipelines should manage them themselves (see 52a5f21) - Removed
SimpleJsonEncoder
. TheJsonEncoder
module provides the same functionality (see 7d5e467) - Removed
EventListSource
. If used withEventList
it can be often replaced with aStreamBuffer
(see f77539b) - Removed
MultiOpener
. There were no concrete opener implementations registered to be used byMultiOpener
. So, the class was completely useless (see 566022f) - Removed
CsvReader
,GenericXmlReader
,LidoReader
,MabReader
,MarcReader
,MetsModsReader
andPicaXmlReader
. Users who use these readers should replace them by the corresponding combination of a record splitting module and a recor...
Metafacture Core 3.5.0
This is a bug fix release with some small new features
Bug fixes
- Fix #255: Metamorph emits null as entity name (9821220)
- Fix #257: Do not reset entity if reset is false (ee9ef99)
New Features
See commits for details
Maven Coordinates
Metafacture core is available on Maven Central:
<dependency>
<groupId>org.culturegraph</groupId>
<artifactId>metafacture-core</artifactId>
<version>3.5.0</version>
</dependency>
Metafacture Core 3.4.0
This is a feature release which adds some new Metafacture modules
New Features
- Add module
RecordToEntity
to turn records into entities (c337ce9) - Add module
EntityPathTracker
for returning the current entity path (0a3ff6f) - Add module
StreamEventDiscarder
(e613a26) - Add
StreamDeferrer
module (3acf7a1) PicaDecoder
: Make whitespace removal in field names optional (3547a43)
See commits for details.
Maven Coordinates
Metafacture core is available on Maven Central:
<dependency>
<groupId>org.culturegraph</groupId>
<artifactId>metafacture-core</artifactId>
<version>3.4.0</version>
</dependency>
Metafacture Core 3.3.1
This is a small bug fix release.
Bug fixes
- Fix #191:
TriplesCollect
Should not outputendRecord()
events incloseStream()
andresetStream()
events if no triples were received.
Maven Coordinates
Metafacture core is available on Maven Central:
<dependency>
<groupId>org.culturegraph</groupId>
<artifactId>metafacture-core</artifactId>
<version>3.3.1</version>
</dependency>
Metafacture Core 3.3.0
This is a minor update.
Bug fixes
SimpleXmlEncoder
did write a closing root tag ifresetStream()
was called even if no root tag was open (see #249, which describes the same issue incloseStream()
for details)
New features
- Fix #252: Add parameter (
setWriteRootTag
) to make output of root tag inSimpleXmlEncoder
optional.
Maven Coordinates
Metafacture core is available on Maven Central:
<dependency>
<groupId>org.culturegraph</groupId>
<artifactId>metafacture-core</artifactId>
<version>3.3.0</version>
</dependency>
Metafacture Core 3.2.0
This is mainly a bug fix release but it has some new features.
Changed Behaviour
- The reset behaviour of the
choose
collector has changed when it is used with anif
-condition: In the old implementation the chosen value was only cleared after it was actually emitted (so only if the condition was met). Now it is also cleared if it was attempted to be emitted but was not due to the condition not being met. This is in-line with the behaviour of the other collectors.
Bug fixes
- Disable javadoc doclint in order to be able to create release builds and submit snapshots builds to http://oss.sonatype.org/ from Travis CI.
- Fix #249: emits closing tag without opening tag in SimpleXmlEncoder
- Fix #235, #237: Add PicaXmlHandler to flux-command.properties
New features
- Resolve #49, #210, #250: The
choose
collector now supportsreset
andsameEntity
and is reset even if anif
-condition prevents emitting a value. - Resolve #247: set namespace map directly in
SimpleXmlEncoder
. It is no longer neccessary to use aMultiMap
with a specially named key that contains the actual namespace map. Instead the namespace map can be passe directly to theSimpleXmlEncoder
. - Resolve #248: support default namespace in
SimpleXmlDecoder
- Resolve #187: Make delimiter in concat is now optional
- Resolve #238: the Travis build status is now shown in the readme file on github
Maven Coordinates
Metafacture core is available on Maven Central:
<dependency>
<groupId>org.culturegraph</groupId>
<artifactId>metafacture-core</artifactId>
<version>3.2.0</version>
</dependency>
Metafacture Runner Distribution 3.1.2
This release updates the metafacture-core dependency to version 3.1.2 Please see the release notes for metafacture-core for a list of changes.
Metafacture Core 3.1.2
This is a bug fix release.
Bug fixes
- Fix #233: Do not trim MarcXML record leader
- Java 8 support: update Antlr and updates for Travis CI (see #211, #230). Please note: metafacture-core is still built for Java 6. The changes only allow to built the library on JDK 8.
Maven Coordinates
Metafacture core is available on Maven Central:
<dependency>
<groupId>org.culturegraph</groupId>
<artifactId>metafacture-core</artifactId>
<version>3.1.2</version>
</dependency>
Metafacture Runner Distribution 3.1.1
This release updates the metafacture-core dependency to version 3.1.1 Please see the release notes for metafacture-core for a list of changes.
Metafacture Core 3.1.1
This is a bug fix release.
Bug fixes
- #231
Marc21Encoder
now ignores literals namedtype
which are emitted by theMarcXmlHandler
. This prevented the conversion of marcxml to marc21. - #232
MarcXmlHandler
accessed some attributes in not by name but by index. This makes the implementation depending on the order of the attributes in the data. This is fixed now.
Maven Coordinates
Metafacture core is available on Maven Central:
<dependency>
<groupId>org.culturegraph</groupId>
<artifactId>metafacture-core</artifactId>
<version>3.1.1</version>
</dependency>