Pipes

Pipes is a PHP Extract Transform Load [ETL] package for Laravel 8+

Installation

composer require jwhulette/pipes

Usage

1. Create a new EtlPipe object.

2. Add an extractor to the object to read the input file

   • You can create your own extractor by implementing the ExtractorInterface.php

3. Add transformers to transform the data

   • You can add as many transformers as you want.

   • You can create your own transformers by implementing the TransformerInterface.php

   • Data is passed to the transformers in the order they are defined

4. Add a loader to write out the transformed data.

   • You can create your own loader by implementing the LoaderInterface.php

(new EtlPipe())
->extract(new CsvExtractor($this->csvFile));
->transforms([
    new CaseTransformer([], 'lower'),
    new TrimTransformer(),
])
->load(new CsvLoader('saved-file.csv'));

Notes:

Built-in extractors:

• CsvExtractor
• XlsxExtractor
• SqlExtractor

Built-in loaders:

• CsvLoader
• SqlLoader

Built-in transformers:

• CaseTransformer - Change the case of a string
• DateTimeTransformer - Change the format of a date string
• PhoneTransformer - Transform a US phone, removing all non numeric characters, and limiting the length to the first 10 digits
• TrimTransformer - Trim a string
• ZipcodeTransformer - Transform a US zip code, removing all non numeric characters, and left pad zeros for zip codes less than 5 digits
• ConditionalTransformer - Transform a column, based on the values of another column

Data is passed to the transformers in the order they are defined

Changelog

Please see CHANGELOG for more information on what has changed recently.

Contributing

Please see CONTRIBUTING for details.

Security Vulnerabilities

Please review our security policy on how to report security vulnerabilities.

Credits

• Wes Hulette
• All Contributors

License

The MIT License (MIT). Please see License File for more information.