bentools-etl

3.0 alpha is out! This is a complete redesign of the library, so don't upgrade from 2.5 if you don't intend to refactor your whole app, since almost everything has changed. This is now a lot simpler:

• No more useless extractors: if your input data is already an iterable, you don't need one.
• Removed ContextElement object to carry data during the ETL lifecycle - no need for that wrapper anymore.
• Transformers now MUST return a \Generator even if they just transform a single value. Because transformers can now transform 1 thing into multiple others.
• As a consequence, a \Generator is expected to be injected into all loaders.
• Redesigned the built-in event dispatcher, in favor of a PSR-14 compliant one.
• Events are now read-only, they can't modify data anymore. They still allow skipping the current item / stopping the process, and ignore exceptions at the item level. Way less difficult to maintain. Implement your own extractor / transformer and/or loader instead, if you want a fine control of what goes in and what goes out.
• Dropped support for asynchronous ETL processing.
• More decoupling.
• New ETL builder with its recipes.
• Newcomers can get started way more easier since there are less concepts to understand.

See Transformers documentation.

Exception Handling

The extract, transform, or load part of the ETL process may sometimes throw exceptions.
Before throwing it, we should be able to hook on the exception, handle it, and decide wether or not to continue the process.

CSV Iterators

You can now use CsvFileIterator or CsvStringIterator to iterate over your CSVs.
Also check KeysAwareCsvIterator to automatically assign the keys of each CSV row (you can turn an indexed array into an associative one). Check out the Iterators documentation.