Magento 2 Data Flow Import/Export Documentation

Installation

You can install the module for Magento 2 using Composer or you can manually install it using FTP.

# Add the FishPig Composer repo
composer config repositories.fishpig composer https://repo.fishpig.com/

# Install the module using Composer
composer require fishpig/magento2-fseo:*

# Enable the module in Magento 2
php bin/magento module:enable FishPig_Flow

# Enable the module in Magento 2
bin/magento module:enable FishPig_Flow

# Run the Magento upgrade system
bin/magento setup:upgrade

Flow allows for free-form data import and export from Magento 2. It can be adapted to work for any data type and be used via the Admin GUI, directly in PHP code or even via the CLI.

Flow has profiles set to work with common data types (eg. products, categories etc) but also includes a SimpleImport and SimpleExport profile that can be adapted to work with any DB table.

↑ Back to Top

Stores & Scope

When running an import or export, it's possible to set the Store ID and Store Mode. By default this will be store_id=0 and store_mode=strict but this can be changed to any store_id and store_mode=blend.

Store Mode: Strict

No fall back to store_id=0 will be used. If importing, values will always be set against the store_id, even if no value is set on store_id=0. When exporting, if no value exists for store_id, no value will be returned, even if a value is against store_id=0

Store Mode: Blend

When exporting in Blend mode, if no data is found for given store ID, store_id=0 will be checked and any values returned instead.

When importing Blended mode, if no value is set for the store_id and store_id=0 has a matching value, no new value will be imported against store_id.

↑ Back to Top

Attribute Labels

For certain attributes, Magento stores both an ID and a label. An example of this would be the 'color' or 'brand' attributes, but it's also true for internal attributes such as 'visibility' and 'status'. By default, Flow will return the label rather than the ID.

For attributes like this, you can determine the type by looking at the column header in the CSV file. If the column starts with a '#' (eg. #color) then this is the label. If it does not start with '#' (eg. color) then it is the ID. This is useful for importing as it allows you to import using a label or an ID.

↑ Back to Top

Exporting

Build an Export via the Admin

Building an export via the Admin is easy and it can only take seconds to generate a complex export. Select the Export profile that you require via the DataFLOW menu item. You will now see the export builder with the required columns already added. You can now add any extra columns that you want and add filters, if required.

When you're ready, you can preview the Export in the browser by clicking the Preview button. Once you're happy, click Export and the CSV file will be downloaded.

The Exported CSV

The exported CSV will include all of the data specified via the export builder, but it may also have some extra columns. For example, if you export 'qty', the is_in_stock column is also added. Also in the category product link export, category and product names are added, to help make identification of the IDs easier.

You will also notice some column headings start with '#'. This indicates that this column will be displaying it's label rather than ID. For more on this, please see option labels and IDs.

Reimporting the Exported CSV

All exported CSVs from Flow can also be reimported using Flow. You can modify the CSV data, add/remove columns and still reimport. As long as the identity columns (eg. entity_id or SKU) are present then the data will import correctly. If these values are not present then Flow will treat the record as a new item and try to create it.

Exporting via PHP

Flow was built first as PHP import/export tool for developers and this module was created as GUI wrapper for this library. This means you can perform flexible and powerful exports via PHP.

Export all Configurable Products

/* \FishPig\Flow\Service\Export\Catalog\ProductExportFactory */
$data = $productExportFactory->create()->setStore(
    \FishPig\Flow\Service\AbstractService::STORE_MODE_STRICT,
    0
)->export([
    'entity_id',
    'sku',
    'name',
    'type_id' => 'configurable',
    '#status'
]);

The above code exports all configurable products and includes the ID, SKU, name and status (label so it will be Enabled rather than 1).

Export Price, QTY for all simple products

use \FishPig\Flow\Service\Export\Filter\RangeFilter;

/* \FishPig\Flow\Service\Export\Catalog\ProductExportFactory */
$data = $productExportFactory->create()->setStore(
    \FishPig\Flow\Service\AbstractService::STORE_MODE_STRICT,
    0
)->export([
    'entity_id',
    'sku',
    'name',
    'type_id' => 'simple',
    'price',
    'qty' => [
        RangeFilter::KEY => RangeFilter::build(1)
    ],
]);

The above code gets all simple products that have a QTY of 1 and above. It also includes the price of the product. This could then be updated with new prices and/or qty and then reimported for quick stock updates.

Exporting via CLI

You can also export via the CLI!

bin/magento fishpig:flow:export --builder=catalog_product --sku --name --#color=Green --#color=Blue

The above command gets the SKU, name and color label or all products that have a color of either Green or Blue. You can specify any fields that you want. This will print the output to screen in an easy to read format and is great for debugging. If you want it in CSV format, that you can pipe to a file, use:

bin/magento fishpig:flow:export --builder=catalog_product --sku --name --#color=Green --#color=Blue --output-format=csv > some-file.csv

↑ Back to Top

Importing

Import CSV Format

Flow uses a flexible file format that allows for powerful and efficient imports. It does this by matching the column names (the top row in your CSV file) against attributes available for the type of import. For example, if you have a column named 'sku' then this will map to the 'sku' product attribute.

You can build your CSV files by hand (in a text editor, Excel, Numbers, Google Sheets etc) or you can first run an export of the data that you want to import and the resulting CSV file be in the exact format that you need!

Attribute Option Labels

We discussed attribute option labels and IDs earlier but it's worth repeating here. Certain Magento attributes (select and multiselect) have both an ID value and a text/label value. A good example of this is color, where you see on the frontend the actual color name (eg. Red) but the system will also see a numerical ID. Flow can handle both the label and ID. Using color as our example, if you import using the column heading 'color' then it will expect an ID. If you want to use the label, just add '#' to the start of the column name, so it becomes '#color'. This will now expect a label.

If you do forget, Flow is smart enough to detect the data entered is not a number and will convert this for you. This won't work though if the label you're adding is actually a number, so it's always best to use the '#' when using labels.

Run an Import

Now that you have your CSV file, to import it, just load the correct import profile via the DataFLOW menu in the Magento Admin and upload the file. The import will happen in real time and you will be given the results straight away.

Any indexes and caches that need updating will be updated automatically and you will be notified of this.

Import via PHP

Flow allows for importing data directly using PHP and is very to do. Just create an array with each item having the column names as the array keys and Flow will do the rest.

/* \FishPig\Flow\Service\Import\Catalog\ProductImportFactory */
$data = $productImportFactory->create()->setStore(
    \FishPig\Flow\Service\AbstractService::STORE_MODE_STRICT,
    0
)->import([
  [
    // -1 here is auto converted to the default attribute_set_id
    'attribute_set_id' => -1,
    'sku' => 'rubix-cube',
    'name' => 'Rubix Cube',
    'type_id' => 'simple',
    '#status' => 'Enabled',
    'visibility' => 4,
    'price' => 9.99,
    'qty' => 50
  ]
]);

You can also import configurable products by specifying the child_skus column, which is a comma separated list or an array of SKUs. You also need to set the _super_attributes column, which contains the attributes to use to create the configurable links.

Import via CLI

You can import CSV files via the CLI using Flow, which makes importing scheduled stock and price feeds so very easy.

bin/magento fishpig:flow:import --builder=catalog_product --file=path/to/data.csv

↑ Back to Top

Use Cases

The possibilities with Flow are literally endless but we thought we would list a few here.

Import Magento 2 Products

Whether it's a new site and you need to migrate your products or you have a regular feed of new products, Flow can import the data in no time whatsoever. This can be done via the Admin manually, or you can build a script to do it, or even do it via the CLI.

Update Prices/Stock QTY in Bulk

Updating prices in M2 is hard. With Flow, you can export your prices. This can be all prices, or you can filter the export by brand or another filter. You can then apply changes in the spreadsheet using formulas or another method and then quickly reimport the CSV. All indexes and caches will be updated in realtime.

You can also get price or stock feeds from suppliers. As long as the column names are correct (sku, price, qty) then the data will import directly. If the column names are different, it will only take a second to change them and then import.

Re-Order Products in Categories

Using the Category Products Links profile to export products in a category, re-order them and then reimport. This is much easier than using the Admin GUI built into Magento 2.

↑ Back to Top