1

I am working on this issue in the Feeds issue queue: https://www.drupal.org/node/2341407

I am trying to make the updates to the importers run in batches. I can't figure out why this won't run in multiple batches when there are multiple importers. What happens is instead of seeing a progress bar reload a few times when running update.php, it loads once and processes all importers in a single batch.

The code works well except for batching. Below is the code. Thanks in advance!

<?php /** * Fix importer mappings for file and image fields to use :uri convention. */ function feeds_update_7210(&$sandbox) { // If this is the first pass through this update function then set some variables. if (!isset($sandbox['total'])) { // Load all importers. $importers = feeds_importer_load_all(TRUE); $sandbox['total'] = count($importers); $sandbox['current'] = 0; } foreach ($importers as $importer) { $processor = $importer->processor; $config = $processor->getConfig(); foreach ($config['mappings'] as $id=>$mapping) { // Act on mappings that do not contain a colon. if (!strpos($mapping['target'], ':')) { // Get field data. Check if $info is empty to weed out non-field mappings like temporary targets. $info = field_info_field($mapping['target']); if (!empty($info)) { // Act on file or image fields. if (in_array($info['type'], array('file', 'image'))) { // Add ':uri' to fix the mapping. $config['mappings'][$id]['target'] = $mapping['target'] . ':uri'; } } } } // Add the updated config and save the importer. $processor->addConfig(array('mappings' => $config['mappings'])); $importer->save(); drupal_set_message(t('@importer fields have been updated.', array('@importer' => $importer->id))); $sandbox['current']++; } // Set the value for finished. If current == total then finished will be 1, signifying we are done. $sandbox['#finished'] = ($sandbox['current'] / $sandbox['total']); if ($sandbox['#finished'] === 1) { drupal_set_message(t('@importers importers processed.', array('@importers' => $sandbox['total']))); } }
Improve this question
0

1 Answer 1

Reset to default
5
+50

Why are you expecting it to work as a batch, it's doing exactly what you're asking it to do and processing all the importers in the foreach loop! foreach ($importers as $importer) It's not magic, it can't assume that's what you want to batch :o)

Try this instead (obviously I haven't tested it, I've just typed it straight into the text editor here - but it's essentially your code so it should be fine); 

/** * Fix importer mappings for file and image fields to use :uri convention. */ function feeds_update_7210(&$sandbox) { // If this is the first pass through this update function then set some variables. if (!isset($sandbox['progress'])) { $sandbox['importers'] = feeds_importer_load_all(TRUE); $sandbox['progress'] = 0; $sandbox['max'] = count($sandbox['importers']); } $importer = array_pop($sandbox['importers']); $processor = $importer->processor; $config = $processor->getConfig(); foreach ($config['mappings'] as $id => $mapping) { // Act on mappings that do not contain a colon. if (!strpos($mapping['target'], ':')) { // Get field data. Check if $info is empty to weed out non-field mappings like temporary targets. $info = field_info_field($mapping['target']); if (!empty($info)) { // Act on file or image fields. if (in_array($info['type'], array('file', 'image'))) { // Add ':uri' to fix the mapping. $config['mappings'][$id]['target'] = $mapping['target'] . ':uri'; } } } } // Add the updated config and save the importer. $processor->addConfig(array('mappings' => $config['mappings'])); $importer->save(); $sandbox['progress']++; // Set the value for finished. If progress == max then finished will be 1, signifying we are done. $sandbox['#finished'] = ($sandbox['progress'] / $sandbox['max']); if (empty($sandbox['importers'])) { return t('@importers importers processed.', array('@importers' => $sandbox['max'])); } }

For hints about my alterations see the API documentation - note that the naming convention changes don't alter anything, I just made them for Drupal core consistency. The most important thing to note is that all I've really done, is to remove the foreach loop so one importer gets used at a time.

NOTE: For clarification (as it's really not obvious) the percentage shown relates to the number of batch operations run - not the amount of passes through one batch operation. See _batch_process():

// Total progress is the number of operations that have fully run plus the // completion level of the current operation. $current = $total - $remaining + $finished; $percentage = _batch_api_percentage($total, $current);

On a normal batch operation you'd pass in several separate operations like this;

$batch = array( 'operations' => array( array('batch_example_one_process', array($options1, $options2)), array('batch_example_two_process', array($options3, $options4)), ),

And you'd see 50% completion only after batch_example_one_process has completed. I guess you can see that your code is working correctly though, as it clearly only processes one processor per pass - as long as they all get processed, you can confirm that batch api is working correctly.

Improve this answer
5
  • Aha, I missed the part about only returning one value. I changed your code to use array_pop(), but when I run the update from update.php, it still seems to run as a single batch (the progress bar just completes in one shot and doesn't break up to run multiple times). If I output $sandbox['progress'], I see it outputting multiple times and incrementing correctly when the update completes. Any ideas why this happens? Here is the current code: gist.github.com/vinmassaro/049109c69d59aa2d33c0 Commented Oct 13, 2014 at 21:58
  • Try the updated code - apologies, the API documentation was a little vague around the return for a successful completion message. Commented Oct 13, 2014 at 22:27
  • Still seems to complete in one batch operation. The progress bar goes immediately to 100%. I still can't figure out why it's not getting the number of items, then running the batch that many times and showing a correctly segmented progress bar. Commented Oct 14, 2014 at 4:42
  • Ah sorry I should have pointed this out before, the percentage is based on operations not batches. It would have saved your confusion. As you can't put code blocks in comments, I've edited my reply post. Commented Oct 14, 2014 at 6:09
  • This was not clear at all from the API docs! Thanks! Commented Oct 14, 2014 at 11:40

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.