Performance & Optimization

Optimize import performance and configure batch processing settings

Overview

PrimeLink includes several performance optimization features to help you import large datasets efficiently without overwhelming your server or causing timeouts.

These settings allow you to control how imports are processed, balance speed with resource usage, and ensure reliable imports even with large files.

Performance Options

Import Speed Optimization

Enable speed optimization for faster imports:

  • Reduces Validation: Skips some non-critical validations
  • Optimizes Queries: Uses optimized database queries
  • Reduces Logging: Minimizes logging overhead
  • Faster Processing: Overall faster import execution

When to Use

Enable speed optimization when:

  • Importing large datasets (thousands of items)
  • Data is already validated and clean
  • You need faster import times
  • Server resources allow for higher load
⚠️ Note: Speed optimization may skip some validations. Only use when you're confident your data is clean and properly formatted.

Iterative Processing

Piece-by-Piece Processing

Process imports in smaller batches (iterations):

  • Prevents Timeouts: Avoids PHP execution time limits
  • Background Processing: Runs in background without blocking
  • Resumable: Can resume if interrupted
  • Progress Tracking: Shows progress for each iteration

Records per Iteration

Configure how many records to process per iteration:

  • Default: 20 records per iteration
  • Range: 1 to 500 records
  • Lower Values: More iterations, slower but safer
  • Higher Values: Fewer iterations, faster but more resource-intensive

Choosing the Right Settings

Small Imports (Less than 100 items)

  • Speed Optimization: Optional (can enable for faster processing)
  • Iterative Processing: Optional (not necessary for small imports)
  • Records per Iteration: N/A (if iterative processing disabled)

Medium Imports (100-1000 items)

  • Speed Optimization: Recommended
  • Iterative Processing: Recommended
  • Records per Iteration: 20-50 (default 20 is good)

Large Imports (1000+ items)

  • Speed Optimization: Highly recommended
  • Iterative Processing: Required
  • Records per Iteration: 20-50 (start with 20, increase if stable)

Server Considerations

PHP Settings

Check your PHP configuration:

  • max_execution_time: Should be sufficient (or use iterative processing)
  • memory_limit: Should be at least 256MB (512MB+ recommended)
  • post_max_size: Should accommodate your file size
  • upload_max_filesize: Should accommodate your file size

Database Performance

Optimize database for imports:

  • Ensure database has sufficient resources
  • Consider database indexing for faster lookups
  • Monitor database load during imports
  • Use iterative processing to reduce database load

WordPress Cron

For scheduled imports:

  • Ensure WordPress cron is working properly
  • Consider using real cron instead of pseudo-cron
  • Monitor cron execution times

Background Processing

How It Works

When iterative processing is enabled:

  • Starts Import: Import begins processing
  • Processes Batch: Processes configured number of records
  • Schedules Next: Schedules next iteration via WordPress cron
  • Repeats: Continues until all records are processed
  • Completes: Marks import as complete when done

Benefits

  • No browser timeout issues
  • Can close browser and let import continue
  • Progress tracking available
  • Can resume if interrupted

Optimization Tips

Data Preparation

  • Clean Data: Remove unnecessary columns and rows
  • Optimize File Size: Compress or split large files
  • Validate Format: Ensure data format is correct
  • Remove Duplicates: Clean duplicates before import

Import Configuration

  • Use Filters: Filter data at source to reduce processing
  • Selective Mapping: Only map necessary fields
  • Skip Unchanged: Enable "Skip Unchanged" option if available
  • Use Templates: Save and reuse optimized configurations

Server Optimization

  • Increase PHP Limits: Adjust max_execution_time and memory_limit
  • Optimize Database: Ensure proper indexing
  • Use Caching: Enable object caching if available
  • Monitor Resources: Watch server resources during imports

Monitoring Performance

Import Progress

Monitor import progress:

  • Check import status in Manage Imports
  • View progress indicators
  • Monitor iteration progress
  • Check completion status

Error Monitoring

Watch for errors:

  • Check import logs for errors
  • Monitor error rates
  • Review failed items
  • Adjust settings if errors occur

Resource Monitoring

Monitor server resources:

  • Check CPU usage during imports
  • Monitor memory usage
  • Watch database load
  • Adjust iteration size if needed

Troubleshooting Performance Issues

Import Too Slow

  • Enable Speed Optimization
  • Increase Records per Iteration (if using iterative processing)
  • Check server performance
  • Optimize source data
  • Reduce number of mapped fields

Timeouts

  • Enable Iterative Processing
  • Reduce Records per Iteration
  • Increase PHP max_execution_time
  • Use background processing

Memory Issues

  • Reduce Records per Iteration
  • Increase PHP memory_limit
  • Enable Speed Optimization
  • Process smaller batches

Database Lock Issues

  • Reduce Records per Iteration
  • Enable Iterative Processing
  • Check for other processes locking database
  • Optimize database queries

Best Practices

  • Start Conservative: Begin with default settings and adjust as needed
  • Test First: Use Dry Run mode to test performance settings
  • Monitor Resources: Watch server resources during imports
  • Adjust Incrementally: Make small adjustments and test
  • Use Background Processing: For large imports, always use iterative processing
  • Optimize Data: Clean and optimize source data before import
  • Document Settings: Keep notes of optimal settings for your server

Next Steps

🧪 Dry Run & Test Mode

Test your import configuration

Learn More →
🔄 Rollback & Tracking

Learn how to rollback imports if needed

Learn More →
📥 Creating Imports

Complete guide to creating imports

Learn More →