Return to Question

Solution I split into small CSV files, applied the tweaks that Shlomi Noach suggested and then loaded each file.

Solution I split into small CSV files, applied the tweaks that Shlomi Noach suggested and then loaded each file.

Solution I split into small CSV files, applied the tweaks that Shlomi Noach suggested and then loaded each file.

Not the perfect. But it loaded a bit faster. Dumping the original table to a CSV file took less than 1 minute, which avoided the table from being locked during the whole process. A script did the job to load each file, which would be very boring to do manually.

In the end I just had to get from the original table the rows inserted/updated in the last hours and update on the new table.

What is 'slow'

Let's split the into several files?

I tried that. Each file contained 100,000 rows.

• File 1 = 2.76 sec
• File 2 = 1 min 40 sec
• File 3 = 4 min 16 sec

I gave up on file 3. I'd have to do it 157 more times and on each iteration the processing time would increase. Too much time...

What is 'slow'