mysql - SQL import CSV file with PHP -

-

i'm trying import pretty big csv file database (locally) file 230mb , 8.8 million lines problem have isn't opening csv or dont know how import it, file opens, imports 500,000 lines , quits , trows no error or timeout or anything, see webpage.

this code: 

    try {     $conn = new pdo("mysql:host=$servername;dbname=adresses_database", $username, $password);     // set pdo error mode exception     $conn->setattribute(pdo::attr_errmode, pdo::errmode_exception);     echo "connected successfully";      $row = 1; if (($handle = fopen("bagadres.csv", "c+")) !== false) {     while (($data = fgetcsv($handle, '', ";")) !== false) {         if (!isset($write_position)) {        // move line previous position, except first line                 $write_position = 0;         $num = count($data); // $num 15         $row++; //i dont need this?         $stmt = $conn->prepare("insert adresses (openbareruimte, huisnummer, huisletter, huisnummertoevoeging, postcode, woonplaats, gemeente, provincie, object_id, object_type, nevenadres, x, y, lon, lat) values (:openbareruimte, :huisnummer, :huisletter, :huisnummertoevoeging, :postcode, :woonplaats, :gemeente, :provincie, :object_id, :object_type, :nevenadres, :x, :y, :lon, :lat)");         $stmt->bindparam(':openbareruimte', $data[0]);         $stmt->bindparam(':huisnummer', $data[1]);         $stmt->bindparam(':huisletter', $data[2]);         $stmt->bindparam(':huisnummertoevoeging', $data[3]);         $stmt->bindparam(':postcode', $data[4]);         $stmt->bindparam(':woonplaats', $data[5]);         $stmt->bindparam(':gemeente', $data[6]);         $stmt->bindparam(':provincie', $data[7]);         $stmt->bindparam(':object_id', $data[8]);         $stmt->bindparam(':object_type', $data[9]);         $stmt->bindparam(':nevenadres', $data[10]);         $stmt->bindparam(':x', $data[11]);         $stmt->bindparam(':y', $data[12]);         $stmt->bindparam(':lon', $data[13]);         $stmt->bindparam(':lat', $data[14]);     $stmt->execute(); } else {                 $read_position = ftell($handle); // actual line                 fseek($handle, $write_position); // move previous position                 fputs($handle, $line);           // put actual line in previous position                 fseek($handle, $read_position);  // return actual position                 $write_position += strlen($line);    // set write position next loop             }             fflush($handle);                         // write pending change file         ftruncate($handle, $write_position);     // drop repeated last line         flock($handle, lock_un);        }    fclose($handle); }     } catch(pdoexception $e)     {     echo "connection failed: " . $e->getmessage();     }  $conn = null;

i came far looking on stackoverflow , php manual, searched if mysql error. cannot figure out, (for suggestions mysql settings im using linux mint 18)

i recommend use mysql's load data infile, fastest , efficient csv data mysql table. command setup this:

load data infile 'bagadres.csv'  table adresses fields terminated ','  enclosed '"' lines terminated '\n' ignore 1 rows

if fields not enclosed quotes, or enclosed other quotes, remove or modify enclosed by clause. also, ignore 1 rows ignore first row, make sense assuming first line of file header row (i.e. not actual data column labels).


Comments

Post a Comment

Popular posts from this blog

Formatting string according to pattern without regex in php -

-
how can format arbitrary string according flexible pattern? solution came using regular expressions, need 2 "patterns" (one search , 1 output). example: $str = '123abc5678"; desired output: 12.3ab-c5-67.8 i use pattern in variable (one user can define without knowledge of regular expressions) this: $pattern = '%%.%%%-%%-%%.%'; so user have use 2 different characters (% , .) a solution regex this: $str = '123abc5678'; $pattern_src = '@(.{2})(.{3})(.{2})(.{2})(.{1})@'; $pattern_rpl = "$1.$2-$3-$4.$5"; $res = preg_replace($pattern_src, $pattern_rpl, $str); //$res eq 12.3ab-c5-67.8 way complicated since user need define $pattern_src , $pattern_rpl. if string vary in length, more complex explain. yes, write function/parser builds required regular expressions based on simple user pattern %%.%%%-%%-%%.%. wonder if there "built in" way achieve php? thinking sprintf etc., doesn't seem trick. ideas? ...
Read more

c - zlib and gdi32 with OpenSSL? -

-
this follow question undefined reference rsa_generate_key in openssl? have openssl program generate rsa key, follows : #include <stdio.h> #include <openssl/rsa.h> #include <openssl/pem.h> int main(int argc, char* argv[]) { rsa *r = null; int ret; r = rsa_new(); bignum *bne = null; ret = rsa_generate_key_ex(r, 2048, bne, null); return 0; } when compile gcc -i../include rsatest.c -lcrypto -l. i undefined reference errors functions in libraries gdi32 , zlib. so correct this, have add -lz , -lgdi32, as gcc -i../include/ rsatest.c -lcrypto -lz -lgdi32 -l. my question is, aren't static libcrypto , libssl self sufficient? in why having add libraries make work? there option should have added while compiling library prevent this? i'm using 64 bit windows mingw compiler. made openssl library of msys , make.
Read more

java - inputmismatch exception -

-
Image
this question has answer here: scanner skipping nextline() after using next(), nextint() or other nextfoo()? 14 answers import java.util.*; public class ngg { static scanner numberentered; static scanner userinput = new scanner(system.in); static int guessednumber; static int randomnumber = (int) (math.random()* 11); static scanner reply; static string answer; public static void main(string[] args) { guesschecker(guessednumber); } public static void guesschecker(int userguess) { while (userguess != randomnumber) { intro(); userguess = intchecker(); if (userguess == randomnumber) { system.out.println("congradulations!"); system.exit(0); } else { system.out.println("that incorrect!");...
Read more