[Postgres-xl-developers] pg_bulkupload

Mason S masonlists at gmail.com
Sat Mar 21 05:12:13 PDT 2015


On Tue, Mar 17, 2015 at 10:50 AM, varunshaji <varunshaji at gmail.com> wrote:

> Hi ,
>
> We are trying to load data into a cluster 4nodes lectures (1CN,3DN) . Its
> a huge data in the tunes of terabytes and hence decided to use pg_bulkload.
> We were able to build and install the extension. But when we try to load
> data..
>
> *execute <unnamed>: SELECT * FROM pg_bulkload($1)*
> *DETAIL:  parameters: $1 =
> '{"infile=/home/qwewqe/qweqwe/wewqe/scripts/Test
> Cases/qweqwe/test10.csv",output=rdw.test*
>
> *4:37:44 write(2, "NOTICE: BULK LOAD END\n\t0 Rows sk"..., 191NOTICE: BULK
> LOAD END*
> * 0 Rows skipped.*
> * 87600096 Rows successfully loaded.*
> * 0 Rows not loaded due to parse errors.*
> * 0 Rows not loaded due to duplicate errors.*
> * 0 Rows replaced with new rows.*
>
> But when we check the table rdw.test , it shows no data . tried all
> combination of Direct load, BUFFERED LOAD and Parallel load still it doest
> insert data.
> However the utility always logs back success status and says data has been
> entered. When i check the data folder of coordinator i could find some I/O
> happening.
>
> Could some one throw some light on how to proceed.  Anything that could
> help me in fixing this would be highly appreciated. I am also running out
> of time, so want to decide whether
> tinkering with bulk upload on postgresql-xl solves my issue or should i go
> with copy command.
>
>
> Thanks for reporting this. There are many tools in the greater PostgreSQL
community that have not been tested with Postgres-XL.  We can try to look
when we can, otherwise, you may want to hire someone from one of the
Postgres-XL/PostgreSQL consulting companies to investigate if it is urgent.
In the meantime, you could use the COPY command, and perhaps kick off
multiple instances in parallel, splitting any particular large input files.

Regards,

Mason
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.postgres-xl.org/private.cgi/postgres-xl-developers-postgres-xl.org/attachments/20150321/3ce1ad91/attachment.htm>


More information about the Postgres-xl-developers mailing list