[Postgres-xl-developers] fix postgres-xl build on FreeBSD

Nikhil Sontakke nikhils at translattice.com
Thu Dec 11 01:59:30 PST 2014


> Our company has plan to provide scalable database solution based on
> Postgres-XL for Analytics product to be released in Q2-2015.
>
>
+1

I recently met representatives from an Indonesian company who are using XL
as a data warehousing solution for one of the largest Indonesian banks.
They have 5TB of historical data in their XL cluster as of now.

Regards,
Nikhils




> Hi,
>
> It would be great if you can review patch and provide comments if any.
>
> Thanks,
> Ramanuj
>
>
>
> On Wed, Dec 10, 2014 at 9:46 PM, Ramanuj Kumar <hari.ramanuj at gmail.com>
> wrote:
>
>> Hi Jov,
>>
>> Thanks a lot for the patch. Let me try that out.
>>
>> Yes, coord log has error message about signal 10 and “bus error”.
>>
>> —full coord log—
>> 2014-12-09 22:52:34 PST [] [1175]: [1-1] user=,db=,e=00000 LOG:  database
>> system was shut down at 2014-12-09 22:52:31 PST
>> 2014-12-09 22:52:34 PST [] [1170]: [1-1] user=,db=,e=00000 LOG:  database
>> system is ready to accept connections
>> 2014-12-09 22:52:34 PST [] [1180]: [1-1] user=,db=,e=00000 LOG:
>>  autovacuum launcher started
>> 2014-12-09 22:52:44 PST [10.102.103.130(19381)] [1348]: [1-1]
>> user=pgxl,db=postgres,e=00000 LOG:  statement: ALTER NODE coord1 WITH
>> (HOST='10.102.103.130', PORT=11921);
>> 2014-12-09 22:52:44 PST [10.102.103.130(19381)] [1348]: [2-1]
>> user=pgxl,db=postgres,e=00000 LOG:  duration: 1.465 ms
>> 2014-12-09 22:52:44 PST [10.102.103.130(19381)] [1348]: [3-1]
>> user=pgxl,db=postgres,e=00000 LOG:  statement: CREATE NODE datanode1 WITH
>> (TYPE='datanode', HOST='10.102.103.130', PORT=12921, PRIMARY, PREFERRED);
>> 2014-12-09 22:52:44 PST [10.102.103.130(19381)] [1348]: [4-1]
>> user=pgxl,db=postgres,e=00000 LOG:  duration: 1.388 ms
>> 2014-12-09 22:52:44 PST [10.102.103.130(16075)] [1352]: [1-1]
>> user=pgxl,db=postgres,e=00000 LOG:  statement: EXECUTE DIRECT ON
>> (datanode1) 'CREATE NODE coord1 WITH (TYPE=''coordinator'',
>> HOST=''10.102.103.130'', PORT=1
>> 1921)';
>> 2014-12-09 22:52:44 PST [10.102.103.130(16075)] [1352]: [2-1]
>> user=pgxl,db=postgres,e=00000 LOG:  Connection error could not send data to
>> server
>> 2014-12-09 22:52:44 PST [10.102.103.130(16075)] [1352]: [3-1]
>> user=pgxl,db=postgres,e=00000 STATEMENT:  EXECUTE DIRECT ON (datanode1)
>> 'CREATE NODE coord1 WITH (TYPE=''coordinator'', HOST=''10.102.103.130'',
>> PORT=11921)
>> ';
>> 2014-12-09 22:52:44 PST [10.102.103.130(16075)] [1352]: [4-1]
>> user=pgxl,db=postgres,e=00000 LOG:  Connection error failed to send data to
>> datanode
>> 2014-12-09 22:52:44 PST [10.102.103.130(16075)] [1352]: [5-1]
>> user=pgxl,db=postgres,e=00000 STATEMENT:  EXECUTE DIRECT ON (datanode1)
>> 'CREATE NODE coord1 WITH (TYPE=''coordinator'', HOST=''10.102.103.130'',
>> PORT=11921)
>> ';
>> 2014-12-09 22:52:44 PST [] [1170]: [2-1] *user=,db=,e=00000 LOG:  server
>> process (PID 1352) was terminated by signal 10: Bus error*
>> 2014-12-09 22:52:44 PST [] [1170]: [3-1] user=,db=,e=00000 DETAIL:
>>  Failed process was running: EXECUTE DIRECT ON (datanode1) 'CREATE NODE
>> coord1 WITH (TYPE=''coordinator'', HOST=''10.102.103.130'', PORT=11921)';
>> 2014-12-09 22:52:44 PST [] [1170]: [4-1] user=,db=,e=00000 LOG:
>>  terminating any other active server processes
>> 2014-12-09 22:52:44 PST [] [1180]: [2-1] user=,db=,e=57P02 WARNING:
>>  terminating connection because of crash of another server process
>> 2014-12-09 22:52:44 PST [] [1180]: [3-1] user=,db=,e=57P02 DETAIL:  The
>> postmaster has commanded this server process to roll back the current
>> transaction and exit, because another server process exited abnormally and p
>> ossibly corrupted shared memory.
>> 2014-12-09 22:52:44 PST [] [1180]: [4-1] user=,db=,e=57P02 HINT:  In a
>> moment you should be able to reconnect to the database and repeat your
>> command.
>> 2014-12-09 22:52:44 PST [] [1170]: [5-1] user=,db=,e=00000 LOG:  all
>> server processes terminated; reinitializing
>> 2014-12-09 22:52:44 PST [] [1357]: [1-1] user=,db=,e=00000 LOG:  database
>> system was interrupted; last known up at 2014-12-09 22:52:34 PST
>> 2014-12-09 22:52:44 PST [] [1357]: [2-1] user=,db=,e=00000 LOG:  database
>> system was not properly shut down; automatic recovery in progress
>> 2014-12-09 22:52:44 PST [] [1357]: [3-1] user=,db=,e=00000 LOG:  redo
>> starts at 0/186B110
>> 2014-12-09 22:52:44 PST [] [1357]: [4-1] user=,db=,e=00000 LOG:  record
>> with zero length at 0/186B660
>> 2014-12-09 22:52:44 PST [] [1357]: [5-1] user=,db=,e=00000 LOG:  redo
>> done at 0/186B630
>> 2014-12-09 22:52:44 PST [] [1357]: [6-1] user=,db=,e=00000 LOG:  last
>> completed transaction was at log time 2014-12-09 22:52:44.820342-08
>> 2014-12-09 22:52:44 PST [] [1361]: [1-1] user=,db=,e=00000 LOG:
>>  autovacuum launcher started
>> 2014-12-09 22:52:44 PST [] [1170]: [6-1] user=,db=,e=00000 LOG:  database
>> system is ready to accept connections
>> —————
>>
>> Thanks,
>> Ramanuj
>>
>>
>>
>>
>> On 10-Dec-2014, at 9:41 pm, Jov <amutu at amutu.com> wrote:
>>
>> the whole patch:
>> http://amutu.com/blog/public_stage/xl_freebsd.patch
>>
>> this patch include the above patch,and also fix the get/send invalid fds
>> from pooler process,and some new dtrace probe.
>>
>> you can check the coord logs,and find if there are some error message
>> about "signal 11",or "bus error"
>>
>>
>> Jov
>> blog: http:amutu.com/blog <http://amutu.com/blog>
>>
>> 2014-12-10 19:53 GMT+08:00 Ramanuj Hari <hari.ramanuj at gmail.com>:
>>
>>> Hi Jov,
>>>
>>> It would be great, if you could please share your other patch with me. I
>>> will try if that works on my environment.
>>>
>>> Thanks,
>>> Ramanuj
>>>
>>>
>>> On Wed, Dec 10, 2014 at 3:52 PM, Jov <amutu at amutu.com> wrote:
>>>
>>>> have you test with the patch above?I install pgxl with pgxc_ctl.
>>>> I run PG-XL on FreeBSD with some more patch.The patch above can only
>>>> make pgxc_ctl install and init cluster,but when you run SQL,there will be
>>>> problem.I have solve the problem,and wait for the above patch accepting and
>>>> then will submit the other patch.
>>>>
>>>> for install and init pgxl,there are 2 problems for FreeBSD user to
>>>> solve:
>>>> 1.install the bash shell and apply the above patch.
>>>> 2.set remote env for ssh.you should add a file named environment in
>>>> the ~/.ssh dir with PATH set to pgxl bin dir.
>>>>
>>>> Jov
>>>> blog: http:amutu.com/blog <http://amutu.com/blog>
>>>>
>>>> 2014-12-10 17:56 GMT+08:00 Ramanuj Hari <hari.ramanuj at gmail.com>:
>>>>
>>>>> Hi Jov,
>>>>>
>>>>> I am also trying to run on FreeBSD r10. But got into issue.
>>>>>
>>>>> ----
>>>>> do_shell.c:dump_cmdList(713) *** cmdList Dump
>>>>> *******************************
>>>>> allocated = 2, used = 1
>>>>> do_shell.c:dump_cmdList(723) === CMD: 0 ===
>>>>> do_shell.c:dump_cmdList(727)    --- CMD-EL: 0:host="NULL",
>>>>> command="psql -p 11921 -h 10.102.103.135 -a postgres pgxl",
>>>>> localStdin="/tmp/STDIN_48989_34", localStdout="NULL"
>>>>> EXECUTE DIRECT ON (datanode1) 'CREATE NODE coord1 WITH
>>>>> (TYPE=''coordinator'', HOST=''10.102.103.135'', PORT=11921)';
>>>>> EXECUTE DIRECT ON (datanode1) 'ALTER NODE datanode1 WITH
>>>>> (TYPE=''datanode'', HOST=''10.102.103.135'', PORT=12921, PRIMARY,
>>>>> PREFERRED)';
>>>>> EXECUTE DIRECT ON (datanode1) 'SELECT pgxc_pool_reload()';
>>>>> do_shell.c:dump_cmdList(737)    ----------
>>>>> do_shell.c:doCmdEl(360) Local command: "psql -p 11921 -h
>>>>> 10.102.103.135 -a postgres pgxl", actual: "( psql -p 11921 -h
>>>>> 10.102.103.135 -a postgres pgxl ) > /tmp/STDOUT_48989_35 2>&1 <
>>>>> /tmp/STDIN_48989_34"
>>>>> EXECUTE DIRECT ON (datanode1) 'CREATE NODE coord1 WITH
>>>>> (TYPE=''coordinator'', HOST=''10.102.103.135'', PORT=11921)';
>>>>> ERROR:  Could not begin transaction on Datanodes
>>>>> EXECUTE DIRECT ON (datanode1) 'ALTER NODE datanode1 WITH
>>>>> (TYPE=''datanode'', HOST=''10.102.103.135'', PORT=12921, PRIMARY,
>>>>> PREFERRED)';
>>>>> ERROR:  Could not begin transaction on Datanodes
>>>>> EXECUTE DIRECT ON (datanode1) 'SELECT pgxc_pool_reload()';
>>>>> ERROR:  Could not begin transaction on data node.
>>>>> coord_cmd.c:configure_datanodes(416) Done.
>>>>> PGXC$
>>>>> -----
>>>>>
>>>>> Could you please help how did you try, whether it is pgxc_ctl or
>>>>> manual configuration.
>>>>>
>>>>> Thanks,
>>>>> Ramanuj
>>>>>
>>>>>
>>>>> On Wed, Oct 15, 2014 at 7:21 AM, Jov <amutu at amutu.com> wrote:
>>>>>
>>>>>> v2 change:
>>>>>> replace "#!/bin/bash" with "#!/usr/bin/env bash" in these files:
>>>>>> contrib/pgxc_ctl/pgxc_ctl.bash
>>>>>> contrib/pgxc_ctl/pgxc_ctl_bash.c
>>>>>> contrib/pgxc_ctl/pgxc_ctl_bash.org
>>>>>> contrib/pgxc_ctl/pgxc_ctl_bash_2
>>>>>> contrib/pgxc_ctl/pgxc_ctl_conf_part
>>>>>>
>>>>>> I am success to setup and init pgxl cluster after this patch on
>>>>>> FreeBSD r10 amd64
>>>>>>
>>>>>> Jov
>>>>>> blog: http:amutu.com/blog <http://amutu.com/blog>
>>>>>>
>>>>>> 2014-10-15 5:52 GMT+08:00 Jov <amutu at amutu.com>:
>>>>>>
>>>>>>> Thanks.I find some runtime problem when running pgxc_ctl,I will
>>>>>>> submit a updated patch later.
>>>>>>> 2014年10月15日 4:52 AM于 "Mason Sharp" <msharp at translattice.com>写道:
>>>>>>>
>>>>>>> Hi Jov,
>>>>>>>>
>>>>>>>> Will try to take a closer look soon.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Mason
>>>>>>>>
>>>>>>>> On Mon, Oct 13, 2014 at 2:50 AM, Jov <amutu at amutu.com> wrote:
>>>>>>>>
>>>>>>>>> any idea?
>>>>>>>>>
>>>>>>>>> Jov
>>>>>>>>> blog: http:amutu.com/blog <http://amutu.com/blog>
>>>>>>>>>
>>>>>>>>> 2014-10-11 17:41 GMT+08:00 赵伟 <amutu at amutu.com>:
>>>>>>>>>
>>>>>>>>>> when build pgxl on FreeBSD,there are 3 problems:
>>>>>>>>>> 1."WEXITSTATUS(status) " macro can't be resolve when compile
>>>>>>>>>> src/gtm/gtm_ctl/gtm_ctl.c ,caused by lacking of sys/wait.h header.
>>>>>>>>>>
>>>>>>>>>> 2.type "pid_t" undefined when compile
>>>>>>>>>> contrib/pgxc_ctl/utils.h,caused by lacking of sys/types.h header
>>>>>>>>>>
>>>>>>>>>> 3.contrib/pgxc_ctl/make_signature bash can't be running bacause
>>>>>>>>>> of it think the bash is installed at /bin/bash,which is not true on
>>>>>>>>>> FreeBSD.replace with "#!/usr/bin/env bash" solve the problem.
>>>>>>>>>>
>>>>>>>>>> patch attached.
>>>>>>>>>>
>>>>>>>>>> thanks,
>>>>>>>>>> Jov
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
>>>>>>>>>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS
>>>>>>>>>> Reports
>>>>>>>>>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White
>>>>>>>>>> paper
>>>>>>>>>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog
>>>>>>>>>> Analyzer
>>>>>>>>>> http://p.sf.net/sfu/Zoho
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Postgres-xl-developers mailing list
>>>>>>>>>> Postgres-xl-developers at lists.sourceforge.net
>>>>>>>>>>
>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xl-developers
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
>>>>>>>>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS
>>>>>>>>> Reports
>>>>>>>>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White
>>>>>>>>> paper
>>>>>>>>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog
>>>>>>>>> Analyzer
>>>>>>>>> http://p.sf.net/sfu/Zoho
>>>>>>>>> _______________________________________________
>>>>>>>>> Postgres-xl-developers mailing list
>>>>>>>>> Postgres-xl-developers at lists.sourceforge.net
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xl-developers
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mason Sharp
>>>>>>>>
>>>>>>>> TransLattice - http://www.translattice.com
>>>>>>>> Clustered and Distributed Database Solutions
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Comprehensive Server Monitoring with Site24x7.
>>>>>> Monitor 10 servers for $9/Month.
>>>>>> Get alerted through email, SMS, voice calls or mobile push
>>>>>> notifications.
>>>>>> Take corrective actions from your mobile device.
>>>>>> http://p.sf.net/sfu/Zoho
>>>>>> _______________________________________________
>>>>>> Postgres-xl-developers mailing list
>>>>>> Postgres-xl-developers at lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xl-developers
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xl-developers mailing list
> Postgres-xl-developers at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/postgres-xl-developers
>
>


-- 
TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions
Postgres-XL Support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.postgres-xl.org/private.cgi/postgres-xl-developers-postgres-xl.org/attachments/20141211/5fe71108/attachment.htm>


More information about the Postgres-xl-developers mailing list