[Postgres-xl-developers] fix postgres-xl build on FreeBSD

Nikhil Sontakke nikhils at translattice.com
Fri Dec 12 02:53:29 PST 2014


Hi Arvind,

Not that am specifically aware of. It'd be a good idea for users to write
back describing their deployment scenarios to get a flavor of the same,
indeed.

Regards,
Nikhils

On Fri, Dec 12, 2014 at 8:48 AM, Arvind N <arvindat.solaris at gmail.com>
wrote:
>
> hi Nikhil,
>               Is there any condensed data on how many deployments of XL
> exist which hold data like 5 to 10 tera bytes ? .
>
> Also I guess this question is for the community, are there any active
> FreeBSD deployments of the above scale ?
>
> thanks
> Arvind
>
> On Thu, Dec 11, 2014 at 3:29 PM, Nikhil Sontakke <nikhils at translattice.com
> > wrote:
>
>>
>>
>>
>>> Our company has plan to provide scalable database solution based on
>>> Postgres-XL for Analytics product to be released in Q2-2015.
>>>
>>>
>> +1
>>
>> I recently met representatives from an Indonesian company who are using
>> XL as a data warehousing solution for one of the largest Indonesian banks.
>> They have 5TB of historical data in their XL cluster as of now.
>>
>> Regards,
>> Nikhils
>>
>>
>>
>>
>>> Hi,
>>>
>>> It would be great if you can review patch and provide comments if any.
>>>
>>> Thanks,
>>> Ramanuj
>>>
>>>
>>>
>>> On Wed, Dec 10, 2014 at 9:46 PM, Ramanuj Kumar <hari.ramanuj at gmail.com>
>>> wrote:
>>>
>>>> Hi Jov,
>>>>
>>>> Thanks a lot for the patch. Let me try that out.
>>>>
>>>> Yes, coord log has error message about signal 10 and “bus error”.
>>>>
>>>> —full coord log—
>>>> 2014-12-09 22:52:34 PST [] [1175]: [1-1] user=,db=,e=00000 LOG:
>>>>  database system was shut down at 2014-12-09 22:52:31 PST
>>>> 2014-12-09 22:52:34 PST [] [1170]: [1-1] user=,db=,e=00000 LOG:
>>>>  database system is ready to accept connections
>>>> 2014-12-09 22:52:34 PST [] [1180]: [1-1] user=,db=,e=00000 LOG:
>>>>  autovacuum launcher started
>>>> 2014-12-09 22:52:44 PST [10.102.103.130(19381)] [1348]: [1-1]
>>>> user=pgxl,db=postgres,e=00000 LOG:  statement: ALTER NODE coord1 WITH
>>>> (HOST='10.102.103.130', PORT=11921);
>>>> 2014-12-09 22:52:44 PST [10.102.103.130(19381)] [1348]: [2-1]
>>>> user=pgxl,db=postgres,e=00000 LOG:  duration: 1.465 ms
>>>> 2014-12-09 22:52:44 PST [10.102.103.130(19381)] [1348]: [3-1]
>>>> user=pgxl,db=postgres,e=00000 LOG:  statement: CREATE NODE datanode1 WITH
>>>> (TYPE='datanode', HOST='10.102.103.130', PORT=12921, PRIMARY, PREFERRED);
>>>> 2014-12-09 22:52:44 PST [10.102.103.130(19381)] [1348]: [4-1]
>>>> user=pgxl,db=postgres,e=00000 LOG:  duration: 1.388 ms
>>>> 2014-12-09 22:52:44 PST [10.102.103.130(16075)] [1352]: [1-1]
>>>> user=pgxl,db=postgres,e=00000 LOG:  statement: EXECUTE DIRECT ON
>>>> (datanode1) 'CREATE NODE coord1 WITH (TYPE=''coordinator'',
>>>> HOST=''10.102.103.130'', PORT=1
>>>> 1921)';
>>>> 2014-12-09 22:52:44 PST [10.102.103.130(16075)] [1352]: [2-1]
>>>> user=pgxl,db=postgres,e=00000 LOG:  Connection error could not send data to
>>>> server
>>>> 2014-12-09 22:52:44 PST [10.102.103.130(16075)] [1352]: [3-1]
>>>> user=pgxl,db=postgres,e=00000 STATEMENT:  EXECUTE DIRECT ON (datanode1)
>>>> 'CREATE NODE coord1 WITH (TYPE=''coordinator'', HOST=''10.102.103.130'',
>>>> PORT=11921)
>>>> ';
>>>> 2014-12-09 22:52:44 PST [10.102.103.130(16075)] [1352]: [4-1]
>>>> user=pgxl,db=postgres,e=00000 LOG:  Connection error failed to send data to
>>>> datanode
>>>> 2014-12-09 22:52:44 PST [10.102.103.130(16075)] [1352]: [5-1]
>>>> user=pgxl,db=postgres,e=00000 STATEMENT:  EXECUTE DIRECT ON (datanode1)
>>>> 'CREATE NODE coord1 WITH (TYPE=''coordinator'', HOST=''10.102.103.130'',
>>>> PORT=11921)
>>>> ';
>>>> 2014-12-09 22:52:44 PST [] [1170]: [2-1] *user=,db=,e=00000 LOG:
>>>>  server process (PID 1352) was terminated by signal 10: Bus error*
>>>> 2014-12-09 22:52:44 PST [] [1170]: [3-1] user=,db=,e=00000 DETAIL:
>>>>  Failed process was running: EXECUTE DIRECT ON (datanode1) 'CREATE NODE
>>>> coord1 WITH (TYPE=''coordinator'', HOST=''10.102.103.130'', PORT=11921)';
>>>> 2014-12-09 22:52:44 PST [] [1170]: [4-1] user=,db=,e=00000 LOG:
>>>>  terminating any other active server processes
>>>> 2014-12-09 22:52:44 PST [] [1180]: [2-1] user=,db=,e=57P02 WARNING:
>>>>  terminating connection because of crash of another server process
>>>> 2014-12-09 22:52:44 PST [] [1180]: [3-1] user=,db=,e=57P02 DETAIL:  The
>>>> postmaster has commanded this server process to roll back the current
>>>> transaction and exit, because another server process exited abnormally and p
>>>> ossibly corrupted shared memory.
>>>> 2014-12-09 22:52:44 PST [] [1180]: [4-1] user=,db=,e=57P02 HINT:  In a
>>>> moment you should be able to reconnect to the database and repeat your
>>>> command.
>>>> 2014-12-09 22:52:44 PST [] [1170]: [5-1] user=,db=,e=00000 LOG:  all
>>>> server processes terminated; reinitializing
>>>> 2014-12-09 22:52:44 PST [] [1357]: [1-1] user=,db=,e=00000 LOG:
>>>>  database system was interrupted; last known up at 2014-12-09 22:52:34 PST
>>>> 2014-12-09 22:52:44 PST [] [1357]: [2-1] user=,db=,e=00000 LOG:
>>>>  database system was not properly shut down; automatic recovery in progress
>>>> 2014-12-09 22:52:44 PST [] [1357]: [3-1] user=,db=,e=00000 LOG:  redo
>>>> starts at 0/186B110
>>>> 2014-12-09 22:52:44 PST [] [1357]: [4-1] user=,db=,e=00000 LOG:  record
>>>> with zero length at 0/186B660
>>>> 2014-12-09 22:52:44 PST [] [1357]: [5-1] user=,db=,e=00000 LOG:  redo
>>>> done at 0/186B630
>>>> 2014-12-09 22:52:44 PST [] [1357]: [6-1] user=,db=,e=00000 LOG:  last
>>>> completed transaction was at log time 2014-12-09 22:52:44.820342-08
>>>> 2014-12-09 22:52:44 PST [] [1361]: [1-1] user=,db=,e=00000 LOG:
>>>>  autovacuum launcher started
>>>> 2014-12-09 22:52:44 PST [] [1170]: [6-1] user=,db=,e=00000 LOG:
>>>>  database system is ready to accept connections
>>>> —————
>>>>
>>>> Thanks,
>>>> Ramanuj
>>>>
>>>>
>>>>
>>>>
>>>> On 10-Dec-2014, at 9:41 pm, Jov <amutu at amutu.com> wrote:
>>>>
>>>> the whole patch:
>>>> http://amutu.com/blog/public_stage/xl_freebsd.patch
>>>>
>>>> this patch include the above patch,and also fix the get/send invalid
>>>> fds from pooler process,and some new dtrace probe.
>>>>
>>>> you can check the coord logs,and find if there are some error message
>>>> about "signal 11",or "bus error"
>>>>
>>>>
>>>> Jov
>>>> blog: http:amutu.com/blog <http://amutu.com/blog>
>>>>
>>>> 2014-12-10 19:53 GMT+08:00 Ramanuj Hari <hari.ramanuj at gmail.com>:
>>>>
>>>>> Hi Jov,
>>>>>
>>>>> It would be great, if you could please share your other patch with me.
>>>>> I will try if that works on my environment.
>>>>>
>>>>> Thanks,
>>>>> Ramanuj
>>>>>
>>>>>
>>>>> On Wed, Dec 10, 2014 at 3:52 PM, Jov <amutu at amutu.com> wrote:
>>>>>
>>>>>> have you test with the patch above?I install pgxl with pgxc_ctl.
>>>>>> I run PG-XL on FreeBSD with some more patch.The patch above can only
>>>>>> make pgxc_ctl install and init cluster,but when you run SQL,there will be
>>>>>> problem.I have solve the problem,and wait for the above patch accepting and
>>>>>> then will submit the other patch.
>>>>>>
>>>>>> for install and init pgxl,there are 2 problems for FreeBSD user to
>>>>>> solve:
>>>>>> 1.install the bash shell and apply the above patch.
>>>>>> 2.set remote env for ssh.you should add a file named environment in
>>>>>> the ~/.ssh dir with PATH set to pgxl bin dir.
>>>>>>
>>>>>> Jov
>>>>>> blog: http:amutu.com/blog <http://amutu.com/blog>
>>>>>>
>>>>>> 2014-12-10 17:56 GMT+08:00 Ramanuj Hari <hari.ramanuj at gmail.com>:
>>>>>>
>>>>>>> Hi Jov,
>>>>>>>
>>>>>>> I am also trying to run on FreeBSD r10. But got into issue.
>>>>>>>
>>>>>>> ----
>>>>>>> do_shell.c:dump_cmdList(713) *** cmdList Dump
>>>>>>> *******************************
>>>>>>> allocated = 2, used = 1
>>>>>>> do_shell.c:dump_cmdList(723) === CMD: 0 ===
>>>>>>> do_shell.c:dump_cmdList(727)    --- CMD-EL: 0:host="NULL",
>>>>>>> command="psql -p 11921 -h 10.102.103.135 -a postgres pgxl",
>>>>>>> localStdin="/tmp/STDIN_48989_34", localStdout="NULL"
>>>>>>> EXECUTE DIRECT ON (datanode1) 'CREATE NODE coord1 WITH
>>>>>>> (TYPE=''coordinator'', HOST=''10.102.103.135'', PORT=11921)';
>>>>>>> EXECUTE DIRECT ON (datanode1) 'ALTER NODE datanode1 WITH
>>>>>>> (TYPE=''datanode'', HOST=''10.102.103.135'', PORT=12921, PRIMARY,
>>>>>>> PREFERRED)';
>>>>>>> EXECUTE DIRECT ON (datanode1) 'SELECT pgxc_pool_reload()';
>>>>>>> do_shell.c:dump_cmdList(737)    ----------
>>>>>>> do_shell.c:doCmdEl(360) Local command: "psql -p 11921 -h
>>>>>>> 10.102.103.135 -a postgres pgxl", actual: "( psql -p 11921 -h
>>>>>>> 10.102.103.135 -a postgres pgxl ) > /tmp/STDOUT_48989_35 2>&1 <
>>>>>>> /tmp/STDIN_48989_34"
>>>>>>> EXECUTE DIRECT ON (datanode1) 'CREATE NODE coord1 WITH
>>>>>>> (TYPE=''coordinator'', HOST=''10.102.103.135'', PORT=11921)';
>>>>>>> ERROR:  Could not begin transaction on Datanodes
>>>>>>> EXECUTE DIRECT ON (datanode1) 'ALTER NODE datanode1 WITH
>>>>>>> (TYPE=''datanode'', HOST=''10.102.103.135'', PORT=12921, PRIMARY,
>>>>>>> PREFERRED)';
>>>>>>> ERROR:  Could not begin transaction on Datanodes
>>>>>>> EXECUTE DIRECT ON (datanode1) 'SELECT pgxc_pool_reload()';
>>>>>>> ERROR:  Could not begin transaction on data node.
>>>>>>> coord_cmd.c:configure_datanodes(416) Done.
>>>>>>> PGXC$
>>>>>>> -----
>>>>>>>
>>>>>>> Could you please help how did you try, whether it is pgxc_ctl or
>>>>>>> manual configuration.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ramanuj
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Oct 15, 2014 at 7:21 AM, Jov <amutu at amutu.com> wrote:
>>>>>>>
>>>>>>>> v2 change:
>>>>>>>> replace "#!/bin/bash" with "#!/usr/bin/env bash" in these files:
>>>>>>>> contrib/pgxc_ctl/pgxc_ctl.bash
>>>>>>>> contrib/pgxc_ctl/pgxc_ctl_bash.c
>>>>>>>> contrib/pgxc_ctl/pgxc_ctl_bash.org
>>>>>>>> contrib/pgxc_ctl/pgxc_ctl_bash_2
>>>>>>>> contrib/pgxc_ctl/pgxc_ctl_conf_part
>>>>>>>>
>>>>>>>> I am success to setup and init pgxl cluster after this patch on
>>>>>>>> FreeBSD r10 amd64
>>>>>>>>
>>>>>>>> Jov
>>>>>>>> blog: http:amutu.com/blog <http://amutu.com/blog>
>>>>>>>>
>>>>>>>> 2014-10-15 5:52 GMT+08:00 Jov <amutu at amutu.com>:
>>>>>>>>
>>>>>>>>> Thanks.I find some runtime problem when running pgxc_ctl,I will
>>>>>>>>> submit a updated patch later.
>>>>>>>>> 2014年10月15日 4:52 AM于 "Mason Sharp" <msharp at translattice.com>写道:
>>>>>>>>>
>>>>>>>>> Hi Jov,
>>>>>>>>>>
>>>>>>>>>> Will try to take a closer look soon.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Mason
>>>>>>>>>>
>>>>>>>>>> On Mon, Oct 13, 2014 at 2:50 AM, Jov <amutu at amutu.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> any idea?
>>>>>>>>>>>
>>>>>>>>>>> Jov
>>>>>>>>>>> blog: http:amutu.com/blog <http://amutu.com/blog>
>>>>>>>>>>>
>>>>>>>>>>> 2014-10-11 17:41 GMT+08:00 赵伟 <amutu at amutu.com>:
>>>>>>>>>>>
>>>>>>>>>>>> when build pgxl on FreeBSD,there are 3 problems:
>>>>>>>>>>>> 1."WEXITSTATUS(status) " macro can't be resolve when compile
>>>>>>>>>>>> src/gtm/gtm_ctl/gtm_ctl.c ,caused by lacking of sys/wait.h header.
>>>>>>>>>>>>
>>>>>>>>>>>> 2.type "pid_t" undefined when compile
>>>>>>>>>>>> contrib/pgxc_ctl/utils.h,caused by lacking of sys/types.h header
>>>>>>>>>>>>
>>>>>>>>>>>> 3.contrib/pgxc_ctl/make_signature bash can't be running bacause
>>>>>>>>>>>> of it think the bash is installed at /bin/bash,which is not true on
>>>>>>>>>>>> FreeBSD.replace with "#!/usr/bin/env bash" solve the problem.
>>>>>>>>>>>>
>>>>>>>>>>>> patch attached.
>>>>>>>>>>>>
>>>>>>>>>>>> thanks,
>>>>>>>>>>>> Jov
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>>>>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
>>>>>>>>>>>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI
>>>>>>>>>>>> DSS Reports
>>>>>>>>>>>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White
>>>>>>>>>>>> paper
>>>>>>>>>>>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog
>>>>>>>>>>>> Analyzer
>>>>>>>>>>>> http://p.sf.net/sfu/Zoho
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Postgres-xl-developers mailing list
>>>>>>>>>>>> Postgres-xl-developers at lists.sourceforge.net
>>>>>>>>>>>>
>>>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xl-developers
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>>>>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
>>>>>>>>>>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS
>>>>>>>>>>> Reports
>>>>>>>>>>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White
>>>>>>>>>>> paper
>>>>>>>>>>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog
>>>>>>>>>>> Analyzer
>>>>>>>>>>> http://p.sf.net/sfu/Zoho
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Postgres-xl-developers mailing list
>>>>>>>>>>> Postgres-xl-developers at lists.sourceforge.net
>>>>>>>>>>>
>>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xl-developers
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Mason Sharp
>>>>>>>>>>
>>>>>>>>>> TransLattice - http://www.translattice.com
>>>>>>>>>> Clustered and Distributed Database Solutions
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> Comprehensive Server Monitoring with Site24x7.
>>>>>>>> Monitor 10 servers for $9/Month.
>>>>>>>> Get alerted through email, SMS, voice calls or mobile push
>>>>>>>> notifications.
>>>>>>>> Take corrective actions from your mobile device.
>>>>>>>> http://p.sf.net/sfu/Zoho
>>>>>>>> _______________________________________________
>>>>>>>> Postgres-xl-developers mailing list
>>>>>>>> Postgres-xl-developers at lists.sourceforge.net
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xl-developers
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>>> Get technology previously reserved for billion-dollar corporations, FREE
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Postgres-xl-developers mailing list
>>> Postgres-xl-developers at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xl-developers
>>>
>>>
>>
>>
>> --
>> TransLattice - http://www.translattice.com
>> Distributed and Clustered Database Solutions
>> Postgres-XL Support
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>> Get technology previously reserved for billion-dollar corporations, FREE
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Postgres-xl-developers mailing list
>> Postgres-xl-developers at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/postgres-xl-developers
>>
>>
>

-- 
TransLattice - http://www.translattice.com
Distributed and Clustered Database Solutions
Postgres-XL Support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.postgres-xl.org/private.cgi/postgres-xl-developers-postgres-xl.org/attachments/20141212/bbfe054f/attachment.htm>


More information about the Postgres-xl-developers mailing list