[Postgres-xl-general] "ERROR: Failed to get pooled connections" with cross-node join

Tobias Oberstein tobias.oberstein at gmail.com
Thu Oct 1 13:32:08 PDT 2015


> Can you do a check on how many backends are running on the machine,
> summed across all the datanodes/coordinators? Since you are not using

I'll do tomorrow.

> gtm_proxy, you might be be hitting the compile time limit of 1024 GTM
> threads as GTM spawns a one thread per connection. GTM proxy is supposed

"one thread per connection": huh, ok. IMO, this is a serious design 
flaw. It limits scalability, induces a lot of overhead and is uneeded? 
Why not using a proper asynchronous approach (epoll, kqueue, libuv, 
libev, .. ) and 1 thread? As long as the GTM isn't inherently CPU bound 
(which I can't believe), it'll be less pain, more performance.

But actually I have very little clue about XL right now, so I might be 
wrong because there are other requirements.

> to reduce that number (and also messages exchanged) by grouping and
> multiplexing messages from various backends to a single (or a few)
> connection(s) to GTM.

I see. I will try GTM proxy tomorrow.

Sidenote: If the whole reason of GTM proxy is to reduce the number of 
connections to GTM to address the issue GTM is unable to cope with high 
connection numbers (thread-per-connection model), the solution might be 
to get rid of GTM proxy and move to asynchronous networking. 1k 
connections is nothing. A single thread can easily do 100k and more 
connections.

But in any case: thanks a lot for taking notice and for all your 
information and help! (I will contact you offlist).

Cheers,
/Tobias

>
> Thanks,
> Pavan
>
> --
>   Pavan Deolasee http://www.2ndQuadrant.com/
>   PostgreSQL Development, 24x7 Support, Training & Services





More information about the postgres-xl-general mailing list