[Postgres-xl-general] "ERROR: Failed to get pooled connections" with cross-node join
Tobias Oberstein
tobias.oberstein at gmail.com
Thu Oct 1 13:32:08 PDT 2015
> Can you do a check on how many backends are running on the machine,
> summed across all the datanodes/coordinators? Since you are not using
I'll do tomorrow.
> gtm_proxy, you might be be hitting the compile time limit of 1024 GTM
> threads as GTM spawns a one thread per connection. GTM proxy is supposed
"one thread per connection": huh, ok. IMO, this is a serious design
flaw. It limits scalability, induces a lot of overhead and is uneeded?
Why not using a proper asynchronous approach (epoll, kqueue, libuv,
libev, .. ) and 1 thread? As long as the GTM isn't inherently CPU bound
(which I can't believe), it'll be less pain, more performance.
But actually I have very little clue about XL right now, so I might be
wrong because there are other requirements.
> to reduce that number (and also messages exchanged) by grouping and
> multiplexing messages from various backends to a single (or a few)
> connection(s) to GTM.
I see. I will try GTM proxy tomorrow.
Sidenote: If the whole reason of GTM proxy is to reduce the number of
connections to GTM to address the issue GTM is unable to cope with high
connection numbers (thread-per-connection model), the solution might be
to get rid of GTM proxy and move to asynchronous networking. 1k
connections is nothing. A single thread can easily do 100k and more
connections.
But in any case: thanks a lot for taking notice and for all your
information and help! (I will contact you offlist).
Cheers,
/Tobias
>
> Thanks,
> Pavan
>
> --
> Pavan Deolasee http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
More information about the postgres-xl-general
mailing list