Connection issues with Postgres flexible server & PGBouncer

Our API (sensotec-app-rekenapps-api-productie) has been having a lot of issues with connection issues leading to very disgruntled end-users and a steady flow of users abandoning the product.
This API connects to a PostgreSQL database (sensotec-psql-rekenapps-production) and it's there that the problems start.

Each morning thousands of people log in and around 9 am the first performance issues show up. Response times go from milliseconds to seconds to minutes, users start to hit Refresh and the system slows down even more. Initially we used the instance connection pools but then we switched to the builtin PGBouncer. We started with Pooling=false and a fairly large PGBouncer pool (10k on Friday) but we keep seeing errors : too many clients and transient network failures. Assuming the network failures are a consequence of the max clients being reached we have done the following:

Increase PGBouncer connections was set to 50k, default pool size to 300.
DB Connections now have Pooling=True
Maximim Pool Size = 20
Minimum Pool Size = 2
Connection Idle Lifetime = 300
Connection Pruning interval = 10

Few people use the application during the weekend so the real test will be on Monday.

Can somebody assist us with checking if this configuration is best practice? (Some articles on MSDN recommend Pooling=False).
Can somebody check the metrics on Monday and help us determine if there is another underlying cause that creates these problems?
Kind regards,
William