| Author |
| |
Messages |
Sort: |
|
MichaelWayneHarwood
Intermediate Member
User Online: 
|
|
Posted:23/03/2006 4:36 PM |
|
|
Hey all,
We are seeing serious lag times in the AAC login process. Can anyone suggest some good places to start looking for what the bottlenecks are? The most obvious ones are not an issue (CPU, disk I/O, etc) but I am wondering if the complex Active Directory configuration we are using is contributing to the problem. I have seen other posts suggesting this could be a problem, but I have not seen it officially documented anywhere not have I been able to determine if any patches or hotfixes addressing these types of issues have been incorporated into v4.2.1....
Can anyone help?
|
|
SteveMcG
SmartAccessCentral Moderator
User Online:
|
|
Posted:23/03/2006 10:20 PM |
|
|
Hi Michael,
Could you explain your current infrastructure/configuration please? It helps understand where your delay is coming from.
Also you might want to look at using IE Inspector in your client browser http://www.ieinspector.com/ which will show you if you have unnecessary components downloading at logon, eg ICA client, Live edit etc.which can cause delays.
Many thanks
|
|
MichaelWayneHarwood
Intermediate Member
User Online:
|
|
Posted:27/03/2006 9:57 PM |
|
We have devices in two separate networks separated by a firewall - one is a DMZ and the other our "internal" network. The CAG and AAC machines are in the DMZ while all other servers (e.g. Citrix licensing server, AD domain controllers, Citrix SQL server, etc) are in the internal network.
What we have seen is that right after a reboot of the AAC and CAG servers everything runs as expected. After a period of time we start to see performance lag and we usually find that one of our two AAC servers is causing the issue, and after a reboot the issue is gone. We have also seen that the CAG appliances contribute to the sluggishness and the same fix seems to correct the problems for a time.
What I was hoping to hear what that someone else is also seeing similar issues and perhaps provide me with some obvious (or not so obvious) things to check.
It doesn't look like there are any bottlenecks I can detect via perfmon (e.g. disk queues, RAM, CPU, network, etc) when it is happening.
|
|
MichaelWayneHarwood
Intermediate Member
User Online:
|
|
Posted:03/04/2006 9:21 PM |
|
We have identified one of the bottlenecks - Active Directory. By
using a local machine account we saw login times go from an avergae of
45 seconds to 10 seconds. We have a support call logged with
Citrix and have been escalated to their "offshore" support.
I have seen a post on this forum as well as the Citrix forums from
someone else having a similar issue. Citrix custom coded a hotfix
for that user and it was suggested that the hotfix would make it's way
into the public's eyes or be rolled up into Citrix'
codebase. Does anyone know if the latest versions off AAC
has this code included?
|
|
icebun
New Member
User Online:
|
|
Posted:11/04/2006 8:28 AM |
|
|
Hi,
I too am experiencing delays both in the time it takes to retrieve the LogonPoint page and from LogonPoint to Navigation page following Authentication.
The CAG is on the DMZ with a mapped private IP address via our Netscreen firewall.
The AAC server is on the LAN running as a VM.
I have resorted to connecting both INT0 and INT1 and my users have said that performance is much better. This is still taking around 2-3 mins.
If there is a connection with AD, I would love to know resolution fix.
Thanks.
|
|
MichaelWayneHarwood
Intermediate Member
User Online:
|
|
Posted:14/04/2006 2:32 PM |
|
icebun - Have you tried running the AAC on a stand alone (non-virtual) server?
|
|
MichaelWayneHarwood
Intermediate Member
User Online:
|
|
Posted:14/04/2006 2:53 PM |
|
Discovery! We have finally determined the root causes for our performance issues and we are working with Citrix directly on the performance issues that are due to the way the AAC works underneath the hood.
We had a variety
of things happening that when experienced in combination decreased login performance significantly. We
make it a practice to hardcode RPC endpoint ports to a specific range on our AD domain controllers
so that we can better manage what ports need to be opened for applications on the other side of firewalls, but we
discovered that 2 DC’s had not been configured properly and so the AAC was
spending a lot of time attempting to communicate via ports that were "closed" on our firewall. Correcting this decreased the logon times by 50%.
Our AAC servers each have 2 NIC’s though we have 1 of them disabled. We
discovered that somehow the binding order of the NIC’s was configured so that
the disabled NIC was at the top of the list. We didn't see any perfornace increase with this, but having a disabled NIC at the top of the binding list seems counter productive.
The biggest performance issues have yet to be addressed and are being caused by the number of web resources we have configured. The AAC server's processors are getting quite a workout due to the way the web proxy's engine processes pages that it serves. We are proxying a Plone content management system and my understanding is that the AAC proxy is having to examine and re-write stylesheets, code, links, etc that are being presented to the end users. The Plone site has a LOT of content and this takes a bit of work to accomplish. Our Plone site also has a lot of external links that need web resources (and a generic policy for all of these resources) defined in the AAC or they will not work, so our web resource list is somewhat extensive. This is where things break down - the more web resources we add the more the AAC proxy has to work when re-writing our Plone site to be presented to an end user, and the processor utilization climbs.
I will continue to post to this thread as we work on the issue.
|
|
SteveMcG
SmartAccessCentral Moderator
User Online:
|
|
Posted:19/04/2006 11:34 PM |
|
|
Hi Michael,
Thanks for the update, keep the information flowing and lets see if we can learn from your experiences!
I have also had a recent experience with a delay after login of up to 30 seconds before the Navui environment was built. In my situation I have a CAG with a single NIC in a DMZ. There is (coincidentally?) a Netscreen firewall between the DMZ and internal Network. I noticed that when I pinged the CAG continuously, approximately one ping in 15 was being dropped.
I am still investigating this, but it would be interesting to see if there are any packets being dropped in icebun's scenario which looks very similar to mine in architecture and issue.
Anyway mike, if you can post more information about the cause of your delays that would be helpful!
|
|
icebun
New Member
User Online:
|
|
Posted:24/04/2006 1:52 PM |
|
|
Hi Micheal /Steve,
I will run through my setup in a little more detail:-
We are using a Nestcreen 25 and I have and an IP address of 10.0.0.x which is configured on INT0 on the Citrix Applicance. I have used the Mapped IP Address (MIP) feature to map this via the "Untrust" interface on the Netscreen which redirects traffic for the real public IP address over to the Applicance.
Steve, I have tried some constant pings and havn't noticed a drop after he fifteen one. By the way, how have you configured your polices for this?
I have done the following:-
Untrust to DMZ - Port 443 to Citrix Appliance
DMZ-Trust (Citrix Applicance all services to LAN)
Trust to DMZ (All LAN traffice to Citrix Appliance)
I have tried to use a local account but it has not made much of a difference.
With a AD setup, its "out of the box" setup with nothing fancy.
By the way, we are using 2 factor authentication va Safewird remote access. Turning this off has made no difference to performance.
If you need any more info please let me know.
Regards.
|
|
MichaelWayneHarwood
Intermediate Member
User Online:
|
|
Posted:03/05/2006 2:33 PM |
|
Citrix has provided us with a private fix that extends the functionality of the AAC web proxy engine. They call it a "forward cache" and essentially it is a server side caching engine that increases performance and efficiency immensely.
If you have a significant number of web resources and policies you would be wise to open a support call with Citrix to see if this fix would benefit your installation.
|
|