Building Drupal Web Farms with IIS - Part 1 Alessandro Pilotti @alexpilotti MVP ASP.NET / IIS MCSD, MCAD, MCSE, MCDBA, MCT Red Hat Certified Engineer
Application Request Routing 2.5  IIS 7.x add-on installable via WebPI or setup available on http://iis.NET  Features  HTTP based routing decisions  Load balance algorithms  Health monitoring  Client affinity  Host name Affinity  Multiple server groups  Managing and monitoring by using the UI  Caching  SSL offloading  Failed Request Tracing rules
Web Farm Framework 2.1  Features  Integration with ARR  Platform provisioning  Including frameworks (PHP, etc)  Application provisioning via WebDeploy  Simplified farm management  Rich UI for monitoring and management
Architecture
Create a Server Farm After installing ARR and/or WFF the node “Server Farms” shows up in the tree
Server Farm  You can enable independently  Load balancing  Server farm provisioning  Enabling server farm provisioning only means that an alternative load balancing solution is planned, e.g.:  Hardware NLB  Varnish  Apache + HAProxy  WFF requires administrator credentials valid on all the hosts belonging to the farm  Creating an ARR farm via GUI creates also URL rewrite rules
Server Farm Details
WFF Server Roles  WFF is based on the concept of a primary host and multiple secondary hosts  Primary host  Is the master  Platform and applications get provisioned to secondary hosts  Secondary hosts  Are “clones” of the primary host  Roles can be switched  A secondary host can become primary  Provisioning is customizable
Adding Servers
CLI  To achieve the same result as seen before:  appcmd.exe set config -section:webFarms /+"[name='myServerFarm']" /commit:apphost  appcmd.exe set config -section:webFarms /+"[name='myServerFarm'].[address=’DrupalWeb1']" /commit:apphost  URL Rewrite rules:  appcmd.exe set config -section:system.webServer/rewrite/globalRules /+"[name='ARR_myServerFarm_loadbalance', patternSyntax='Wildcard',stopProces sing='True']" /commit:apphost  appcmd.exe set config -section:system.webServer/rewrite/globalRules /[name='ARR_myServerFarm_loadbalance',patternSyntax='Wildcard',stopProcessin g='True'].match.url:"*" /commit:apphost  appcmd.exe set config -section:system.webServer/rewrite/globalRules /[name='ARR_myServerFarm_loadbalance',patternSyntax='Wildcard',stopProcessin g='True'].action.type:"Rewrite" /[name='ARR_myServerFarm_loadbalance',patternSyntax='Wildcard',stopProcessin g='True'].action.url:"http://myServerFarm/{R:0}" /commit:apphost
ARR Configuration
Health Test
Load Balance
Routing Rules
Server Affinity
Caching Primary and secondary disk caches are configured at the server level
Add Drives
Cache Configuration
Cache Warm-up
Cache Rules
Enable Caching in the Farm
Platform Provisioning
Application Provisioning
Failed Request Tracing  Detailed requests log  Excellent troubleshooting tool  Launch IIS Manager  Select web site  e.g. Default Web Site  Select Failed Request Tracing…
Enable Failed Request Tracing
Tracing Rules  Select Failed Request Tracing Rules in the features pane  Add…
Trace Conditions
Select Providers
Analyze Traces  Traces are located by default in  %SystemDrive%inetpubLogsFailedReqLogFiles  Sections to check:  GENERAL_REQUEST_HEADERS  ARR_REQUEST_ROUTED  ARR_SERVER_STATS  GENERAL_RESPONSE_HEADERS  GENERAL_RESPONSE_ENTITY_BUFFER
CLI  To obtain what we did via GUI:  appcmd set site "Default Web Site" - traceFailedRequestsLogging.enabled:"true" /commit:apphost  appcmd.exe set config "Default Web Site" - section:system.webServer/tracing/traceFailedRequests /+"[path='*']”  appcmd.exe set config "Default Web Site" - section:system.webServer/tracing/traceFailedRequests /+"[path='*'].traceAreas.[provider='WWW Server',areas='Rewrite,RequestRouting',verbosity='Verbose']”  appcmd.exe set config "Default Web Site" - section:system.webServer/tracing/traceFailedRequests /[path='*'].failureDefinitions.statusCodes:"200-399"
Shared Config  Share the same IIS server configuration among two or more hosts  Obtained by saving the configuration files on a network share  An encryption key is set to protect sensitive data  Web sites reside on shared paths
Export Config
Enable Shared Config
Shared Config
Scenario
Roles  Load balancer / Farm controller (DrupalARR)  Web Server 1 (DrupalWeb1)  Web Server 2 (DrupalWeb2)  File Server (DrupalFileServer)  MySQL Server (DrupalDB)  Note: In our demo and lab, we will install MySQL on DrupalFileServer to limit the number of required VMs
DBMS  Install MySQL on the DBMS server  Can be Windows, Linux, etc  CREATE DATABASE MyNewDrupalSite2;  Enable access to the MySQL database from the web servers. E.g.:  CREATE USER 'MyNewDrupalSite2'@'%' IDENTIFIED BY 'Passw0rd';  GRANT ALL On MyNewDrupalSite2.* TO 'MyNewDrupalSite2'@'%’;  Note: in the Drupal web apps, connecting to MySQL using an IP address is preferred  Avoid name resolution
Web Farm Hosts  Web farm hosts (DrupalWeb1 and DrupalWeb2)  Install Web Deploy via WebPI, if not already installed  Disable UAC (User Account Control)  Enable the following feature through the local firewall  Core Networking  File And Printer Sharing  Remote Administration
Web Farm Hosts  We need a common account with administrator rights on the hosts. It can be:  A domain user  A user with same username/password on all hosts  Must be a member of the “Administrators” group  Check that the user password is not set to expire  net user wfadmin Passw0rd /add  wmic path Win32_UserAccount where Name='wfadmin' set PasswordExpires=false  net localgroup administrators wfadmin /add
Drupal Web Site  Create a Drupal web site on the primary server (DrupalWeb1)  Via WebPI or manually  As seen in the previous lessons  Install all necessary components  PHP 5.3, PHP Manager, WinCache, URL Rewrite  Point the Drupal DB configuration to the database on the DBMS server  For performance reasons, consider using an IP address instead of name resolution  WFF will later use WebDeploy to replicate the site to the secondary servers (DrupalWeb2 in our case)
Load Balancer ARR  Install ARR 2.5 and WFF 2.1 via WebPI
ARR  Open IIS Manager and create Web Farm.  Note: if IIS Manager was open during the setup of ARR, it needs to be closed and reopened to be able to access the installed features. 
Server Farm Details  Set the wfadmin user you created in the previous steps
Add Primary Host to the Farm  In our case, DrupalWeb1 will become the primary server  This host contains the Drupal web site(s) to be served by the farm (can be added later)
Add Secondary Hosts to the Farm
URL Rewrite Rules  Create local URL Rewrite rules. This is the base of the redirection process!
Platform Provisioning  At this point, platform and application synchronization between the hosts takes place.  Platform tools and frameworks that are available on the primary server get installed on the other servers.  Afterwards, application pools, sites and applications will be synchronized  Provisoning can be started manually  Check secondary web farm hosts afterwards
Platform Provisioning
ARR – Configure Health Test
ARR – Check Health Test
ARR – Set Affinity Cookie Note: a distributed session management should also be evaluated.
The “drupalwp” User  Create a common user on:  DrupalFileServer  DrupalWeb1  DrupalWeb2  No privileges required  net user drupalwp Passw0rd /add  wmic path Win32_UserAccount where Name=’drupalwp' set PasswordExpires=false  Check that the user password is not set to expire  Repeat on all the above hosts
WP Identity  On the primary server, in IIS Manager, open the server node, click on Application Pools and on the Drupal web site application pool.  Click on “Advanced Settings…”  Change the Identity to the drupalwp user and set it to “drupalwp”  In the PHP.ini configuration file on the PHP version used by the Drupal web site, set  fastcgi.impersonate = 0  For performance, make sure also that WinCache is enabled.
Shared Storage Access  Drupal requires a shared “files” directory accessed by the web farm hosts  Can be hosted on a SMB share  As an alternative FS sync solutions could be applied  DFS, rsync, WebDeploy, etc  A valid user must exist on the web farm hosts and on the file server. It can be:  A domain account  A local user with same username and password  Proper permissions need to be set on the SMB share
Folder Permissions  Create a directory where you want to store your Drupal shared data  E.g: C:DataMyNewDrupalSite  Add write permissions for the user “drupalwp”
SMB Share  Create an SMB share for the directory and add read/write permissions for the user “drupalwp”
CLI  md C:DataMyNewDrupalSite  icacls C:DataMyNewDrupalSite /grant drupalwp:(OI)(CI)(RX,W)  net share MyNewDrupalSite=C:DataMyNewDrupalSite /grant:wpuser,Change
Troubleshoot SMB Access  From the web farm hosts (DrupalWeb1, DrupalWeb2) try to access the shared folder with the identity of the WP user:  Runas /u:drupalwp cmd  In the drupalwp command prompt:  net use drupaltest05MyNewDrupalSite  echo test > drupaltest05MyNewDrupalSitetest.txt  dir drupaltest05MyNewDrupalSite  File exists: ok  del drupaltest05MyNewDrupalSitetest.txt
Drupal Configuration  Log on the drupal web site as administrator and go to:  Administer -> Site Configuration -> File System  Set the “File System”, “Temp” path on the shared storage area.  Create a separate share on the file server for the “Private” path if necessary
Drupal Configuration
Drupal 6 Caveat  Drupal 6.19 published on WebPI doesn’t support UNC paths. This can be easily solved with the following code patch: File includes/file.inc Line 126 Original if (!is_writable($directory)) { Modified if (!touch($directory)) {

Building drupal web farms with IIS - part 1

  • 1.
    Building Drupal Web Farmswith IIS - Part 1 Alessandro Pilotti @alexpilotti MVP ASP.NET / IIS MCSD, MCAD, MCSE, MCDBA, MCT Red Hat Certified Engineer
  • 2.
    Application Request Routing 2.5  IIS 7.x add-on installable via WebPI or setup available on http://iis.NET  Features  HTTP based routing decisions  Load balance algorithms  Health monitoring  Client affinity  Host name Affinity  Multiple server groups  Managing and monitoring by using the UI  Caching  SSL offloading  Failed Request Tracing rules
  • 3.
    Web Farm Framework2.1  Features  Integration with ARR  Platform provisioning  Including frameworks (PHP, etc)  Application provisioning via WebDeploy  Simplified farm management  Rich UI for monitoring and management
  • 4.
  • 5.
    Create a ServerFarm After installing ARR and/or WFF the node “Server Farms” shows up in the tree
  • 6.
    Server Farm  Youcan enable independently  Load balancing  Server farm provisioning  Enabling server farm provisioning only means that an alternative load balancing solution is planned, e.g.:  Hardware NLB  Varnish  Apache + HAProxy  WFF requires administrator credentials valid on all the hosts belonging to the farm  Creating an ARR farm via GUI creates also URL rewrite rules
  • 7.
  • 8.
    WFF Server Roles WFF is based on the concept of a primary host and multiple secondary hosts  Primary host  Is the master  Platform and applications get provisioned to secondary hosts  Secondary hosts  Are “clones” of the primary host  Roles can be switched  A secondary host can become primary  Provisioning is customizable
  • 9.
  • 10.
    CLI  To achievethe same result as seen before:  appcmd.exe set config -section:webFarms /+"[name='myServerFarm']" /commit:apphost  appcmd.exe set config -section:webFarms /+"[name='myServerFarm'].[address=’DrupalWeb1']" /commit:apphost  URL Rewrite rules:  appcmd.exe set config -section:system.webServer/rewrite/globalRules /+"[name='ARR_myServerFarm_loadbalance', patternSyntax='Wildcard',stopProces sing='True']" /commit:apphost  appcmd.exe set config -section:system.webServer/rewrite/globalRules /[name='ARR_myServerFarm_loadbalance',patternSyntax='Wildcard',stopProcessin g='True'].match.url:"*" /commit:apphost  appcmd.exe set config -section:system.webServer/rewrite/globalRules /[name='ARR_myServerFarm_loadbalance',patternSyntax='Wildcard',stopProcessin g='True'].action.type:"Rewrite" /[name='ARR_myServerFarm_loadbalance',patternSyntax='Wildcard',stopProcessin g='True'].action.url:"http://myServerFarm/{R:0}" /commit:apphost
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
    Caching Primary and secondarydisk caches are configured at the server level
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
  • 22.
  • 23.
  • 24.
    Failed Request Tracing Detailed requests log  Excellent troubleshooting tool  Launch IIS Manager  Select web site  e.g. Default Web Site  Select Failed Request Tracing…
  • 25.
  • 26.
    Tracing Rules  SelectFailed Request Tracing Rules in the features pane  Add…
  • 27.
  • 28.
  • 29.
    Analyze Traces  Tracesare located by default in  %SystemDrive%inetpubLogsFailedReqLogFiles  Sections to check:  GENERAL_REQUEST_HEADERS  ARR_REQUEST_ROUTED  ARR_SERVER_STATS  GENERAL_RESPONSE_HEADERS  GENERAL_RESPONSE_ENTITY_BUFFER
  • 30.
    CLI  To obtainwhat we did via GUI:  appcmd set site "Default Web Site" - traceFailedRequestsLogging.enabled:"true" /commit:apphost  appcmd.exe set config "Default Web Site" - section:system.webServer/tracing/traceFailedRequests /+"[path='*']”  appcmd.exe set config "Default Web Site" - section:system.webServer/tracing/traceFailedRequests /+"[path='*'].traceAreas.[provider='WWW Server',areas='Rewrite,RequestRouting',verbosity='Verbose']”  appcmd.exe set config "Default Web Site" - section:system.webServer/tracing/traceFailedRequests /[path='*'].failureDefinitions.statusCodes:"200-399"
  • 31.
    Shared Config  Sharethe same IIS server configuration among two or more hosts  Obtained by saving the configuration files on a network share  An encryption key is set to protect sensitive data  Web sites reside on shared paths
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
    Roles  Load balancer/ Farm controller (DrupalARR)  Web Server 1 (DrupalWeb1)  Web Server 2 (DrupalWeb2)  File Server (DrupalFileServer)  MySQL Server (DrupalDB)  Note: In our demo and lab, we will install MySQL on DrupalFileServer to limit the number of required VMs
  • 37.
    DBMS  Install MySQLon the DBMS server  Can be Windows, Linux, etc  CREATE DATABASE MyNewDrupalSite2;  Enable access to the MySQL database from the web servers. E.g.:  CREATE USER 'MyNewDrupalSite2'@'%' IDENTIFIED BY 'Passw0rd';  GRANT ALL On MyNewDrupalSite2.* TO 'MyNewDrupalSite2'@'%’;  Note: in the Drupal web apps, connecting to MySQL using an IP address is preferred  Avoid name resolution
  • 38.
    Web Farm Hosts Web farm hosts (DrupalWeb1 and DrupalWeb2)  Install Web Deploy via WebPI, if not already installed  Disable UAC (User Account Control)  Enable the following feature through the local firewall  Core Networking  File And Printer Sharing  Remote Administration
  • 39.
    Web Farm Hosts We need a common account with administrator rights on the hosts. It can be:  A domain user  A user with same username/password on all hosts  Must be a member of the “Administrators” group  Check that the user password is not set to expire  net user wfadmin Passw0rd /add  wmic path Win32_UserAccount where Name='wfadmin' set PasswordExpires=false  net localgroup administrators wfadmin /add
  • 40.
    Drupal Web Site Create a Drupal web site on the primary server (DrupalWeb1)  Via WebPI or manually  As seen in the previous lessons  Install all necessary components  PHP 5.3, PHP Manager, WinCache, URL Rewrite  Point the Drupal DB configuration to the database on the DBMS server  For performance reasons, consider using an IP address instead of name resolution  WFF will later use WebDeploy to replicate the site to the secondary servers (DrupalWeb2 in our case)
  • 41.
    Load Balancer ARR Install ARR 2.5 and WFF 2.1 via WebPI
  • 42.
    ARR  Open IISManager and create Web Farm.  Note: if IIS Manager was open during the setup of ARR, it needs to be closed and reopened to be able to access the installed features. 
  • 43.
    Server Farm Details Set the wfadmin user you created in the previous steps
  • 44.
    Add Primary Hostto the Farm  In our case, DrupalWeb1 will become the primary server  This host contains the Drupal web site(s) to be served by the farm (can be added later)
  • 45.
  • 46.
    URL Rewrite Rules Create local URL Rewrite rules. This is the base of the redirection process!
  • 47.
    Platform Provisioning  Atthis point, platform and application synchronization between the hosts takes place.  Platform tools and frameworks that are available on the primary server get installed on the other servers.  Afterwards, application pools, sites and applications will be synchronized  Provisoning can be started manually  Check secondary web farm hosts afterwards
  • 48.
  • 49.
    ARR – ConfigureHealth Test
  • 50.
    ARR – CheckHealth Test
  • 51.
    ARR – SetAffinity Cookie Note: a distributed session management should also be evaluated.
  • 52.
    The “drupalwp” User Create a common user on:  DrupalFileServer  DrupalWeb1  DrupalWeb2  No privileges required  net user drupalwp Passw0rd /add  wmic path Win32_UserAccount where Name=’drupalwp' set PasswordExpires=false  Check that the user password is not set to expire  Repeat on all the above hosts
  • 53.
    WP Identity  Onthe primary server, in IIS Manager, open the server node, click on Application Pools and on the Drupal web site application pool.  Click on “Advanced Settings…”  Change the Identity to the drupalwp user and set it to “drupalwp”  In the PHP.ini configuration file on the PHP version used by the Drupal web site, set  fastcgi.impersonate = 0  For performance, make sure also that WinCache is enabled.
  • 54.
    Shared Storage Access Drupal requires a shared “files” directory accessed by the web farm hosts  Can be hosted on a SMB share  As an alternative FS sync solutions could be applied  DFS, rsync, WebDeploy, etc  A valid user must exist on the web farm hosts and on the file server. It can be:  A domain account  A local user with same username and password  Proper permissions need to be set on the SMB share
  • 55.
    Folder Permissions  Createa directory where you want to store your Drupal shared data  E.g: C:DataMyNewDrupalSite  Add write permissions for the user “drupalwp”
  • 56.
    SMB Share  Createan SMB share for the directory and add read/write permissions for the user “drupalwp”
  • 57.
    CLI  md C:DataMyNewDrupalSite icacls C:DataMyNewDrupalSite /grant drupalwp:(OI)(CI)(RX,W)  net share MyNewDrupalSite=C:DataMyNewDrupalSite /grant:wpuser,Change
  • 58.
    Troubleshoot SMB Access From the web farm hosts (DrupalWeb1, DrupalWeb2) try to access the shared folder with the identity of the WP user:  Runas /u:drupalwp cmd  In the drupalwp command prompt:  net use drupaltest05MyNewDrupalSite  echo test > drupaltest05MyNewDrupalSitetest.txt  dir drupaltest05MyNewDrupalSite  File exists: ok  del drupaltest05MyNewDrupalSitetest.txt
  • 59.
    Drupal Configuration  Logon the drupal web site as administrator and go to:  Administer -> Site Configuration -> File System  Set the “File System”, “Temp” path on the shared storage area.  Create a separate share on the file server for the “Private” path if necessary
  • 60.
  • 61.
    Drupal 6 Caveat Drupal 6.19 published on WebPI doesn’t support UNC paths. This can be easily solved with the following code patch: File includes/file.inc Line 126 Original if (!is_writable($directory)) { Modified if (!touch($directory)) {