CF.Objective Notes -- Maintaining Healthy ColdFusion Servers

July 02, 2012

Maintaining Healthy ColdFusion Servers -- Rob Brooks Bilson

ColdFusion server can be perfect, but if the network or other things are bad, you'll still have problems.
Step through each phase of the ecosystem
Web server, app code, app server, os, hardware, external systems, network, etc.

Where do we start with all of this?
Get organized

Are you virtualized?
Virtualization to adopted everywhere
but it gets you the ability to set something up ONCE, save that VM, and now all of the sudden I can bring up another CF server.
Just spin up another copy of the VM I've already done.
In a virtual environment, disaster recovery is way easier
No longer concerned with physical hardware

Pre-installation checklist
latest drivers?
OS patched?
Current web server version and patches?
Separate drives/partitions
C: OS
D: CF Install
E: CFML and website assets

Don't want app server to compete with OS for disk IO, etc.

Installation considerations
set up service accounts (don't want to run CF under "root" account, etc)
Only install what you need now
-- solr, .NET integration, examples, etc.
If you're not going to use those things, don't install them
You can always rerun the installer later
If you don't need them, your'e just tying up additional disk space, potentially leaving server open to new vulnerabilities.
Multi-tier architecture (will CF run on the same box as the web server? what if i want to scale out?)

CF10 Secure Profile
in installer
if you choose secure profile, will install with Adobe's recommended security settings.
May be unhappy with those default settings, read the docs, can make the assessment if that's a good starting point for you or not.

Post-install checklist
Update JVM
Secure the CF Administrator
For CF9 make sure the JRun build in web server is not running (if applicable)
-- check JRun4/servers/instance/server-inf/jrun.xml

CF Admin settings
settings, request timeout after N seconds, etc
request tuning
caching
memory variable timeouts
turn off debugging
logging
-log slow pages
-log scheduled tasks

System Probes

Logging
Do as much logging as possible
in the CF Admin
- customize your log location
set a max log file
specify the number of log archives to keep

in your code
- consider segmenting your logs by application or module
- aggressively log within your application

3rd party tools for logging and analysis
-- Splunk, both open source and commercial versions

Security -- where do you start
CF9, Pete Freitag -- CF Lockdown Guide

"Security" tab in CF Admin
If you don't use RDS, turn it OFF
if you DO use it, make sure it has a password

Don't forget about the web server

Code Review early and often
-- write secure code
-- new security features in CF10
-- new toolset for cryptography in CF10

Crucible -- 5 user license for 10 bucks
Can go in and comment code for code reviews

CF10 Audit logging
logs most admin changes
5 servers, different settings in each, now we can tell who changed what, when, etc
(as long as you use individual usernames for each person that has access to CF Admin)

ESAPI -- included in CF9 and 10 (but not implemented natively until CF10, but the files are there in 9)

Server Management
AIR app
gets a consolidated view of all your servers
Can "diff" server configurations
start/stop instances
etc

Hotfix Installer in CF10
(and there's talk of rolling this back into CF9, tho no more details on that right now)

Monitoring
Proactive vs reactive
Logigng
-- custom log files
Probes
Server monitor
3rd party apps
- fusion reactor
- seefusion
- cftracker
- new relic
- pingdom

Sending an email when the system errors
- bad ideal, you get USED to the way your system breaks, and start ignoring the emails

Don't' want to address on a "problem by problem" basis, want to be more proactive than that

Performance ramifications
- yes, everything has a performance side effect
- really comes down to understanding the types of things you're doing
- i.e. writing log files is a small performance hit

"A fool with a tool is still a fool."
really easy to run this stuff and not understand it
take the time to learn what the metrics mean

CF Server Monitor

JMeter

Managing Robots and Spiders
robots.txt
-- not all bots/spiders honor this
no-index, no follow directives
-- again, not always respected
be mindful of the effect on CF sessions

Tuning and Optimization
the importance of load testing
-- how will your app react with 10 users? 100? 1000?
-- need to understand the capacity of your app and server
-- are there things i can do to tune my app, squeeze more performance out?
-- am i database constrained? network?
-- what happens if 50 users log in at the same time? or create PDFs at the same time? that's why it's important to load-test all of your app, not just "yeah i logged in and clicked a couple pages"

check out JMeter, easy to setup and walk through
GZip compression
caching

When garbage collection happens...everything STOPS
As more garbage gets in the system, the pauses for GC tale longer and longer, more frequently
can end up with a "spiral" that brings down the server
recommended: instead of "1 big CF instance", run multiple smaller ones
doesn't ALWAYS work, but might work for your server depending on what you are running

Resources:
CF Lockdown Guide
Apache JMeter
etc

@cftracker
@cfwhisperer
@fusion_reactor
@carehart

full slide deck for this press is on Rob's blog:
blog.brooks-bilson.com