![]() |
Sun ONE Application Server Administrator's Guide to Security |
Administering SSL/TLS Encryption
Before beginning any of the tasks described in this section, you should already have a certificate and be familiar with the basic concepts of public-key cryptography, including encryption and decryption; public and private keys; digital certificates; and encryption protocols.
This section addresses the following topics:
About Encryption
Enabling SSL Communication with LDAP For more information on encryption and related topics, see Introduction to SSL in the Sun ONE Directory Server document collection.
About Encryption
Encryption is the process of transforming information so it is unintelligible to anyone but the intended recipient; decryption is the process of transforming encrypted information so that it is intelligible again. The Sun ONE Application Server 7 supports the Secure Sockets Layer (SSL) and Transport Layer Security (TLS) encryption protocols.
A cipher is a cryptographic algorithm (a mathematical function), used for encryption or decryption. The SSL and TLS protocols contain numerous cipher suites. Some ciphers are stronger and more secure than others. Generally speaking, the more bits a cipher uses, the harder it is to decrypt the data.
In any two-way encryption process, both parties must use the same ciphers. Because a number of ciphers are available, you need to enable your Sun ONE Application Server for those that are most commonly used in your environment.
SSL and TLS Protocols
The Sun ONE Application Server supports the Secure Sockets Layer (SSL) 3.0 and the Transport Layer Security (TLS) 1.0 protocols for encrypted communication. SSL and TLS are application independent, and higher-level protocols can be layered transparently on them.
SSL and TLS protocols support a variety of ciphers used to authenticate the server and client to each other, transmit certificates, and establish session keys. Clients and servers may support different cipher suites, or sets of ciphers, depending on factors such as which protocol they support, company policies on encryption strength, and government restrictions on export of encrypted software. Among other functions, the SSL and TLS handshake protocols determine how the server and client negotiate which cipher suites they will use to communicate.
During a secure connection, the client and the server agree to use the strongest cipher they have both enabled for communication. You can choose ciphers from the SSL2, SSL3, and TLS protocols.
[[Does the Note above still apply?]]
Public and Private Keys
The encryption cipher process alone isn't enough to secure your server's confidential information. A key must be used with the encrypting cipher to produce the actual encrypted result, or to decrypt previously encrypted information. The encryption process uses two keys to achieve this result: a public key and a private key. Information encrypted with a public key can be decrypted only with the associated private key. The public key is published as part of a certificate; only the associated private key is safeguarded.
For description of the various cipher suites, and more information about keys and certificates, see Introduction to SSL in the Sun ONE Directory Server document collection.
Enabling SSL Communication with LDAP
You should immediately configure your Administration Server to communicate with the LDAP using SSL.
To enable SSL, perform the following steps:
Access App Server Instances and select the server instance in the left pane.
Access Security in the left pane.
Select Configure Directory Service.
Click Yes next to Use Secure Sockets Layer (SSL) for connections?
You are asked if you want to switch to the standard port.
Click OK to change your port to the standard port for LDAP over SSL.
Turning Security On
You must turn security on before you can configure any other security settings. The following sections describe the ways of turning security on:
Turning Security On When Creating a Listen Socket
Turning Security On When Creating a Listen Socket
To turn security on when creating a new listen socket, perform the following steps:
Access App Server Instances and select the server instance in the left pane.
Access HTTP Server and HTTP Server Listeners in the left pane.
The HTTP Listeners page is displayed.
Click New.
The listener settings page is displayed.
While you are entering the information for the new listener, check Security Enabled under the SSL/TLS Settings to turn security on.
Turning Security On When Editing a Listen Socket
To turn security on when editing an existing listen socket, perform the following steps:
Access App Server Instances and select the server instance in the left pane.
Access HTTP Server and HTTP Listeners in the left pane.
The HTTP Listeners settings page is displayed.
Note If you have an external module installed, the Manage Server Certificates page appears requiring the external module's password before you can continue.
Under the SSL/TLS Settings, check Security Enabled to turn security on.
Enabling SSL and TLS
To protect the security of your application server, you should enable SSL by enabling the SSL2, SSL3, and TLS encryption protocols and selecting the various cipher suites.
The default settings allow the most commonly-used ciphers. Unless you have a compelling reason for not using a specific cipher suite, you should allow them all. For more information regarding specific ciphers, see Introduction to SSL.
Do not select "No Encryption, only MD5 message authentication". If no other ciphers are available on the client side, the server will default to this setting and no encryption will occur.
Before enabling SSL and TLS, security must be turned on, and you must have at least one certificate installed.
To enable SSL and TLS, perform the following steps:
Access App Server Instances and select the server instance in the left pane.
Access HTTP Servers and HTTP Server Listeners in the left pane.
The HTTP Listeners setting page is displayed.
Note If you have an external module installed, the Manage Server Certificates page appears requiring the external module's password before you can continue.
Under the SSL/TLS Settings, check the appropriate boxes associated with SSL and TLS, including all the ciphers.
Note Unless you have a compelling reason for not using a specific cipher suite, you should allow them all.
TLS must also be enabled on the browser seeking access to your server:
Check both TLS and SSL3 for Netscape Navigator 6.0.
Use the TLS Rollback option for Microsoft Internet Explorer 5.0 and 5.5.
For TLS Rollback also check TLS, and make sure both SSL3 and SSL2 are disabled.
Click Save.
Restart the server for changes to take effect.
The init.conf file is automatically modified to show security on, and all virtual servers are automatically assigned the default security parameters.
After you have enabled SSL on a server, its URLs use https instead of http. URLs that point to documents on an SSL-enabled server have this format:
https://servername.[domain.[dom]]:[port#]
For example:
Configuring Security Globally
Installing an SSL-enabled server creates directive entries in the init.conf file for global security parameters. Security must be enabled for virtual server security settings to work. SSL properties for virtual servers can be found on a per-server basis in the SSLPARAMS element of the server.xml file.
The Security directive globally enables or disables SSL by making certificates available to the server instance. It must be on for virtual servers to use SSL. If enabled, the user is prompted for the administrator password to access certificates, and so on).
The following topics address global security configuration:
SSL Configuration File Directives
SSL Configuration File Directives
To configure security globally, you must set values for the following SSL configuration file directives in the init.conf file:
SSLCacheEntries
Specifies the number of SSL sessions that can be cached. There is no upper limit.
If the number is 0, the default value, which is 10000, is used.
SSLClientAuthDataLimit
Specifies the maximum amount of application data, in bytes, that is buffered during the client certificate handshake phase. The default value is 1048576 (1 MB).
SSLClientAuthTimeout
Specifies the number of seconds after which the client certificate handshake phase times out. The default value is 60.
SSLSessionTimeout
Controls SSL2 session caching.
Syntax
SSLSessionTimeout seconds
The seconds value is the number of seconds until a cached SSL2 session becomes invalid. If the SSLSessionTimeout directive is specified, the value of seconds is silently constrained to be between 5 and 100 seconds. The default value is 100.
SSL3SessionTimeout
Controls SSL3 session caching.
Syntax
SSL3SessionTimeout seconds
The seconds value is the number of seconds until a cached SSL3 session becomes invalid. The default value is 86400 (24 hours). If the SSL3SessionTimeout directive is specified, the value of seconds is silently constrained to be between 5 and 86400 seconds.
Note A single connection group on a listen socket must have the same SSLPARAMS; multiple groups can have different SSLPARAMS.
Setting Values for SSL Directives
To set values for your SSL configuration file directives, perform the following steps:
Access App Server Instances and select the server instance in the left pane.
Access HTTP Server and HTTP Listener in the left pane.
The HTTP Listener settings page is displayed.
In the SSL/TLS Settings section, check the Security Enabled checkbox.
Select HTTP Server in the left pane.
The HTTP Server page is displayed.
Select the Advanced tab.
Click OK.
Using External Encryption Modules
The Sun ONE Application Server supports the two methods for using external cryptographic modules (such as smart cards or token rings): Public Key Cryptography Standard (PKCS#1) and Federal Information Processing Standards (FIPS-140).
Note You will need to add the PKCS #11 module before enabling the FIPS-140 encryption standard.
This section addresses the following topics:
Installing the PKCS11Module
Installing the PKCS11Module
The Sun ONE Application Server supports Public Key Cryptography Standard (PKCS) #11, which defines the interface used for communication between SSL and PKCS11 modules. PKCS11 modules are used for standards-based connectivity to SSL hardware accelerators. Imported certificates and keys for external hardware accelerators are stored in the secmod.db file, which is generated when the PKCS11 module is installed.
Using modutil to Install a PKCS11 Module
You can install PKCS11 modules in the form of .jar files or object files using the modutil tool.
To install the PKCS11 module using modutil, perform the following steps:
Verify that all servers, including the Sun ONE Application Server, are turned off.
Go to the instance_dir/config directory containing the databases.
Add to your PATH: instance_dir/bin/https/admin/bin
Locate the modutil tool in instance_dir/bin/https/admin/bin.
Set the environment. For example:
On Unix: setenv
LD_LIBRARY_PATH instance_dir/bin/https/lib:${LD_LIBRARY_PATH}
On IBM-AIX: LIBPATH
On Windows, add it to the PATH:
LD_LIBRARY_PATH instance_dir\bin\https\bin
You can find the PATH for your machine listed under: instance_dir\https-admin\start.
Invoke the modutil tool:
modutil
The options are displayed.
Perform the actions required.
For example, to add the PCKS#11 module in Unix, use the following syntax:
modutil -add <PCKS#11_filename> -libfile <libfile_name_for_PCKS#11> -nocertdb -dbdir <db _directory>
Using pk12util to Import and Export
The pk12util tool allows you to export certificates and keys from your internal database and to import them into an internal or external PKCS11 module. You can always export certificates and keys to your internal database, but most external tokens will not allow you to export certificates and keys.
By default, the pk12util tool uses certificate and key databases named cert7.db and key3.db.
Exporting With pk12util
To export a certificate and key from an internal database, perform the following steps:
Go to the server_root/config directory containing the databases.
Add to your PATH: instance_dir/bin/https/admin/bin
Locate the pk12util tool in instance_dir/bin/https/admin/bin.
Set the environment. For example:
On Unix: setenv
LD_LIBRARY_PATH/instance_dir/bin/https/lib:${LD_LIBRARY_PATH}
On IBM-AIX: LIBPATH
On Windows, add it to the PATH:
LD_LIBRARY_PATH instance_dir\bin\https\bin
You can find the PATH for your machine listed under: instance_dir\https-admin\start.
Invoke the modutil tool:
pk12util
The options are displayed.
Perform the actions required.
For example, in Unix you would enter:
pk12util -o certpk12 -n Server-Cert [-d /instance_dir/config] [-P https-test-host]
Enter the database password.
Importing With pk12util
To import a certificate and key into an internal or external PKCS11 module, perform the following steps:
Go to the instance_dir/config directory containing the databases.
Add to your PATH: instance_dir/bin/https/admin/bin
Locate pk12util in instance_dir/bin/https/admin/bin.
Set the environment. For example:
On Unix: setenv
LD_LIBRARY_PATH/instance_dir/bin/https/lib:${LD_LIBRARY_PATH}
On IBM-AIX: LIBPATH
On Windows, add it to the PATH:
LD_LIBRARY_PATH instance_dir/bin/https/bin
You can find the PATH for your machine listed under: instance_dir/https-admin/start
Enter the command: pk12util.
The options are displayed.
Perform the actions required.
For example, in Unix you would enter:
pk12util -i pk12_sunspot [-d certdir][-h "nCipher"][-P https-jones.redplanet.com-jones-]
-P must follow the -h and be the last argument.
Enter the exact token name including capital letters and spaces between quote marks.
Enter the database password.
Starting the Server with an External Certificate
If you install a certificate for your server into an external PKCS11 module (for example, a hardware accelerator), the server will not be able to start using that certificate until you edit the server.xml file, or specify the certificate name as described below.
The server always tries to start with the certificate named Server-Cert. However, certificates in external PKCS11 modules include one of the module's token names in their identifier. For example, a server certificate installed on an external smartcard reader called smartcard0 would be named smartcard0:Server-Cert.
To start a server with a certificate installed in an external module, you'll need to specify the certificate name for the connection group it runs on.
To select the certificate name for the connection group, perform the following steps:
Access App Server Instances and select the server instance in the left pane. [[I can't find the UI path for this procedure. Query to Siraj 7/11]]
Click the Edit Listen Sockets link.
The Listen Socket Table page appears.
Use the drop-down Action list to select Edit, if not already displayed, for the connection group you are enabling security for.
Use the drop-down list to turn Security on for that connection group, if it is off.
The Attributes link appears.
Click the Attributes link.
The Security Settings of Listen Socket page appears.
Use the drop-down CertificateName list to select the external server certificate. You can tell the Sun ONE Application Server to start with that server certificate by manually editing the server.xml file. Change the servercertnickname attribute in the SSLPARAMS element to:
$TOKENNAME:Server-Cert
To find what value to use for $TOKENNAME, go to the server's Security tab and select the Manage Certificates link. When you log in to the external module where Server-Cert is stored, its certificates are displayed in the list in the $TOKENNAME:$NICKNAME form.
Enabling FIPS-140 Standard
PKCS11 APIs enable communication with software or hardware modules that perform cryptographic operations. Once PKCS11 is installed on your Sun ONE Application Server, you can configure the server to be Federal Information Processing Standards (FIPS)-140 compliant.
Note These libraries are included only in SSL version 3.0.
To enable FIPS-140, perform the following steps:
Install the plug-in following the FIPS-140 instructions. [[Can't find the UI path for this procedure. Query to Siraj 7/11]]
Access Admin Server and select the server instance in the left pane.
Select the Preferences tab, if not already selected.
Click the Edit Listen Sockets link.
The Listen Socket Table page appears.
Use the drop-down Action list to select Edit, if not already displayed, for the connection group you are enabling FIPS-140 on.
Use the drop-down list to turn Security on for that connection group, if it is off.
The Security Settings of Listen Socket page appears.
The Security Feature window appears.
Check Allow: SSL version 3, if it is not already checked.
Select the appropriate FIPS-140 cipher suite:
(FIPS) DESwith 56 bit encryption and SHA message authentication
(FIPS) Triple DESwith 168 bit encryption and SHA message authentication
Click OK to close the Security Features window.
Setting Strong Ciphers
The Strong Ciphers option presents a choice of 168, 128, or 56-bit secret key size for access or no restriction. You can specify a file to be served when the restriction is not met. If no file is specified, the Sun ONE Application Server returns a Forbidden status.
If you select a key size for access that is not consistent with the current cipher settings, the Sun ONE Application Server warns that you need to enable ciphers with larger secret key sizes.
The implementation of the key size restriction is based on an NSAPI PathCheck directive in the obj.conf file, rather than Service fn=key-toosmall. This directive is:
PathCheck fn="ssl-check" [secret-keysize=<nbits>] [bong-file=filename]
where:
nbits is the minimum number of bits required in the secret key.
filename is the name of a file (not a URI) to be served if the restriction is not met.
PathCheck returns REQ_NOACTION if SSL is not enabled, or if the secret-keysize parameter is not specified. If the secret key size for the current session is less than the specified secret-keysize, the function returns REQ_ABORTED with a status of PROTOCOL_FORBIDDEN if bong-file is not specified, or else REQ_PROCEED, and the path variable is set to the bong-file filename. Also, when a key size restriction is not met, the SSL session cache entry for the current session is invalidated, so that a full SSL handshake will occur the next time the same client connects to the server.
Note The Strong Ciphers option removes any Service fn=key-toosmall directives that it finds in an object when it adds a PathCheck fn=ssl-check.
To use the Strong Ciphers option, perform the following steps:
Access App Server Instances and select the server instance in the left pane.
Access HTTP Server in the left pane.
Select Virtual Servers and click the virtual server you want.
Select the HTTP/HTML tab and click the Strong Ciphers link.
The Enforce Strong Security Requirements page is displayed.
Select Stronger Ciphers.
Select the secret key size restriction:
Enter the file location of the message to reject access.
Preventing Clients from Caching SSL Files
You can prevent pre-encrypted files from being cached by a client by adding the following line inside the <HEAD> section of a file in HTML:
<meta http-equiv="pragma" content="no-cache">
Previous Contents Index DocHome Next
Copyright © 2001 Sun Microsystems, Inc. Some preexisting portions Copyright © 2001 Netscape Communications Corp. All rights reserved.
Last Updated July 12, 2002