![]() |
|||
![]() |
![]() ![]() |
![]() |
![]() ![]() |
![]() |
![]() ![]() |
![]() |
| ||
Putting Jobs on HoldIf you select a pending job and click Hold, the Set Hold dialog box appears. ![]() The Set Hold dialog box enables setting and resetting user, operator, and system holds. User holds can be set or reset by the job owner as well as by grid engine managers and operators. Operator holds can be set or reset by managers and operators. System holds can be set or reset by managers only. As long as any hold is assigned to a job, the job is not eligible for running. Alternate ways to set or reset holds are the qalter, qhold, and qrls commands. Putting Array Job Tasks on HoldThe Tasks field on the Set Hold dialog box applies to Array jobs. Use this button to put a hold on particular subtasks of an array job. Note the format of the text in the Tasks field. The task ID range specified in this field can be a single number, a simple range of the form n-m, or a range with a step size. Thus the task ID range specified by, for example, 2-10:2 results in the task ID indexes 2, 4, 6, 8, and 10. This range represents a total of five identical tasks, with the environment variable SGE_TASK_ID containing one of the five index numbers. For detailed information about job holds, see the qsub(1) man page. Changing Job PriorityWhen you click Priority on the Job Control dialog box, the following dialog box appears. ![]() This dialog box enables you to enter the new priority of selected pending or running jobs. The priority ranks a single user's jobs among themselves. Priority tells the scheduler how to choose among a single user's jobs when several jobs are in the system simultaneously. When you select a pending job and click Qalter, the Submit Job window appears. All the entries of the dialog box are set corresponding to the attributes of the job that were defined when the job was submitted. Entries that cannot be changed are grayed out. The other entries can be edited. The changes are registered with the grid engine system when you click Qalter on the Submit Job dialog box. The Qalter button is a substitute for the Submit button. Verifying Job ConsistencyThe Verify flag on the Submit Job dialog box has a special meaning when the flag is used in the Qalter mode. You can check pending jobs for their consistency, and you can investigate why jobs are not yet scheduled. Select the desired consistency-checking mode for the Verify flag, and then click Qalter. The system displays warnings on inconsistencies, depending on the checking mode you select. See Submitting Advanced Jobs With QMON and the -w option on the qalter(1) man page for more information. Using the Why? Button to Get Information About Pending JobsAnother method for checking why jobs are still pending is to select a job and click Why? on the Job Control dialog box. Doing so opens the Object Browser dialog box. This dialog box displays a list of reasons that prevented the scheduler from dispatching the job in its most recent pass. An example of a Browser window that displays such a message is shown in the following figure: ![]() Note - The Why? button delivers meaningful output only if the scheduler configuration parameter schedd_job_info is set to true. See the sched_conf(5) man page. The displayed scheduler information relates to the last scheduling interval. The information might not be accurate by the time you investigate why your job was not scheduled. Clearing Error StatesClick Clear Error to remove an error state from a pending job that failed due to a job-dependent problem. For example, the job might have insufficient permissions to write to the specified job output file. Error states are displayed using a red font in the pending jobs list. You should remove jobs only after you correct the error condition, for example, using qalter. Such error conditions are automatically reported through email if the job requests to send email when the job is aborted. For example, the job might have been aborted with the qsub -m a command. Customizing the Job Control DisplayTo customize the default Job Control display, click Customize. The Job Customize dialog box appears. Click the Select Job Fields tab. The Select Job Fields tab looks like the following figure: ![]() Use the Job Customize dialog box to configure the set of information to display. With the Job Customize dialog box, you can select more entries of the job object to be displayed. You can also filter the jobs that you are interested in. The example in the preceding figure selects the additional fields Projects, Tickets, and Submit Time. The following figure shows the enhanced look after customization is applied to the Finished Jobs list. ![]() Note - Use the Save button on the Customize Job dialog box to store the customizations in the file .qmon_preferences. This file is located in the user's home directory. By saving your customizations, you redefine the appearance of the Job Control dialog box. Filtering the Job ListThe following example of the filtering facility selects only those jobs owned by aa114085 that are suitable to be run on the architecture solaris64. ![]() The following figure shows the resulting Running Jobs tab of the Job Control dialog box. ![]() The Job Control dialog box that is shown in the previous figure is also an example of how QMON displays array jobs. Getting Additional Information About Jobs With the QMON Object BrowserYou can use the QMON Object Browser to quickly retrieve additional information about jobs without having to customize the Job Control dialog box, as explained in Monitoring and Controlling Jobs With QMON. You can open the Object Browser to display information about jobs in two ways:
The following Browser window shows an example of the job information that is displayed: ![]() Monitoring and Controlling Jobs From the Command LineThis section describes how to use the commands qstat, qdel, and qmod to monitor, delete, and modify jobs from the command line. Monitoring Jobs With qstatTo monitor jobs, type one of the following commands, guided by information that is detailed in the following sections:
qstat with no options provides an overview of submitted jobs only. qstat -f includes information about the currently configured queues in addition. qstat -ext contains details such as up-to-date job usage and tickets assigned to a job. In the first form, a header line indicates the meaning of the columns. The purpose of most of the columns should be self-explanatory. The state column, however, contains single character codes with the following meaning: r for running, s for suspended, q for queued, and w for waiting. See the qstat(1) man page for a detailed explanation of the qstat output format. The second form is divided into two sections. The first section displays the status of all available queues. The second section, titled PENDING JOBS, shows the status of the sge_qmaster job spool area. The first line of the queue section defines the meaning of the columns with respect to the queues that are listed. The queues are separated by horizontal lines. If jobs run in a queue, they are printed below the associated queue in the same format as in the qstat command in its first form. The pending jobs in the second output section are also printed as in qstat`s first form. The following columns of the queue description require more explanation.
The qstat(1) man page contains a more detailed description of the qstat output format. In the third form, the usage and ticket values assigned to a job are contained in the following columns:
In addition, the deadline initiation time is displayed in the column deadline, if applicable. The share column shows the current resource share that each job has with respect to the usage generated by all jobs in the cluster. See the qstat(1) man page for further details. Various additional options to the qstat command enhance the functionality. Use the -r option to display the resource requirements of submitted jobs. Furthermore, the output can be restricted to a certain user or to a specific queue. You can use the -l option to specify resource requirements, as described in Defining Resource Requirements, for the qsub command. If resource requirements are used, only those queues, and the jobs that are running in those queues, are displayed that match the resource requirement specified by qstat. Note - qstat command has been enhanced so that the administrator and the user may define files (see sge_qstat(5)) which can contain useful options. A cluster wide sge_qstat file may be placed under $xxQS_NAME_Sxx_ROOT/$xxQS_NAME_Sxx_CELL/common/sge_qstat The user private file is processed under the location $HOME/.sge_qstat. The home directory request file has the highest precedence, then the cluster global file. You can use the command line to override the flags contained in a file. Example 4-2 and Example 4-1 show examples of output from the qstat and qstat -f commands. Example 4-1 Example of qstat -f Output
| ||
| ||
![]() |