Skip to content

CONFIGURATION.md

Latest commit

 

History

History
289 lines (206 loc) · 13.2 KB

CONFIGURATION.md

File metadata and controls

289 lines (206 loc) · 13.2 KB
Raw

Configuration Guide

This document outlines the key configuration files and settings for deploying Open OnDemand (OOD) on the Eureka HPC cluster. Examples use Eureka-oriented hostnames and paths; adjust every hostname, OIDC client ID, and allowlist for your environment.

Eureka’s Open OnDemand web service is hosted on eureka.paice-ua.com (PAICE), not on *.alliancecan.ca. The Alliance Canada IdP (https://idp.alliancecan.ca) is used for OIDC; register redirect URIs and client ID for your PAICE hostname with whoever operates the IdP.

Some interactive apps (e.g. Jupyter, OpenRefine) export CC_CLUSTER=vulcan in before.sh.erb so the Alliance CVMFS / environment-modules stack resolves the correct builds. That is independent of the OOD URL or cluster name in SLURM.

Core Configuration Files

1. Main Portal Configuration (/etc/ood/config/ood_portal.yml)

Purpose: Apache virtual host configuration with OIDC authentication

Key Settings:

  • Server: eureka.paice-ua.com (OOD vhost; must match DNS and TLS certificate)
  • Authentication: OpenID Connect via Alliance Canada
  • SSL: Custom certificate paths
  • Session: 8-hour timeout with optional Redis caching
  • Security: OIDC email claim integration

Required Changes for Deployment:

  • Update servername to your domain
  • Configure OIDC provider metadata URL
  • Set OIDC client credentials
  • Point SSL certificates to your files

Note: OIDC authentication requires SSSD (System Security Services Daemon) to be properly configured on all systems for user authentication and home directory mapping.

2. Cluster Configuration (/etc/ood/config/clusters.d/eureka.yml)

Purpose: SLURM cluster integration and job submission settings

Key Settings:

  • Scheduler: SLURM with /usr/bin/ binaries
  • Login Host: SSH/login hostname for SLURM (e.g. eureka.paice-ua.com or a dedicated login name from your site)
  • Host Allowlist: rack* compute nodes and Eureka login nodes (patterns depend on your naming scheme)
  • Environment: No shell environment copying

Required Changes for Deployment:

  • Cluster YAML name: Use eureka.yml for this cluster, or rename to match another cluster (e.g., mycluster.yml)
  • Update cluster title and login hostname
  • Modify host allowlist patterns for your nodes
  • Verify SLURM binary and config paths

3. Dashboard Configuration (/etc/ood/config/ondemand.d/ondemand.yml)

Purpose: Web interface branding, layout, and application management

4. Dashboard View Customizations (/etc/ood/config/apps/dashboard/views/)

Purpose: Custom dashboard appearance and navigation elements

Key Files:

  • layouts/_footer.html.erb - Custom footer template
    • Contains branding and support information
    • Can be customized for institutional branding
  • layouts/nav/_logo.html.erb - Custom navigation logo
    • Replaces default OOD logo in navigation bar
    • Supports custom branding and institutional identity

Required Changes for Deployment:

  • Customize footer content for your institution
  • Replace logo with your institutional branding
  • Update support contact information

Key Settings:

  • Branding: University of Alberta green theme with AMII logo
  • Help Menu: Alliance documentation and support links
  • Pinned Apps: Shell, job management, development tools
  • Globus Integration: File transfer endpoints for /home, /project, /scratch

Required Changes for Deployment:

  • Update branding colors and logos
  • Configure help menu links for your institution
  • Select appropriate pinned applications
  • Set Globus endpoints for your file systems
  • Globus Endpoints: Use UUIDs, not smart names (e.g., 97bda3da-a723-4dc0-ba7e-728f35183b43)

4. Nginx Stage Configuration (/etc/ood/config/nginx_stage.yml)

Purpose: Per-User Nginx (PUN) environment variables

Key Settings:

  • MOTD: Message of the Day integration with markdown support
  • Locale: English-Canadian (en-CA) default
  • Format: Markdown-based or Text-based MOTD display for dashboard

Required Changes for Deployment:

  • Set appropriate locale for your region
  • Configure MOTD path if different
  • Choose MOTD format: txt, markdown, markdown_erb, txt_erb, or rss

5. Shell Application Security (/etc/ood/config/apps/shell/env)

Purpose: SSH host access control for shell application

Key Settings:

  • Host Allowlist: Restricted to rack nodes and Eureka login nodes (example pattern only)
  • Pattern: rack[0-9][0-9]-[0-9][0-9]-[0-9][0-9][0-9][0-9]:eureka1:eureka2:rack* (replace hostnames with your login nodes)

Required Changes for Deployment:

  • Update host patterns to match your compute node naming
  • Ensure SSH key-based authentication is configured

6. Localization (/etc/ood/config/locales/)

Purpose: Multi-language interface support

Files:

  • en-CA.yml: English (Canada) translations
  • fr-CA.yml: French (Canada) translations

Required Changes for Deployment:

  • Add your preferred languages
  • Customize translations for your institution

System Integration Files

7. Message of the Day (MOTD)

Purpose: Welcome messages for different interfaces

Dual MOTD Setup:

  • /etc/motd - System-wide MOTD displayed in terminal sessions
    • ASCII art welcome message for Eureka cluster
    • Support contact information and portal links
  • /etc/ood/config/ondemand.d/motd - OOD dashboard MOTD
    • Markdown-formatted welcome message
    • Rich formatting with links and sections
    • Displayed in the web dashboard interface

Current Configuration:

  • OOD dashboard uses /etc/ood/config/ondemand.d/motd with markdown format
  • System terminals continue to use /etc/motd
  • Both MOTDs can be maintained independently

Required Changes for Deployment:

  • Update welcome messages for your cluster
  • Modify support contact information
  • Customize dashboard MOTD with markdown formatting
  • Ensure MOTD_PATH in nginx_stage.yml points to your preferred file

8. XDG Runtime Setup (/etc/sudoers.d/create-ice-xdg)

Purpose: Allow users to create X11 forwarding directories

Content: ALL ALL=(ALL) NOPASSWD: /usr/local/bin/create-ice.sh

Required Changes for Deployment:

  • Ensure create-ice.sh script exists on compute nodes
  • Verify script creates proper XDG runtime directories

Compute Node Automation

9. Ansible Playbooks (/etc/ansible/playbooks/)

Purpose: Self-configuring automation for compute hosts

Desktop Environment Setup (40-install-desktop-env.yamls)

Purpose: Installs and configures desktop environments for remote desktop applications

Key Features:

  • Multiple Desktop Environments: GNOME, XFCE, MATE, LXQt
  • VirtualGL Integration: GPU acceleration for remote desktop sessions
  • X11 Utilities: Complete X11 forwarding and desktop application support
  • File Management: Dolphin, Nautilus, Thunar file managers
  • Terminal Applications: GNOME Terminal, Konsole, XFCE Terminal
  • System Tools: Control panels, system monitors, and utilities

Required Changes for Deployment:

  • Verify package availability in your APT repositories
  • Ensure VirtualGL is available for GPU acceleration
  • Test desktop environment compatibility with your applications

Chrome Installation (41-install-chrome.yaml)

Purpose: Installs Google Chrome for web-based applications

Key Features:

  • Web Browser Access: Enables web applications in remote desktop sessions
  • Internal Repository: Uses institutional APT repository for security
  • Desktop Integration: Seamless integration with desktop environments

Required Changes for Deployment:

  • Ensure Google Chrome package is available in your APT repositories
  • Verify internal repository configuration
  • Test Chrome compatibility with your desktop environments

Usage:

# Run desktop environment setup
ansible-playbook /etc/ansible/playbooks/40-install-desktop-env.yamls

# Install Chrome
ansible-playbook /etc/ansible/playbooks/41-install-chrome.yaml

Note: These playbooks are designed to run on compute nodes to prepare them for remote desktop applications. They should be executed after the base system is configured and before users start accessing desktop applications.

Automated Configuration

Auto-Generated Files

The following files are automatically generated and should not be edited manually:

  • /etc/ood/config/apps/dashboard/initializers/paice_cluster_info.rb: SLURM partition information
  • /etc/ood/config/apps/dashboard/initializers/paice_gpu_info.rb: GPU types, max counts (including fractional T.2 / T.4 when sharing exists), gpu_sharing_available, and slices_per_gpu
  • /etc/ood/config/apps/dashboard/initializers/paice_app_versions.rb: Available software versions

GPU slice policy (softmig_slices)

Interactive app forms read GPU metadata from paice_gpu_info.rb, generated by gen_gpu_rb.sh.

  • /etc/ood/config/softmig_slices: Lines starting with # are ignored; the first line containing an integer sets the tier (see softmig_slices.example in the repo).
    • 0: Whole-GPU Slurm types only (no fractional rows in the dropdown).
    • 2: Whole + half (T.2) when Slurm Gres= includes shard and slices_per_gpu ≥ 2.
    • 4: Whole + half + quarter (T.4) when shard is present and slices_per_gpu ≥ 4 (default in repo / Ansible).
  • Precedence: environment variable SOFTMIG_SLICES (debug), then the file above, then default 4 inside gen_gpu_rb.sh if the file is missing.
  • Shard detection: Fractional rows are emitted only if a shard:… segment appears in scontrol show node Gres=. Without shard, base gpu:type:count types still appear; without any gpu: segments, the GPU list is empty.
  • slices_per_gpu: Derived as shard total ÷ GPU count per Gres= line (minimum ratio across lines if heterogeneous), or cluster-wide max(shard) / max(gpu count) when shard and GPU are not on the same line. Used to cap fractional gpu_count in forms and in submit.yml.erb.
  • Ansible: The bootstrap playbook does not write softmig_slices; it comes from the repo under etc/ood/config/softmig_slices (deployed with the rest of /etc/ood/) or you set it on the server / use SOFTMIG_SLICES. Slurm already gates fractional rows (no shard in Gres= → no T.2/T.4). Forks can add an Ansible copy task or group_vars if they want to force a tier on every deploy.
  • After changing this file or Slurm GRES, run sudo /opt/ood/cron/gen_gpu_rb.sh and reload OOD / user PUNs as needed.

Generation Scripts

Located in /opt/ood/cron/:

  • gen_cluster_rb.sh: Updates cluster information from SLURM
  • gen_gpu_rb.sh: Discovers GPU configuration from scontrol, applies softmig_slices, emits base and optional fractional Slurm type ids (T, T.2, T.4)
  • gen_app_rb.sh: Queries environment modules for versions

Important: Run these cron scripts before starting OOD to populate the initial configuration files. The auto-generated files contain essential cluster information that OOD needs to function properly.

Configuration Updates

When making configuration changes, use these commands to apply updates:

# Update OOD portal configuration
sudo /opt/ood/ood-portal-generator/sbin/update_ood_portal -f

# Clean up user PUN directories (run on all OOD nodes)
sudo /opt/ood/nginx_stage/sbin/nginx_stage nginx_clean --force

Note: The nginx_clean command removes all user PUN directories and should be run on all OOD nodes when making configuration changes.

Cron Setup:

  • Copy or symlink scripts to /etc/cron.weekly/ or /etc/cron.d/
  • Run as root since scripts need access to SLURM commands and system paths
  • Recommended frequency: Weekly (cluster configuration changes infrequently)

Example cron entries (add to /etc/cron.d/ood-config):

# Update cluster partition information weekly
0 2 * * 0 root /opt/ood/cron/gen_cluster_rb.sh

# Update GPU configuration weekly  
0 3 * * 0 root /opt/ood/cron/gen_gpu_rb.sh

# Update application versions weekly
0 4 * * 0 root /opt/ood/cron/gen_app_rb.sh

Deployment Requirements

Initial Setup

  • Run cron scripts first to populate auto-generated files before starting OOD
  • Rename cluster configuration file to match your cluster (e.g., mycluster.yml)
  • Update all hostnames and domains throughout configuration files
  • Configure OIDC identity provider with proper client credentials and metadata URLs
  • Generate SSL certificates valid for your domain
  • Set SLURM paths and host patterns for your cluster
  • Customize branding and help menu for your institution
  • Configure file system paths and Globus endpoint UUIDs (not smart names)

Additional Configuration Options

Other configuration options are available for Redis session caching, custom applications, monitoring, and backup systems. Refer to the Open OnDemand documentation for advanced configuration options.

Security Considerations

  • SSL Certificates: Must be valid for your domain
  • OIDC Secrets: Store securely, not in plain text
  • Host Allowlists: Restrict access to authorized nodes only
  • File Permissions: Private keys should be 600, configs 644
  • Network Access: Limit to required ports (80, 443, SSH)