ETAIS self-service portal is a single entry point for provisioning and managing computational and storage resources shared by ETAIS consortium members - UT, TTU, NICPB and HITSA - as well as public cloud providers. It is aimed at research groups affiliated with Estonian research and development institutions from both public and private sector.
Access it at https://minu.etais.ee.
Self-service portal offers research groups a way to collaborate on using and sharing research infrastructure to minimise bureaucracy of negotiating access, quotas and payments.
The main concepts are as follows:
- Organization is representing a customer of ETAIS. Organization is responsible for the actions of users connected to it in different roles. Organizations that provide Offerings are called Service Providers
- Project is an entity within an organization that aggregates and isolates teams and resources.
- Offering is a concrete service provided in Marketplace that can be ordered.
- Resource - Represents an instance of an Offerings. For example, it can be a Virtual Private Cloud or quota in a batch processing queue.
- User - Represents end-users of the system (humans or robots).
Marketplace provides a common way to provision resources.
The following resource categories are offered at the moment:
- Private clouds - a pool of resources dedicated to a particular organization.
- Virtual Machines aka VMs (requires pre-provisioned VPC) - a server with network connectivity for running customer payloads.
- Block Devices (requires pre-provisioned VPC) - persistent volumes for storage of the data.
- HPC - access to SLURM-based processing farms.
- Platform - access to Slurm-based processing farms.
Private clouds in more details
- Private Cloud (aka Virtual Private Cloud / VPC) is a compute service that allows procuring and managing a pool of virtualized infrastructure resources - like RAM, CPU, storage volumes and network resources - required to run virtual machines.
- VPC limits define total resources available in a cloud for creating virtual machine and volumes.
- VPC limits can be changed at any moment.
- Private clouds are accounted on a daily basis with the price of the most expensive set of limits during that day.
ETAIS self-service portal supports user accounts coming from TaaT federated identity system, which allows to use home organisation accounts for login. Most of the larger education and research institutions in Estonia are connected to TaaT already.
Alternatively, one can login using an account from eduGAIN.
Self-Service Portal is available from: https://minu.etais.ee
NB! Users need to accept Terms of Service presented on the first login for account activation!
ETAIS self-service is built around the concept of workspaces. Workspace defines structural context for the user. Each workspace type shows information and possible actions relevant to the user in a certain role. There are several workspace types available in the system:
Users are connected to the organizations and their projects through roles. Users may have several roles, specific to each workspace they have access to. Currently the following roles are available in the system:
- Organization owners (owners)
- Project managers (managers)
- System administrators (admins)
User roles are hierarchical in a way that organization owners can do everything that project managers and system administrators can do.
- Can access organization workspace.
- Can invite other users to participate in the organization.
- Can create and manage projects, including policies and cost limitations.
- Can do everything that project managers and system administrators can do.
- Can access project workspace when appointed by organization owner.
- Can manage project team from the users already connected to the organization.
- Can do everything that system administrators can do.
- Can access project workspace if appointed by organization owner or project manager.
- Can provision and manage cloud resources.
User workspace is a personal profile management space. It allows to configure user notifications, SSH public keys, update personal profile data etc.
Menu entries available within user workspace:
- Dashboard: listing all organizations and projects where user is participating
- Audit logs: listing events related to user
- SSH keys: managing public SSH keys for the user
- Notifications: managing notifications for the user
- Manage: editing and updating user profile details
Accessing user workspace
Access is done by clicking on user avatar and selecting one of the entries from a pop-up menu.
Adding a public SSH key to a profile
- Generate your SSH keypair (on Windows: using PuTTYgen, Linux: using OpenSSH).
- Click “Details” on the left side of the screen.
- Select “SSH keys”.
- Click “Add SSH key”.
- Paste there contents of a public SSH key.
Organization and project workspace selector
Navigation between different organization and project workspaces are done with the help of the workspace selector available in the header row.
Selecting organization workspace
Open workspace selector and click on “Select” button of target organization.
Selecting project workspace
Open workspace selector, mark target organization and click on “Select” button of target project.
Organization workspace allows to manage projects, subscriptions to resource providers and organization members. It is also intended to provide summary, accounting and auditing information regarding organization, projects and providers. To be able to access organization workspace, you need to have a organization owner role.
Menu entries available within organization workspace:
- Dashboard: overview of managed resources and projects
- Projects: projects management
- Marketplace: catalog of Offerings available for provisioning
- Analytics: resource usage reports
- Audit logs: event logs related to organization, its projects and resources
- Team: management of organization members and their project/role affiliations
- Accounting: resource usage accounting information
- Manage: management of organization details
Adding a project
Projects can be added by selecting “Projects” from the menu and clicking on “Add project” button.
“Create project” form requires you to enter project name and optionally project description. If you need to attach security class label for the project you should select one from the list presented. Submit form by clicking on “Add project” button.
Inviting a user
User workspace access and role management can be done on two separate levels:
- Organization workspace allows owners to invite users as organization members and to manage their project role assignments.
- Project workspace allows project managers to assign available organization members a role in their project.
Organization level invites can be created by selecting “Team” from the organization workspace menu and clicking on “Invitations” management tab.
For creating a new invitation please click on “Invite user” button.
NB! By sending an invite to a user you also accredit this user to become an organization member! In order to complete the joining process invited user needs to login with the URL provided in the invitation email.
“Invite user” form requires target user email address, initial project and role selection. Submit form by clicking on “Invite user” button.
Project workspace provides tools and information required for day-to-day work and oversight over the managed IT infrastructure. Access is done via workspace selector in the top section of user interface.
Menu entries available within project dashboard:
- Dashboard: overview of project resources and latest events
- Marketplace: catalog of Offerings available for provisioning
- Resources: provisioned resource listings and management views by resource category (VMs, Private Clouds, Storage, etc.)
- Audit logs: event logs related to project and its resources
- Team: project team management
Adding a VPC
Virtual Private Cloud resource package can be added by selecting “Resources” and “Private Clouds” from the menu and clicking on “Add private cloud” button.
NB! There are several Virtual Private Cloud providers available from the Marketplace. You need to provision at least one VPC package from suitable provider in order to be able to create virtual machines.
It is mandatory to input VPC Tenant name and choose initial resource package, by clicking on “VPC package: Show choices” selector.
Currently there are four VPC resource packages categories listed: trial, small, medium and large. Each category can have several resources packages mapped. For selecting a VPC resource package please mark suitable package entry and click on “Select” button, returning to the previous form.
We also strongly suggest to fill also VPC description field. Other input fields are autofilled and can be optionally customized, if required. “Checkout summary” on the right pane will provide detailed overview of VPC resouce package purchase.
NB! Provisioned VPC resource package will be automatically enabled for the project as a VM provider! For other projects it can be enabled by the organization owner under Provider management within organization workspace.
Adding a VM
Projects need to have at least one VPC resource package enabled, before any virtual machines can be created!
VMs can be added by selecting “Resources: Virtual machines” from the menu and clicking on “Add virtual machine” button.
In case you have multiple VPC providers enabled within the project you will need to select also VM provider. “Create Openstack instance” form requires VM name and selection of VM image.
Please select operating system for a VM and click on “Select” button, returning to the form.
It is mandatory to select initial VM resource profile ie flavor, by clicking on “Flavor: Show choices” selector.
Flavor will set initial resource profile for a VM - how much RAM, CPU cores and storage it will have.
NB! VM images contain their minimum requirements information and non-matching VM flavors are disabled automatically!
Selecting VM flavor will also update “System volume size” with the option to override it manually (to higher custom value). Data volume is always provisioned with a VM and its size can be customized and incremented in 1GB steps.
By default provisioned virtual machines expect users to login using SSH keys. Initial SSH key for a login should be selected by clicking on “SSH public key: Show choices” selector.
There has to be at least one SSH public key added to user profile for it to appear in SSH key selector list!
NB! Different VM images have different default user names for SSH logins! For example: CentOS images use “centos” user, Ubuntu images use “ubuntu” user, Windows images use “Administrator” user.
By default no incoming connections will be allowed for a VM. Predefined Security Groups that contain firewall rules must be linked to a VM in order to open up access (like ssh, http, etc).
NB! VM create form will automatically include “default” security group which enables egress (ie outgoing) traffic for a VM and which is required in order to reply to any of the incoming packets!
VM needs to be connected to at least one of the VPC (internal) networks and also to external network via floating IP - if extenal/public access to VM is required.
Floating IP is technically realized as 1:1 NAT between VM internal ip and public network ip.
We strongly suggest to add also VM description. In order to provision the VM please click on “Purchase” button.
On the right pane there will be “Checkout summary” with the purchase overview and indicative VM cost (as part of VPC package cost).
VM should reach into “Active” status when successfully provisioned. “Access” field will show IP address to access VM over SSH (Linux) or over RDP (Windows).
VM access over SSH or RDP should be permitted by Security Groups linked to VM!
OpenStack VPC VMs will have additional 64MB virtual hard disk attached to VM which functions as cloud-init configuation drive (not supported by self-service yet, only user-data support at the moment).
VPC Security Groups management
Security Groups are scoped and managed under VPC package. For managing Security Groups and their rules please go into VPC detailview by clicking on provisioned VPC package name.
Existing Security Groups present within current VPC are listed under “Security groups” tab.
For adding new Security Group please click on “Create” button.
It is required to enter Security Group name and to add at least one rule. We recommend adding a description as well.
Adding a batch queue allocation
In order to be able to use batch queues in ETAIS you need to setup your FreeIPA profile. You can do it in your profile.
Batch resource allocation in one of ETAIS HPC centers can be created by selecting “Batch processing” from Marketplace or by going to “Resources: Batch processing” and clicking on “Create” button.
After selecting an HPC provider, you will be able to fill in the form setting the planned monthly limits for the batch resources. Accounting for resources is usage based, summary on the right will show the maximum cost of all limits are reached. Provision allocation by clicking on “Purchase” button.
Once allocation has been created, you can see its access information:
- Login information for SLURM head node. You will be able to use your FreeIPA account and any of the uploaded public SSH keys.
- Account ID that needs to be passed to sbatch command when scheduling a job.
Adding a Kubernetes cluster
Kubernetes (K8s) clusters are created and managed using Rancher management server. K8s clusters are deployed into a selected OpenStack Project (aka VPC), so it must exist before cluster can be created.
To create a K8S cluster select “Platform” category in the Marketplace.
Fill in the form for cluster creation. The following settings need to be defined:
- SSH public key to inject as authorized key into K8S nodes;
- Define private cloud where to deploy K8s as well as subnet where the nodes should be connected to.
- Define a K8s node plan, specifying roles, flavors and sizes.
Once the form is filled and validated, click on Purchase to proceed to provisioning.
Please note that creating a cluster takes on average 10 minutes per node.
Once the provisioning has started, you can see the details in the K8s detail view. During the initial creation, nodes are created one-by-one and added to the cluster. Note the linked OpenStack VMs - by default nodes are provisioned with internal IPs only. If you want to add Load Balancer or connect K8s directly to external network, please use the corresponding VM management.
Note that if an OpenStack Instance is part of a K8S cluster, it will include a short-cut link in its detail view.
Once created, it is possible to access K8s cluster also via Rancher management. Link is available from the cluster detail view. Rancher provides a rich set of options for management of K8s clusters. Initial access credentials are generated and e-mailed to users with access to cluster (project and organization roles) after the cluster is created. Note that if you already have access credentials, they won’t be created each time, instead permissions for your account would be added.
Once the cluster is active, you can also download kubeconfig file to access and manage K8s cluster. Please use Actions -> Generate Kubeconfig file for that.
Using GPU accelerator in a VM
GPU servers can be added by selecting “Resources” and “VMs” from the menu and clicking on “Add resource” button.
Once new resource is added, select GPU serverid UT HPC
Please choose name and plan for the offering, select correct tenant and click “Add to cart”.
Please review the request and if everything is correct, click “Purchase”.
Now your request will be processed and you should receive notification about the acceptance soon.