Model as a Service (MaaS)

What is it

A solution providing AI models as a Service to a larger audience

  • IT serves common models centrally or remotely

    • Generative AI focus but applicable to any model type

    • Centralized pool of hardware

    • Platform Engineering for AI

    • AI model management (versioning, regression testing, etc)

  • Models made available through API Gateway

  • Developers consume provided models, build AI applications

    • For end users (private assistants, etc)

    • To improve products or services through AI functionality

  • Shared Resources keeps costs down

  • Avoids shadow IT and associated costs

  • Can be used to prevent data leakage

MaaS
Figure 1. High Level View

Why do Customers need it

Self-Service is good for small teams with plentiful resources but struggles when teams or team siezes grow

Throwing GPUs at the problem is wateful and ineffective

  • Few people know how to use them correctly and effectively

  • Leads to duplication and underutilization of hardware

  • Leads to high costs

Most people want an Model serving endpoint, not a GPU

MaaS
Figure 2. Why do customers need it

What does Red Hat currently have

MAAS BUHosted
Figure 3. In House solution

Developed by the BU CAI team using RH Technology stack

Example 1. Try it out
https://maas.apps.prod.rhoai.rh-aiservices-bu.com/

Overview

The MaaS solution stack consists of Red Hat OpenShift AI, API Gateway and SSO.

  • Keycloak – Identity provider (IdP) for OAuth2/OIDC authentication.

  • 3scale API Management – Secures and manages the API (model endpoint).

  • OpenShift AI (RHOAI) – Hosts your deployed AI model (via ModelMesh or custom service).

  • User/Client – Requests a token from Keycloak, then calls the model via 3scale using that token.

MAAS Arch.RedhatProd
Figure 4. Red Hat Products
MAAS 3scale architecture
Figure 5. Solution architecture

Logical Architecture of MaaS to move to capability focus instead of product focus

MaaS
Figure 6. Logical Architecture

How do we deliver MAAS to customers

Things to Consider

  • 3Scale is end of life (Confirmed)

  • Red Hat Connectivity Link isn’t GA or functionally equivilent

  • LLMd is on the roadmap

  • Engineering is working on a MAAS product (early days)

  • Customer often have their own API gateways

  • Customer often have their own oidc/auth/authz/security providers

Delivery Approach

  • We must assume customers will have their own API gateway and security providers

  • We will have to work WITH the customers teams to deploy a MAAS solution

    • API G/W team

    • Security Team

    • Logging and Metrics

  • This will be done collaboratively with the customer as part of an AI Incubator engagement.

The next section goes into more techical details around what’s required.