Accelerate your learning towards AWS Certification exams with automated quiz generation using Amazon SageMaker foundations models

Earning AWS certification can help you advance your career, whether you want to find a new role, show off your skills to take on a new project, or become the go-to expert on your team. And because AWS certification exams create them as experts in the relevant function or technical area, preparing for one of these exams helps you develop the necessary skills identified by qualified professionals in the field.

It is important to read the AWS services FAQ page relevant to your certification exam in order to gain a deeper understanding of the service. However, this could take quite some time. Reading FAQs for even one service can take half a day to read and understand. For example, the Amazon SageMaker FAQ contains about 33 (printed) pages of content on SageMaker alone.

Wouldn’t it be an easier and more fun learning experience if you could use a system to test yourself on AWS service FAQ pages? You can actually develop this system using state-of-the-art language models and a few lines of Python.

In this post, we present a complete guide to implementing a multiple-choice quiz solution for the FAQ pages of any AWS service, based on the AI21 Jurassic-2 Jumbo Instruct base model in Amazon SageMaker Jumpstart.

Great linguistic models

In recent years, language models have experienced a huge increase in size and popularity. In 2018, BERT-large made its debut with its 340 million parameters and innovative transformer architecture, setting the benchmark for performance on NLP tasks. In a few years, the state of the art in terms of model size has increased more than 500 times; OpenAI’s GPT-3 and Bloom 176 B, both with 175 billion parameters, and the AI21 Jurassic-2 Jumbo Instruct with 178 billion parameters, are just three examples of large language models (LLMs) that raise the bar for natural language processing (NLP) accuracy.

SageMaker foundation models

SageMaker offers a variety of models from popular model hubs, such as Hugging Face, PyTorch Hub, and TensorFlow Hub, and AI21’s proprietary ones, Cohere, and LightOn, that you can access within your machine learning development workflow (ML) in SageMaker. Recent advances in ML have given rise to a new class of models known as foundation models, which have billions of parameters and are trained on massive amounts of data. These foundational models can be adapted to a wide range of use cases, including text summarization, digital art generation, and language translation. Because these models can be expensive to train, customers want to use pre-trained foundation models and tune them as needed, rather than train these models themselves. SageMaker provides a curated list of models that you can choose from in the SageMaker console.

With JumpStart, you can find foundation models from different suppliers, allowing you to get started with foundation models quickly. You can review model features and terms of use and test these models using a test UI widget. When you’re ready to use a base model to scale, you can easily do so without leaving SageMaker using prebuilt notebooks from model providers. Your data, whether used to evaluate or use the model at scale, is never shared with third parties because the models are hosted and deployed on AWS.

AI21 Jurassic-2 Jumbo Instruct

Jurassic-2 Jumbo Instruct is an AI21 Labs LLM that can be applied to any language generation or understanding task. It’s optimized to follow natural language instructions and context, so you don’t need to provide an example. The endpoint comes preloaded with the model and is ready to serve queries using an easy-to-use API and Python SDK, so you can get started. Jurassic-2 Jumbo Instruct performs better on HELM, especially on tasks related to reading and writing.

Solution overview

In the following sections, we will go through the steps to test the Jumbo Jurassic-2 instruction model in SageMaker:

1. Choose the Jumbo Jurassic-2 instruction model in the SageMaker console.
2. Evaluate the model using the playground.
3. Use a notebook associated with the base model to implement it in your environment.

Access Jurassic-2 Jumbo Instruct through the SageMaker console

The first step is to log in to the SageMaker console. Under jump start in the navigation pane, choose Foundation models to request access to the model list.

After your account is listed, you can view a list of models on this page and search for the Jurassic-2 Jumbo Instruct model.

Evaluate the Jurassic-2 Jumbo Instruct model in the model yard

From the AI21 Jurassic-2 Jumbo Instruct list, choose See model. You will see a description of the model and the tasks you can perform. Please read the template’s EULA before continuing.

We first test the model to generate a test based on the SageMaker FAQ page. Navigate to Children’s park tab

At the Children’s park tab, you can provide sample instructions to the Jurassic-2 Jumbo Instruct model and see the output.

Note that you can use a maximum of 500 tokens. We set the maximum length to 500, which is the maximum number of tiles to generate. This model has a context window of 8,192 tiles (request duration plus completion should be at most 8,192 tiles).

To make the message easier to view, you can enlarge it warning Box.

Since we can use a maximum of 500 tiles, we take a small part of the Amazon SageMaker FAQ page, the Low code ML section, for our test message.

We use the following indication:

Below is SageMaker Low-code ML FAQ:

##
Q: Will my data (from inference or training) be used or shared to update the base model that is offered to customers using Amazon SageMaker JumpStart?
No. Your inference and training data will not be used nor shared to update or train the base model that SageMaker JumpStart surfaces to customers.

Q: Can I see the model weights and scripts of proprietary models in preview with Amazon SageMaker JumpStart?
No. Proprietary models do not allow customers to view model weights and scripts.

Q: Which open-source models are supported with Amazon SageMaker JumpStart?
Amazon SageMaker JumpStart includes 150+ pre-trained open-source models from PyTorch Hub and TensorFlow Hub. For vision tasks such as image classification and object detection, you can use models such as ResNet, MobileNet, and Single-Shot Detector (SSD). For text tasks such as sentence classification, text classification, and question answering, you can use models such as BERT, RoBERTa, and DistilBERT.

Q: What solutions come pre-built with Amazon SageMaker JumpStart?
SageMaker JumpStart includes solutions that are preconfigured with all necessary AWS services to launch a solution into production. Solutions are fully customizable so you can easily modify them to fit your specific use case and dataset. You can use solutions for over 15 use cases including demand forecasting, fraud detection, and predictive maintenance, and readily deploy solutions with just a few clicks. For more information about all solutions available, visit the SageMaker getting started page. 

Q: What built-in algorithms are supported in Amazon SageMaker Autopilot?
Amazon SageMaker Autopilot supports 2 built-in algorithms: XGBoost and Linear Learner.

Q: Can I stop an Amazon SageMaker Autopilot job manually?
Yes. You can stop a job at any time. When an Amazon SageMaker Autopilot job is stopped, all ongoing trials will be stopped and no new trial will be started.
##

Create a multiple choice quiz on the topic of SageMaker Low-code ML FAQ consisting of 4 questions. Each question should have 4 options. Also include the correct answer for each question using the starting string 'Correct Answer:`

Rapid engineering is an iterative process. You need to be clear and specific, and give the model time to think.

Here we have specified the context with ## as stop sequences, which tells the model to stop generating after generating this character or string. It is useful when using a few shots indication.

Below is SageMaker Low-code ML FAQ:

##
<SageMaker Low-code ML FAQ content>
##

Next, we are clear and very specific in our message, asking for a multiple-choice quiz, consisting of four questions with four options. We ask the model to include the correct answer for each question using the initial string 'Correct Answer:' so we can parse it later using Python:

Create a multiple choice quiz on the topic of SageMaker Low-code ML FAQ consisting of 4 questions. Each question should have 4 options. Also include the correct answer for each question using the starting string 'Correct Answer:`

A well-designed message can make the model more creative and generalizable so that it can be easily adapted to new tasks. Cues can also help embed domain knowledge in specific tasks and improve interpretability. Fast engineering can greatly improve the performance of zero- and few-shot learning models. Creating high-quality cues requires careful consideration of the task at hand, as well as a deep understanding of the model’s strengths and limitations.

In the scope of this publication, we do not cover this broad area further.

Copy the request and enter it in the file warning box, and then choose Generate text.

This sends the request to Jurassic-2’s Jumbo Instruct model for inference. Please note that experimenting in the yard is free.

Also note that despite the cutting-edge nature of LLMs, they are still prone to biases, errors, and hallucinations.

After reading the model output thoroughly and carefully, we can see that the model generated a pretty good quiz!

After you’ve played around with the model, it’s time to use the notebook and deploy it as an endpoint to your environment. We use a small Python function to parse the output and simulate an interactive test.

Deploy the Jurassic-2 Jumbo Instruct base model from a notebook

You can use the sample notebook below to deploy Jurassic-2 Jumbo Instruct using SageMaker. Note that this example uses an ml.p4d.24xlarge instance. If your default limit for your AWS account is 0, you will need to request a limit increase for this GPU instance.

We create the endpoint using SageMaker inference. First, we set the necessary variables and then deploy the model from the model package:

endpoint_name = "j2-jumbo-instruct"

content_type = "application/json"

real_time_inference_instance_type = (
"ml.p4d.24xlarge"
)

# create a deployable model from the model package.
model = ModelPackage(
role=role, model_package_arn=model_package_arn, sagemaker_session=sagemaker_session
)

# Deploy the model
predictor = model.deploy(1, real_time_inference_instance_type, endpoint_name=endpoint_name,
model_data_download_timeout=3600,
container_startup_health_check_timeout=600,
)

Once the endpoint is deployed, you can run inference queries against the model.

Once the model is deployed, you can interact with the deployed endpoint using the following code snippet:

response = ai21.Completion.execute(sm_endpoint=endpoint_name,
prompt=instruction,
maxTokens=2048,
temperature=0.7,
numResults=1,
stopSequences=['##'])

output = response['completions'][0]['data']['text']

With the Jurassic-2 Jumbo Instruct base model deployed on a SageMaker ml.p4d.24xlarge endpoint, you can use a gauge with 4,096 tiles. You can follow the same prompt we used in the playground and add many more questions. In this example, we’ve added the entire FAQ Low code ML section as context in the indicator.

We can see the output of the model, which generated a multiple choice quiz with four questions and four options for each question.

Now you can develop a Python function to parse the output and create an interactive multiple-choice quiz.

It is quite simple to develop this function with a few lines of code. You can easily analyze the answer because the model created a line with “Correct Answer:” for each question, exactly as we asked you in the application. We do not provide the Python code for generating the quiz within the scope of this post.

Run the test in the notebook

Using the Python function we created earlier and the output of the Jurassic-2 Jumbo Instruct base model, we run the interactive quiz in the notebook.

You can see that I answered three of the four questions correctly and got a mark of 75%. Maybe I need to read the SageMaker FAQ a few more times!

Clean up

After you have tested the endpoint, be sure to remove the SageMaker inference endpoint and the model to avoid charges:

model.sagemaker_session.delete_endpoint(endpoint_name)
model.sagemaker_session.delete_endpoint_config(endpoint_name)

model.delete_model()

conclusion

In this post, we showed you how you can test and use AI21’s Jurassic-2 Jumbo Instruct model using SageMaker to create an automated question generation system. This was achieved using a fairly simple prompt with embedded text from the publicly available SageMaker FAQ page and a few lines of Python code.

Similar to this example mentioned in the post, you can customize a base model for your business with just a few labeled examples. Because all data is encrypted and never leaves your AWS account, you can trust that your data will remain private and confidential.

Request access to test the base model in SageMaker today and let us know your feedback.

About the author

Eitan Sela is a solutions architect specializing in machine learning with Amazon Web Services. Work with AWS customers to provide guidance and technical support, helping them build and operate machine learning solutions on AWS. In his free time, Eitan enjoys running and reading the latest machine learning articles.