AWS Certified AI Practitioner - (AIF-C01) Exam Questions

Correct option:

The company should enable model invocation logging, which allows for detailed logging of all requests and responses during model invocations in Amazon Bedrock

You can use model invocation logging to collect invocation logs, model input data, and model output data for all invocations in your AWS account used in Amazon Bedrock. With invocation logging, you can collect the full request data, response data, and metadata associated with all calls performed in your account. Logging can be configured to provide the destination resources where the log data will be published. Supported destinations include Amazon CloudWatch Logs and Amazon Simple Storage Service (Amazon S3). Only destinations from the same account and region are supported. Model invocation logging is disabled by default.

This is the correct option because enabling invocation logging on Amazon Bedrock allows the company to capture detailed logs of all model requests and responses, including input data, output predictions, and any errors that occur during model execution. This method provides comprehensive monitoring capabilities, enabling the company to effectively track, audit, and troubleshoot model performance and usage.

via - https://aws.amazon.com/blogs/mt/monitoring-generative-ai-applications-using-amazon-bedrock-and-amazon-cloudwatch-integration/

Incorrect options:

The company should use AWS CloudTrail logs for tracking Amazon Bedrock API calls, which records API activities and calls made to various AWS services, including Amazon Bedrock - While AWS CloudTrail is useful for tracking API calls and monitoring who accessed which AWS resources, it does not capture the actual input and output data involved in model invocations. CloudTrail logs are primarily intended for auditing access and managing security rather than monitoring detailed data flow or model performance on Amazon Bedrock.

The company should analyze Amazon Bedrock model invocation events in Amazon EventBridge - Amazon EventBridge is designed to react to changes and events across AWS resources and trigger workflows or automate responses. Although it can track when a model invocation occurs, it does not provide detailed logging of the input and output data associated with these invocations, limiting its usefulness for comprehensive monitoring purposes.

The company should use AWS Config to monitor changes in resource configurations initiated by Amazon Bedrock - AWS Config is specifically designed for monitoring and managing AWS resource configurations and compliance, not for tracking or logging the input and output data of machine learning models on Amazon Bedrock. AWS Config focuses on configuration management and does not provide the level of detail required to monitor data traffic or model performance in machine learning applications.

References:

https://docs.aws.amazon.com/bedrock/latest/userguide/model-invocation-logging.html

https://aws.amazon.com/blogs/mt/monitoring-generative-ai-applications-using-amazon-bedrock-and-amazon-cloudwatch-integration/

Correct option:

Define the use case of the application narrowly

A narrowly defined use case provides clear and specific requirements for the application, helping the research team understand exactly what the model needs to accomplish. This clarity is crucial for selecting the most appropriate model that fits the specific needs and constraints of the application.

Incorrect options:

Define the target audience of the application broadly - Defining the target audience broadly can lead to ambiguity and lack of focus on the specific requirements of the application. It can make it challenging to tailor the model to meet precise needs.

Identify potential data sources for the application - While identifying data sources is important, it does not directly ensure the selection of the best model. It is a preliminary step that supports data collection rather than model selection.

Determine the cost constraints for model training - Determining cost constraints is important for budget management, but it does not directly influence the selection of the best model for the application. Cost constraints affect the resources available for model training rather than the suitability of the model itself.

Reference:

https://aws.amazon.com/what-is/machine-learning/

Correct option:

OpenSearch Serverless vector store

Knowledge Bases for Amazon Bedrock takes care of the entire ingestion workflow of converting your documents into embeddings (vector) and storing the embeddings in a specialized vector database. Knowledge Bases for Amazon Bedrock supports popular databases for vector storage, including vector engine for Amazon OpenSearch Serverless, Pinecone, Redis Enterprise Cloud, Amazon Aurora, and MongoDB.

If you do not have an existing vector database, Amazon Bedrock creates an OpenSearch Serverless vector store for you.

via - https://aws.amazon.com/bedrock/knowledge-bases/

Incorrect options:

Redis Enterprise Cloud

Amazon Aurora

MongoDB

These three options contradict the explanation provided above, so these options are incorrect.

References:

https://aws.amazon.com/bedrock/knowledge-bases/

https://aws.amazon.com/about-aws/whats-new/2024/04/watermark-detection-amazon-titan-image-generator-bedrock/

Correct option:

This concept is referred to as a context window, which determines the amount of text or information the model can consider at once while generating a response, typically measured in tokens rather than characters

The context window is the correct answer because it defines how much text (measured in tokens) the AI model can process at one time to generate a coherent output. It determines the limit of input data that the model can use to understand context, maintain conversation history, or generate relevant responses. The context window is measured in tokens (units of text), not characters, making it the key concept for understanding data processing limits in AI models.

via - https://aws.amazon.com/blogs/security/context-window-overflow-breaking-the-barrier/

Incorrect options:

This concept is referred to as character count, which directly counts the number of individual characters in a text - Character count measures the number of characters in a piece of text, but AI models typically do not limit their input based on characters alone. Instead, they rely on tokens, which can represent words, subwords, or punctuation marks. The concept that defines how much text can be processed at one time is the context window, which is measured in tokens, not character count.

This concept is called tokens, which are units of text (like words or subwords) that the model processes - While tokens are the individual units of text that the model processes, the concept that describes the total amount of text the model can handle at one time is the context window, not tokens themselves. Tokens are components within the context window, and the model's capacity is defined by how many tokens can fit within this window, rather than just the tokens themselves.

This concept is known as embeddings, which are numerical representations of text data used by AI models to understand and process the meaning of words in context - Embeddings are vector representations that encode the semantic meaning of words or phrases, enabling the AI model to understand relationships and context in text data. However, embeddings do not define the amount of text or the number of characters considered at one time; they are a representation technique used within the model once the text is processed.

Reference:

https://aws.amazon.com/blogs/security/context-window-overflow-breaking-the-barrier/

Correct option:

The company should use hyperparameters for model tuning, which involves adjusting parameters such as regularization, learning rates, and dropout rates to enhance the model's ability to generalize well to new data

Hyperparameter tuning is the most effective solution in this scenario because it allows the company to adjust the settings that control the learning process of the model. By fine-tuning hyperparameters, such as increasing regularization or early stopping or adjusting dropout rates, the model can avoid overfitting to the training data and better generalize to new, unseen data in production. This approach helps improve the model's performance across various data distributions.

via - https://aws.amazon.com/what-is/overfitting/

Incorrect options:

The company should increase the amount of training data, which can help the model learn more diverse patterns and improve its performance on new, unseen data by exposing it to a wider range of examples - Increasing the amount of training data can improve a model's generalization by providing it with more examples and reducing the risk of overfitting. However, this approach is only effective if additional data is representative of the new data encountered in production. It is a less direct and potentially more resource-intensive solution compared to hyperparameter tuning, which specifically targets the overfitting problem.

The company should reduce the amount of training data, which can help eliminate noise in the data - Reducing the amount of training data is generally not a recommended approach for improving model generalization, as it can result in underfitting. Underfitting occurs when a model is too simple or does not have enough information to learn meaningful patterns, leading to poor performance on both training and new data. Therefore, reducing the training data would likely exacerbate the problem rather than solve it.

The company should swap the existing model with a state-of-the-art generative AI model - This options acts as a distractor, as it does not address the underlying issue of model-overfitting.

References:

https://aws.amazon.com/what-is/overfitting/

https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-fine-tune.html

https://docs.aws.amazon.com/whitepapers/latest/ml-best-practices-public-sector-organizations/model-training-and-tuning.html

Correct option:

Generative AI can automate the creation of new data based on existing patterns, enhancing productivity and innovation

Generative AI in the AWS cloud environment is advantageous because it automates the creation of new data from existing patterns, which can significantly boost productivity and drive innovation. This capability allows businesses to generate new insights, designs, and solutions more efficiently.

via - https://aws.amazon.com/what-is/generative-ai/

Incorrect options:

Generative AI can replace all human roles in software development - Generative AI is not designed to replace all human roles in software development but to assist and enhance human capabilities by automating certain tasks and creating new data based on patterns. So, this option is incorrect.

Generative AI ensures 100% security against all cyber threats - While generative AI can improve security by identifying patterns and anomalies, it does not guarantee 100% security against all cyber threats. Security in the cloud involves a combination of multiple strategies and tools. Therefore, this option is incorrect.

Generative AI can perform all cloud maintenance tasks without any human intervention - Generative AI can assist in cloud maintenance tasks by predicting issues and suggesting solutions, but it cannot perform all maintenance tasks without human oversight and intervention. So, this option is not the right fit.

References:

https://aws.amazon.com/what-is/generative-ai/

https://aws.amazon.com/ai/generative-ai/services/

Correct option:

Top K

Top K represents the number of most likely candidates that the model considers for the next token. Choose a lower value to decrease the size of the pool and limit the options to more likely outputs. Choose a higher value to increase the size of the pool and allow the model to consider less likely outputs.

via - https://docs.aws.amazon.com/bedrock/latest/userguide/inference-parameters.html

Incorrect options:

Temperature - Temperature is a value between 0 and 1, and it regulates the creativity of the model's responses. Use a lower temperature if you want more deterministic responses, and use a higher temperature if you want more creative or different responses for the same prompt on Amazon Bedrock.

Top P - Top P represents the percentage of most likely candidates that the model considers for the next token. Choose a lower value to decrease the size of the pool and limit the options to more likely outputs. Choose a higher value to increase the size of the pool and allow the model to consider less likely outputs.

Stop sequences - Stop sequences specify the sequences of characters that stop the model from generating further tokens. If the model generates a stop sequence that you specify, it will stop generating after that sequence.

References:

https://docs.aws.amazon.com/bedrock/latest/userguide/inference-parameters.html

https://docs.aws.amazon.com/bedrock/latest/userguide/general-guidelines-for-bedrock-users.html

Correct option:

Enable multilingual user experiences in your applications

Amazon Translate is a text translation service that uses advanced machine learning technologies to provide high-quality translation on demand. You can enable multilingual user experiences in your applications by integrating Amazon Translate.

Amazon Rekognition cannot be used to create multilingual user experiences.

Incorrect options:

Face-based user identity verification

Searchable media libraries

Celebrity recognition

Face-based user identity verification, searchable media libraries, and celebrity recognition are classic use cases of Amazon Rekognition.

Reference:

https://docs.aws.amazon.com/translate/latest/dg/what-is.html

Correct option:

Utilize a Retrieval-Augmented Generation (RAG) system by indexing all product catalog PDFs and configuring the LLM chatbot to reference this system for answering queries

Using a RAG approach is the least costly and most efficient solution for providing up-to-date and relevant responses. In this approach, you convert all product catalog PDFs into a searchable knowledge base. When a customer query comes in, the RAG framework first retrieves the most relevant pieces of information from this knowledge base and then uses an LLM to generate a coherent response based on the retrieved context. This method does not require re-training the model or modifying every incoming query with large datasets, making it significantly more cost-effective. It ensures that the chatbot always has access to the most recent information without needing expensive updates or processing every time.

via - https://aws.amazon.com/bedrock/knowledge-bases/

Incorrect options:

Attach the contents of all product catalog PDFs to each customer query request to ensure the LLM has all available information for generating responses - Attaching all PDF content to every query is highly inefficient and expensive. This method significantly increases data transmission costs, processing time, and latency because the LLM would have to parse extensive amounts of information with each query, making it impractical for real-time interactions. It also burdens the LLM's context window, potentially degrading the quality of responses.

Attach the contents of the single product catalog PDF to each customer query request to ensure the LLM has the relevant information for generating responses - This option acts as a distractor. There is no low-cost way of identifying the single product catalog PDF that is relevant to the given query request. You would need to code a custom solution to search the most relevant PDF for each query and then attach it to the query context.

Fine-tune the LLM with data extracted from the product catalog PDFs to enhance its ability to provide accurate responses - Fine-tuning the LLM using product PDF data is a costly and resource-intensive approach. It requires substantial computational resources and time to adjust the model's parameters to a specific dataset. Moreover, this solution is not scalable; the model would need to be re-fine-tuned frequently whenever the product literature is updated, resulting in recurring costs and downtime.

References:

https://aws.amazon.com/bedrock/knowledge-bases/

https://aws.amazon.com/what-is/retrieval-augmented-generation/

Correct option:

Generative AI encompasses models and algorithms capable of creating new content such as text, images, and audio based on patterns learned from existing data

Generative artificial intelligence (generative AI) is a type of AI that can create new content and ideas, including conversations, stories, images, videos, and music. AI technologies attempt to mimic human intelligence in nontraditional computing tasks like image recognition, natural language processing (NLP), and translation.

Incorrect options:

Generative AI refers to algorithms that analyze existing data to generate new insights without creating new content - Generative AI is not just about generating insights but also about creating new content based on learned patterns.

Generative AI refers to AI systems that are limited to performing predefined tasks without adapting to new data or contexts - Generative AI is capable of adapting and creating new content, not just performing predefined tasks.

Generative AI is a subset of AI that focuses exclusively on improving data retrieval efficiency - Generative AI is not limited to improving data retrieval; it is focused on content creation.

Reference:

https://aws.amazon.com/what-is/generative-ai/