Microsoft AI-300 Dumps

Managed online endpoints with traffic splitting are the best fit because Azure Machine Learning supports multiple deployments behind a single online endpoint and allows production traffic to be distributed between them. This enables a blue-green or progressive rollout strategy : deploy the new model version alongside the current version, validate it independently, send a small percentage of production traffic to it, monitor behavior, and then gradually increase traffic. If problems occur, traffic can be redirected immediately to the previous deployment, providing a practical rollback mechanism.

This approach also aligns with Fabrikam’s requirement to favor managed services and automation because the team has limited Azure DevOps experience. Managed online endpoints handle much of the serving, scaling, security, and monitoring infrastructure automatically and support both traffic splitting and mirrored traffic for safer validation.

Option B can also implement blue-green deployment, but AKS introduces substantially more infrastructure and operational management than Fabrikam requires. Option A would require custom deployment, routing, scaling, and rollback logic. Option D is intended for asynchronous batch inference rather than staged real-time production serving.

Study Guide Reference: Implement machine learning model lifecycle and operations — managed online endpoints, multiple deployments, traffic splitting, blue-green deployment, staged rollout, monitoring, and rollback.

The first action should be to evaluate the model output . Fabrikam already has evaluation data containing manually defined expected responses, and the technical requirements explicitly state that the organization must use its existing evaluation datasets based on real data with input-output pairs . Microsoft Foundry supports evaluating a model against an existing curated dataset to establish measurable baseline performance before optimization. Evaluation results can identify weaknesses in accuracy, relevance, groundedness, safety, or other quality dimensions and provide objective evidence for subsequent changes.

This sequencing is critical because Fabrikam also requires that advanced fine-tuning be applied only when prompt engineering is insufficient . Therefore, immediately fine-tuning the model would bypass the required baseline evaluation and optimization process. Fine-tuning should be driven by demonstrated performance gaps, not applied automatically.

Deploying directly to production is also inappropriate because leadership requires low operational risk and gradual rollout. Production traffic should not be used as the first evaluation mechanism when curated evaluation data already exists. Synthetic interaction data is useful when evaluation coverage is insufficient, but Microsoft specifically positions synthetic generation as an option when an adequate test dataset is unavailable. Fabrikam already has suitable evaluation data.

Study Guide Reference: Optimize generative AI systems and model performance — evaluate model performance, benchmark against curated datasets, iterate on prompts, and apply advanced fine-tuning when justified.

MLflow is the industry-standard open-source platform for experiment tracking, and Azure Machine Learning has first-class native integration with it. When you use MLflow within an Azure ML job, parameters, metrics, and artifacts are automatically logged to the run history of the AML workspace, making every run reproducible and comparable. Option A (AML job output logs) only captures console output and lacks structured parameter and metric logging. Option C (Application Insights logs) is designed for application-level telemetry, not ML experiment metadata. Option D (Azure Monitor alerts) is a reactive notification tool, not a tracking system. MLflow ' s autologging capability means that for common frameworks such as scikit-learn, XGBoost, and PyTorch, parameters and metrics are captured without a single line of custom code, directly answering Fabrikam ' s requirement for consistent experiment tracking.

Microsoft Learn Reference Topic: Track ML experiments with MLflow – Azure Machine Learning MLflow integration

In Azure Machine Learning, a workspace is the top-level resource that acts as a centralized hub for every ML activity. It stores experiments, pipelines, datasets, models, environments, and compute targets all in one place. Before you can register assets or deploy endpoints, the workspace itself must exist and be shared across the team. Fabrikam ' s core challenge is inconsistent experiment tracking, ad-hoc versioning, and no standardized deployment path — all symptoms of the absence of a single, governed workspace. Option A (registering assets) is a subsequent action only possible after the workspace exists. Option C (online endpoint) is a deployment concern, not an asset-management foundation. Option D (Foundry project) is appropriate for generative AI workloads, not for the traditional ML pipeline standardization described here. The workspace is the prerequisite that makes every other governance action possible.

Microsoft Learn Reference Topic: Azure Machine Learning workspaces – Microsoft Learn: Manage Azure Machine Learning workspaces

Domain specialization: Supervised fine-tuning

Poor response accuracy of a RAG-based solution: Apply prompt engineering

For domain specialization , Supervised Fine-Tuning (SFT) is the correct first strategy. Microsoft describes SFT as the foundational fine-tuning technique for training a model from labeled input-output pairs , and specifically identifies domain specialization as one of its principal use cases. Microsoft also recommends starting with SFT for most customization projects because it supports task specialization, instruction following, style, and domain-specific behavior. Fabrikam already possesses evaluation data containing input-output pairs, which aligns directly with the SFT data model.

For poor RAG response accuracy , the first action is prompt engineering . The case explicitly requires advanced fine-tuning only when prompt engineering is insufficient. In a RAG system, prompt engineering determines how the model interprets retrieved context, constrains answers to grounding information, handles missing evidence, and formats responses. Microsoft notes that inadequate RAG prompting can produce false or incomplete answers even when retrieval returns appropriate content.

DPO is primarily appropriate for alignment using preferred versus non-preferred responses, while RFT targets complex reward-based reasoning optimization. Neither is the initial technique for domain specialization in this requirement.

Study Guide Reference: Optimize generative AI systems and model performance — prompt engineering, RAG optimization, supervised fine-tuning, preference optimization, and model customization strategy.

Imagine a hospital analytics firm with data scientists who kick off dozens of training jobs throughout the week. During peak hours, five jobs compete for the same GPU cluster and fail or queue for hours. On quiet nights, that cluster sits completely idle, burning money. Managed compute targets with autoscaling solve both problems: the cluster scales out automatically when multiple jobs arrive simultaneously and scales back to zero when idle. Option A (single shared cluster) is exactly the resource-contention problem Fabrikam already has. Option B (fixed-size cluster) wastes money during off-peak hours. Option C (dedicated per-experiment clusters) eliminates contention but is prohibitively expensive for a cost-conscious healthcare firm. Autoscaling managed compute is the cloud-native answer to variable workload demand.

Microsoft Learn Reference Topic: Azure Machine Learning compute targets – Configure autoscale for compute clusters

A multi-turn chatbot application intermittently produces responses that are grammatically correct and on-topic but contradict earlier turns in the conversation, creating a confusing user experience. CoherenceEvaluator measures exactly this: whether the flow of ideas across a multi-turn conversation is logically consistent and non-contradictory without requiring an external knowledge base. RelevanceEvaluator (option A) measures whether responses are topically on-point but often requires a reference context or knowledge base. SimilarityEvaluator (option B) requires a reference answer for comparison. QAEvaluator (option C) is a composite evaluator for question-answering tasks that requires a ground-truth context document. CoherenceEvaluator is the only option that works purely from the conversation history itself with no external knowledge base, perfectly matching the stated constraint and the multi-turn chatbot use case.

Microsoft Learn Reference Topic: Evaluate conversational AI applications in Microsoft Foundry – CoherenceEvaluator for multi-turn chatbots

Git commits on the main branch provide the immutable, ordered history that auditors need: every change is timestamped, attributed to a specific developer, and reversible via git revert — this is the audit trail and rollback mechanism required by the first requirement. Git branches allow developers to create and test multiple prompt variants in complete isolation from the production prompt on the main branch, addressing the second requirement. A developer on an experiment branch can run full evaluations without touching the production prompt. When a variant is approved, it is merged via pull request. You cannot use branches alone for audit history because branches can be deleted, and you cannot use main-branch commits alone for parallel variant comparison without disrupting the history. Git ' s branch-and-merge model provides both capabilities simultaneously.

Microsoft Learn Reference Topic: Version control for AI prompts – Git branch strategies for prompt management in Microsoft Foundry

Azure Machine Learning ' s experiment tracking is built around two complementary concepts. First, experiments are named containers that group related runs. By calling mlflow.set_experiment with an experiment name at the start of your training code, all subsequent runs are grouped under that experiment name in the AML workspace, creating the centralized record required. Second, metrics are scalar values such as accuracy, loss, or AUC that represent model performance. Calling mlflow.log_metric with a metric name and value during training persists these values to the run ' s record in the experiment history. These values appear on the Azure ML Studio run detail page and can be compared across runs using the experiment comparison view. Without set_experiment, runs fall into a default experiment. Without log_metric, the run history has no performance data to display or compare.

Microsoft Learn Reference Topic: Track machine learning experiments with MLflow in Azure Machine Learning

Azure Machine Learning distinguishes between uri_folder, uri_file, and mltable data assets. Because the registered asset in this scenario is a folder data asset containing CSV data , the appropriate MLTable API is mltable.from_delimited_files(). Microsoft documents that a registered uri_folder containing CSV files can be retrieved through MLClient, its storage path supplied as a folder path, and then passed to mltable.from_delimited_files(). The resulting MLTable object can subsequently be materialized into a Pandas DataFrame by calling to_pandas_dataframe().

A representative workflow is conceptually:

tbl = mltable.from_delimited_files(paths=[{ " folder " : data_asset.path}])

followed by:

df = tbl.to_pandas_dataframe()

mltable.load() is primarily appropriate when loading an existing MLTable definition , such as a registered table data asset containing an MLTable file. from_parquet_files() is intended for Parquet-formatted data, while from_delta_lake() targets Delta Lake data sources. Neither matches the CSV-based folder described here. Microsoft also identifies uri_folder as the appropriate asset type for folders containing CSV or Parquet files used with Pandas or Spark.

Study Guide Reference: Design and implement an MLOps infrastructure — Azure Machine Learning data assets, MLTable APIs, interactive data access, and Pandas-based data wrangling.

Continue with Azure ML data access

In Microsoft Foundry ' s built-in evaluation framework, each evaluator measures a specific dimension of language quality. Fluency evaluates the grammatical correctness, sentence structure, and overall linguistic smoothness of generated text. A high fluency score means the response reads naturally, with proper grammar, punctuation, and sentence construction — regardless of factual accuracy or relevance to the topic. Coherence (option A) measures whether ideas flow logically from one sentence to the next — it is about logical structure, not grammar. Textual Similarity (option B) measures how closely the generated text matches a reference text using metrics like ROUGE or BLEU — it is a comparison metric, not a standalone quality dimension. Groundedness (option C) measures whether claims are supported by the provided context — it is a factual fidelity metric. For grammatical correctness specifically, Fluency is the correct evaluator.

Microsoft Learn Reference Topic: Built-in AI evaluators in Microsoft Foundry – Fluency, Coherence, Groundedness, and Similarity

According to Microsoft ' s Azure Machine Learning SDK v2 documentation on command jobs, input arguments in the command string must exactly match the argument names defined in the Python script ' s argparse configuration. The script reads an argument named training_data with an underscore, but the proposed command passes --trainingdata without the underscore. Python ' s argparse treats these as completely different argument names, so the script would receive None for its --training_data argument, causing a runtime error. The placeholder syntax for referencing the input in the command string is correct, which is a point in this solution ' s favor. However, the argument name mismatch alone is sufficient to make this solution fail. The corrected command would use --training_data exactly matching the argument name defined in the script.

Microsoft Learn Reference Topic: Configure command job inputs and outputs – Azure Machine Learning SDK v2 reference

Correct sequence:

Collect retrieval logs.

Run RAG evaluators.

Adjust chunking strategy.

Re-index documents.

The process should begin by collecting retrieval logs . Retrieval diagnostics provide the evidence needed to understand which queries were issued, which document chunks were returned, and whether relevant content is being surfaced. Without retrieval telemetry, optimization becomes speculative rather than measurable.

Next, run RAG evaluators . Microsoft Foundry provides retrieval-oriented evaluators that assess how relevant retrieved context is to a query. The Retrieval evaluator measures contextual relevance without requiring ground truth, while the Document Retrieval evaluator can calculate metrics such as Fidelity and NDCG when labeled retrieval ground truth exists. These measurements help establish the retrieval-quality baseline before modifications are made.

After identifying retrieval deficiencies, adjust the chunking strategy . Microsoft specifically recommends reviewing chunk size and chunking methodology when retrieval returns irrelevant or incomplete passages. Chunks that are too small can lose context, while oversized chunks can introduce irrelevant material and reduce retrieval precision.

Finally, re-index the documents so the revised chunking configuration is reflected in the searchable corpus. Changing temperature or regenerating the prompt template primarily affects generation behavior rather than correcting the underlying retrieval pipeline.

Study Guide Reference: Implement generative AI quality assurance and observability — RAG evaluation, retrieval telemetry, retrieval-quality metrics, chunk optimization, indexing, and grounded-response assessment.

The following custom role can do everything in the workspace except for the following actions:

It can ' t create or update a compute resource.

It can ' t delete a compute resource.

It can ' t add, delete, or alter role assignments.

It can ' t delete the workspace.

To create a custom role, first construct a role definition JSON file that specifies the permission and scope for the role. The following example defines a custom role named " Data Scientist Custom " scoped at a specific workspace level:

data_scientist_custom_role.json :

{

" Name " : " Data Scientist Custom " ,

" IsCustom " : true,

" Description " : " Can run experiment but can ' t create or delete compute. " ,

" Actions " : [ " * " ],

" NotActions " : [

" Microsoft.MachineLearningServices/workspaces/*/delete " ,

" Microsoft.MachineLearningServices/workspaces/write " ,

" Microsoft.MachineLearningServices/workspaces/computes/*/write " ,

" Microsoft.MachineLearningServices/workspaces/computes/*/delete " ,

" Microsoft.Authorization/*/write "

],

" AssignableScopes " : [

" /subscriptions/ < subscription_id > /resourceGroups/ < resource_group_name > /providers/Microsoft.MachineLearningServices/workspaces/ < workspace_name > "

]

}