Deal of The Day! Hurry Up, Grab the Special Discount - Save 25% - Ends In 00:00:00 Coupon code: SAVE25
Welcome to Pass4Success

- Free Preparation Discussions
Mail Us support@pass4success.com
Location US
Mob_nav_barsMENU

Amazon Exam MLS-C01 Topic 1 Question 106 Discussion

Actual exam question for Amazon's MLS-C01 exam
Question #: 106
Topic #: 1
[All MLS-C01 Questions]

An insurance company developed a new experimental machine learning (ML) model to replace an existing model that is in production. The company must validate the quality of predictions from the new experimental model in a production environment before the company uses the new experimental model to serve general user requests.

Which one model can serve user requests at a time. The company must measure the performance of the new experimental model without affecting the current live traffic

Which solution will meet these requirements?

Show Suggested Answer Hide Answer
Suggested Answer: C

The best solution for this scenario is to use shadow deployment, which is a technique that allows the company to run the new experimental model in parallel with the existing model, without exposing it to the end users. In shadow deployment, the company can route the same user requests to both models, but only return the responses from the existing model to the users.The responses from the new experimental model are logged and analyzed for quality and performance metrics, such as accuracy, latency, and resource consumption12. This way, the company can validate the new experimental model in a production environment, without affecting the current live traffic or user experience.

The other solutions are not suitable, because they have the following drawbacks:

A: A/B testing is a technique that involves splitting the user traffic between two or more models, and comparing their outcomes based on predefined metrics.However, this technique exposes the new experimental model to a portion of the end users, which might affect their experience if the model is not reliable or consistent with the existing model3.

B: Canary release is a technique that involves gradually rolling out the new experimental model to a small subset of users, and monitoring its performance and feedback.However, this technique also exposes the new experimental model to some end users, and requires careful selection and segmentation of the user groups4.

D: Blue/green deployment is a technique that involves switching the user traffic from the existing model (blue) to the new experimental model (green) at once, after testing and verifying the new model in a separate environment.However, this technique does not allow the company to validate the new experimental model in a production environment, and might cause service disruption or inconsistency if the new model is not compatible or stable5.

References:

1:Shadow Deployment: A Safe Way to Test in Production | LaunchDarkly Blog

2:Shadow Deployment: A Safe Way to Test in Production | LaunchDarkly Blog

3:A/B Testing for Machine Learning Models | AWS Machine Learning Blog

4:Canary Releases for Machine Learning Models | AWS Machine Learning Blog

5:Blue-Green Deployments for Machine Learning Models | AWS Machine Learning Blog


by Ira at Sep 16, 2024, 02:52 AM

Limited Time Offer

25%

Off

Get Premium MLS-C01 Questions as Interactive Web-Based Practice Test or PDF

Contribute your Thoughts:

Submit Cancel
Irene
10 days ago
As a model enthusiast, I'm all about option C. Shadow deployment is the perfect way to put the new model through its paces without causing any drama. Now, if only I could train my cat to be a machine learning expert...
upvoted 0 times
...
Mari
15 days ago
Can you explain why you think Canary release is the best option?
upvoted 0 times
...
Glendora
21 days ago
This is a tough one, but I think option C is the winner. Shadow deployment sounds like the safest way to test the new model without causing any major disruptions. Kudos to the insurance company for being so cautious!
upvoted 0 times
...
Hobert
23 days ago
I'm feeling a bit mischievous today, so I'll go with option B - Canary release. Who doesn't love a little bit of controlled chaos, am I right?
upvoted 0 times
Mozelle
3 days ago
Definitely! It's a smart way to ensure the new model is ready to serve user requests without any disruptions.
upvoted 0 times
...
Sabra
5 days ago
I agree, it's a great way to validate the new model's performance in a production environment.
upvoted 0 times
...
Kanisha
16 days ago
Canary release sounds like a fun choice! It allows you to test the new model without affecting live traffic.
upvoted 0 times
...
...
Ashlyn
30 days ago
I disagree, I believe the correct answer is B) Canary release.
upvoted 0 times
...
Svetlana
1 months ago
Definitely, shadow deployment is the way to go here. Measure the performance of the new model without affecting the current users? Sign me up!
upvoted 0 times
...
Mari
1 months ago
I think the answer is C) Shadow deployment.
upvoted 0 times
...
Anthony
1 months ago
Hmm, this seems like a classic case of trying to test the new model without disrupting the current live traffic. I'd go with option C - shadow deployment, seems like the most logical choice.
upvoted 0 times
Earleen
16 days ago
I agree, shadow deployment is the best solution for testing the new model in a production environment.
upvoted 0 times
...
Cheryl
17 days ago
Option C - shadow deployment allows the company to test the new model without affecting live traffic.
upvoted 0 times
...
...

Save Cancel