Segment Anything Model Explained


Hi Reader,,

Welcome to the PYCAD newsletter, where every week you receive doses of machine learning and computer vision techniques and tools to help you learn how to build AI solutions to empower the most vulnerable members of our society, patients.

Segment Anything Model

Segment Anything Model or SAM for short is one of the coolest computer vision models that we saw lately.
​
Here's how it works.
​
The model can be split into 2 parts: inputs encoder and mask decoder.
​
The inputs of the model are:
- An image.
- A prompt.
​
The outputs of the model are 3 valid masks. They represent:
​
- The whole of an object.
- A part of an object.
- A subpart of an object.
​
There are 2 types of prompts: Sparse and dense.
​
Sparse prompts can be:
​
- Text.
- Point coordinates.
- Bounding box coordinates.
​
There is only one type of dense prompts: a mask (not to be confused with the output masks!).
​
Now, let's go through how inputs and outputs are mapped.
​
The image goes through an encoder, which is a vision transformer (ViT).
​
The prompt goes through an encoder as well, but there are 2 different encoders depending on the type of the prompt:
​
- If the prompt is sparse, then the text encoder from CLIP is used to encode the information.
- If the prompt is dense (a mask), then a CNN is used to encode the information.
​
The mask decoder efficiently maps the image embedding, prompt embeddings, and an output token to a mask.
​
For more details about the model, check out Facebook's blog. Also, you can check the code on github.

​

Generative AI Courses by Google

Google just released a new learning path for Generative Machine Learning. It looks awesome!
​
Here are the Courses found in this learning path:
​
- Intro to Generative AI.
- Intro to Large Language Models.
- Intro to Responsible AI.
- Intro to Image Generation.
- Encoder-Decoder.
- Attention Mechanism.
- Transformers and BERT Models.
- Create Image Captioning Models.
- Intro to Gen AI Studio.

Check out the courses here.


Cool AI Tools (Affiliates)

​Chatbase: An AI chatbot builder that lets you build, train, and embed smart chatbots powered by ChatGPT right on your website.

​EzyCourse​: Create and sell your courses, services, products, or online communities from one platform.

​Katteb: The first fact-checked, real-time, and localized AI writer.


​

What'd you think of today's edition?

​

Machine Learning for Medical Imaging

πŸ‘‰ Learn how to build AI systems for medical imaging domain by leveraging tools and techniques that I share with you! | πŸ’‘ The newsletter is read by people from: Nvidia, Baker Hughes, Harvard, NYU, Columbia University, University of Toronto and more!

Read more from Machine Learning for Medical Imaging

Hello Reader, Welcome to another edition of PYCAD newsletter where we cover interesting topics in Machine Learning and Computer Vision applied to Medical Imaging. The goal of this newsletter is to help you stay up-to-date and learn important concepts in this amazing field! I've got some cool insights for you below ↓ DICOM Viewer with Volume Rendering What you're seeing below is a DICOM viewer with 4 views: axial, coronal, sagittal and volume rendering. All integrated into one web app!We've...

Hello Reader, I haven't sent the newsletter in a while! Sorry about that! I have been extremely busy with our agency PYCAD. We've been working on several projects for our clients and it's taking most of my time. Since this is the first newsletter I send in 2025, I thought what's better than sharing the new trends in AI for MedTech! So below, you'll see some of the trends and interesting news that are coming from the field! Top Healthcare Tech Trends for 2025 When it comes to healthcare...

Hello Reader, Welcome to another edition of PYCAD newsletter where we cover interesting topics in Machine Learning and Computer Vision applied to Medical Imaging. The goal of this newsletter is to help you stay up-to-date and learn important concepts in this amazing field! I've got some cool insights for you below ↓ How to transcribe medical prescriptions with AI Lately I was chatting with a business man who has done amazing things in the medical imaging field. While conversing, he told me...