01
AI Inference Integration
Applied CatVTON as the core virtual try-on model and built the inference flow required to generate fitting results from user, garment, and mask images.
GENERATIVE AI · COMPUTER VISION
DressMyFit
Generative AI
Virtual Fitting Service
PERIOD
2024
PROJECT
Sookmyung Women's University · Graduation Project
ROLE
AI Inference · Full Stack · Mask Generation
CatVTON · FastAPI · SAM · OpenCV · PyTorch · Python
01 — OVERVIEW
Turning a generative AI model into a personalized fitting service.
DressMyFit is a personalized virtual fitting web service built around CatVTON, a diffusion-based virtual try-on model.
Users can enter their body information, select a matching avatar or upload their own image, choose a garment and fitting type, and generate a virtual fitting result through the web service.
I worked across the AI inference pipeline, backend and frontend integration, and mask-generation logic required to adapt the model to personalized avatar images.
02 — SERVICE FLOW
From body information to virtual fitting result.
The service combines personalized avatar selection, garment processing, mask generation, and CatVTON inference into a single user-facing flow.
User
Input
→
Body / Image
User Information
→
Avatar
Body-type Matching
→
Garment
Top · Bottom · Dress
→
Mask
SAM · OpenCV
→
CatVTON
AI Inference
→
Result
Virtual Fitting
FastAPI connected the virtual try-on inference pipeline with the web application, allowing model execution to operate as part of the end-to-end service flow.
03 — MY ROLE
02
Backend & Frontend
Built the FastAPI backend and frontend interfaces and connected the AI inference process to the user-facing web service.
03
Mask Generation Improvement
Replaced inaccurate avatar-mask generation with SAM-based segmentation and OpenCV processing adapted to different garment categories.
04
Fitting Type Extension
Implemented Fit, Standard, and Loose fitting options by dynamically adjusting generated masks with OpenCV.
04 — PERSONALIZATION
BODY-TYPE AVATARS
Matching users to 40 body-type avatars.
Avatar images were prepared using Unity and the Ready Player Me SDK. The avatar set was divided by gender, BMI range, and height so that the service could display an avatar corresponding to the user's body information.
GENDER
2
Male and female avatar groups.
BMI
4
Four BMI ranges for body-type variation.
HEIGHT
5
Five height ranges from 140 cm to 190 cm.
TOTAL
40
Personalized avatar combinations.
Gender 2 × BMI 4 × Height 5=40 avatar types
05 — MASK GENERATION
CHALLENGE
The original mask-generation approach did not work reliably on avatar images.
The original virtual try-on pipeline relied on DensePose and SCHP-based cloth-agnostic masks for real human images. However, customized avatar images produced inaccurate masks, which directly affected the virtual fitting result.
ORIGINAL APPROACH
DensePose + SCHP
Designed around real human images, making body-part parsing and cloth-agnostic mask generation less reliable for customized avatars.
REDESIGNED APPROACH
SAM + OpenCV
Used SAM segmentation as the base mask and applied image-processing rules to generate masks adapted to avatar images and garment categories.
REDESIGNED MASK PIPELINE
Avatar Image
Input
SAM
Segmentation
Base Mask
Selected Region
OpenCV
Mask Processing
Garment Rule
Category Adjustment
CatVTON
Virtual Try-on
01
Top
Mask processing adapted to upper-body garments.
02
Bottom
Mask processing adapted to lower-body garments.
03
Dress
Mask processing adapted to garments spanning a larger body region.
SOLUTION
Instead of depending on a mask pipeline designed primarily for real human images, I used SAM to segment avatar regions and OpenCV operations to reshape the mask according to the selected garment category.
06 — FITTING TYPES
Extending one virtual try-on model into three fitting styles.
The generated mask was further adjusted using OpenCV so users could choose among three fitting types. The mask area changes according to the selected style before being passed to CatVTON.
01
Fit
A more closely fitted mask region for a tighter visual style.
02
Standard
The baseline fitting option using the standard adjusted mask.
03
Loose
An expanded mask region designed to represent a looser fitting style.
SAM Mask
Base Region
OpenCV
Morphological Processing
Fit Type
Fit · Standard · Loose
Adjusted Mask
Inference Input
CatVTON
Fitting Result
07 — SERVICE INTEGRATION
END-TO-END AI SERVICE
Connecting model inference to a usable web experience.
The project was not limited to running CatVTON independently. I connected the model to a FastAPI backend and frontend flow so users could provide inputs, start inference, and receive the generated fitting result as part of a complete application.
01
FastAPI Backend
Connected user inputs and mask-generation logic to the CatVTON inference pipeline.
02
Model-service Integration
Integrated image processing, model execution, and result delivery into the application workflow.
03
Progress Tracking UI
Added progress feedback so users could see the status of the longer-running AI inference process.
08 — RESULT
01
40 Avatars
Personalized avatar combinations based on gender, BMI, and height.
02
3 Fitting Types
Fit, Standard, and Loose options implemented through mask adjustment.
03
Automated Masks
SAM and OpenCV-based mask generation for tops, bottoms, and dresses.
04
Progress UI
Inference progress feedback added to improve the waiting experience.
09 — WHAT I LEARNED
Building an AI service requires more than selecting a model.
This project showed me that a model that works well under its original assumptions may not generalize directly to a new service environment. Adapting the preprocessing logic, improving masks, and integrating inference with the application were all necessary to turn the model into a usable service.
It also highlighted that improving generative AI systems requires both diverse data and better algorithms and preprocessing methods, rather than relying on training data alone.
PROJECT LINKS
←Back to Projects
DressMyFit · 2024