The Marketplace Problem
Oylkka Graphics is a marketplace where graphic designers upload and sell digital assets — photos, illustrations, mock-ups, templates. The technical challenge wasn't the marketplace logic (product listings, checkout, user accounts). It was the infrastructure: handling large image uploads in a serverless environment, making 2000+ assets searchable in milliseconds, and giving creators real-time feedback on their earnings.
Each of these problems had a non-obvious solution. Here's how I solved them.
The Serverless Upload Problem
The first blocker was file uploads. Vercel's serverless functions have a 4.5-second execution limit. A 10MB source image takes about 8 seconds to upload through a serverless proxy, process with Sharp, and store in the database. The function times out before the upload completes.
The naive approach — proxying uploads through the API — doesn't work at scale. I tried it, got timeout errors on 60% of uploads, and knew there had to be a better way.
The answer: presigned URLs. Instead of uploading through your server, the client uploads directly to cloud storage. Your server just generates a signed URL that authorizes the upload.
// Server: generate a presigned upload URL
import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
const s3 = new S3Client({ region: 'ap-south-1' });
export async function createUploadUrl(filename: string, contentType: string) {
const key = `uploads/${Date.now()}-${filename}`;
const command = new PutObjectCommand({
Bucket: process.env.S3_BUCKET,
Key: key,
ContentType: contentType,
// Allow public read after upload
ACL: 'public-read',
});
const presignedUrl = await getSignedUrl(s3, command, { expiresIn: 300 });
return { presignedUrl, key };
}
The client uploads directly to S3 using the presigned URL:
async function uploadAsset(file: File) {
// Step 1: Get a presigned URL from our server
const { presignedUrl, key } = await fetch('/api/upload-url', {
method: 'POST',
body: JSON.stringify({
filename: file.name,
contentType: file.type,
}),
}).then(r => r.json());
// Step 2: Upload directly to S3 (no server involved)
await fetch(presignedUrl, {
method: 'PUT',
body: file,
headers: { 'Content-Type': file.type },
});
// Step 3: Notify our server that upload is complete
await fetch('/api/upload-complete', {
method: 'POST',
body: JSON.stringify({ key, filename: file.name }),
});
return key;
}
The upload takes about 2 seconds for a 10MB file (limited by network speed, not serverless timeout). The server only handles two lightweight requests: generating the URL (100ms) and processing the upload notification (200ms). Both fit comfortably within the 4.5-second limit.
The Image Processing Pipeline
After the upload completes, the server needs to generate thumbnails, WebP variants, and responsive sizes. This is the heavy work — running Sharp on a 10MB image takes 2-3 seconds. In a serverless environment, you can't do this synchronously in the API route.
I moved image processing to a background job triggered by the upload-complete webhook:
// Server: webhook handler for upload completion
export async function handleUploadComplete(key: string, filename: string) {
// Queue the processing job (non-blocking)
await queue.add('process-image', { key, filename }, {
attempts: 3,
backoff: { type: 'exponential', delay: 5000 },
});
return { status: 'processing' };
}
// Background job: process the uploaded image
async function processImage(job: Job<{ key: string; filename: string }>) {
const { key, filename } = job.data;
// Download from S3
const original = await downloadFromS3(key);
// Generate variants
const fullSize = await sharp(original)
.resize(1200, null, { withoutEnlargement: true })
.webp({ quality: 80 })
.toBuffer();
const thumbnail = await sharp(original)
.resize(400, null, { withoutEnlargement: true })
.webp({ quality: 70 })
.toBuffer();
const blurPlaceholder = await sharp(original)
.resize(20)
.blur()
.webp({ quality: 20 })
.toBuffer();
// Upload variants to S3
await Promise.all([
uploadToS3(`processed/${key}-full.webp`, fullSize),
uploadToS3(`processed/${key}-thumb.webp`, thumbnail),
uploadToS3(`processed/${key}-blur.webp`, blurPlaceholder),
]);
// Update database with processed URLs
await db.asset.update({
where: { s3Key: key },
data: {
fullUrl: `processed/${key}-full.webp`,
thumbUrl: `processed/${key}-thumb.webp`,
blurUrl: `data:image/webp;base64,${blurPlaceholder.toString('base64')}`,
status: 'published',
},
});
}
The total processing pipeline takes about 4 seconds. The user sees "Processing..." in their dashboard, and the asset appears in the marketplace once all variants are uploaded. No timeout issues, no cold start problems.
Atlas Search for Instant Faceted Search
MongoDB's default text search is slow once you have more than a few thousand documents. I needed faceted search — filter by category, price range, popularity, and sort by relevance. MongoDB Atlas Search (built on Apache Lucene) handles this natively.
Setting up Atlas Search is a one-time operation — you create a search index on your collection:
{
"mappings": {
"dynamic": false,
"fields": {
"title": { "type": "autocomplete" },
"description": { "type": "text" },
"tags": { "type": "stringFacet" },
"category": { "type": "stringFacet" },
"price": { "type": "number" },
"popularity": { "type": "number" },
"createdAt": { "type": "date" }
}
}
}
The query combines text search with faceted filtering:
async function searchAssets(query: string, filters: SearchFilters) {
const pipeline = [
{
$search: {
compound: {
should: [
{
autocomplete: {
query,
path: 'title',
fuzzy: { maxEdits: 2 },
},
},
{
text: {
query,
path: ['description', 'tags'],
score: { boost: { value: 0.5 } },
},
},
],
},
highlight: { path: 'title' },
},
},
// Faceted filters
{
$match: {
...(filters.category && { category: filters.category }),
...(filters.minPrice && { price: { $gte: filters.minPrice } }),
...(filters.maxPrice && { price: { $lte: filters.maxPrice } }),
},
},
// Sort
{
$sort: filters.sortBy === 'popularity'
? { popularity: -1 }
: { _score: -1 },
},
// Pagination
{ $skip: filters.offset || 0 },
{ $limit: filters.limit || 20 },
];
return db.asset.aggregate(pipeline);
}
The autocomplete on title means users see results as they type. The fuzzy: { maxEdits: 2 } setting handles typos — "phooto" still finds "photo." And the facet fields on category and tags enable instant filter counts without extra queries.
The performance difference is dramatic. MongoDB text search took 800ms for a query across 2000 assets. Atlas Search returns results in 30-50ms, including highlights and facets.
Real-Time Creator Earnings
Creators want to know when their assets sell. Not "check back in an hour" — immediately. The challenge was that payment processing is asynchronous. Stripe sends a webhook when payment completes, but the creator might have closed their dashboard tab.
I used Server-Sent Events to push earnings updates to connected creators:
// Server: SSE connection for creator earnings
const creatorConnections = new Map<string, ReadableStreamDefaultController>();
export async function connectCreator(creatorId: string, controller: ReadableStreamDefaultController) {
creatorConnections.set(creatorId, controller);
// Send initial earnings data
const earnings = await getCreatorEarnings(creatorId);
controller.enqueue(`data: ${JSON.stringify({ type: 'initial', earnings })}\n\n`);
}
// Server: called when a sale completes
export async function notifyCreator(creatorId: string, sale: Sale) {
const controller = creatorConnections.get(creatorId);
if (controller) {
const earnings = await getCreatorEarnings(creatorId);
controller.enqueue(`data: ${JSON.stringify({ type: 'update', earnings, sale })}\n\n`);
}
}
The Stripe webhook handler calls notifyCreator after processing a sale:
// Stripe webhook handler
export async function handleStripeWebhook(event: Stripe.Event) {
if (event.type === 'payment_intent.succeeded') {
const paymentIntent = event.data.object as Stripe.PaymentIntent;
const sale = await processSale(paymentIntent);
// Notify the creator in real time
await notifyCreator(sale.creatorId, sale);
// Send email notification (non-blocking)
await queue.add('send-sale-email', { saleId: sale.id });
}
}
The key was separating the real-time notification from the email. The SSE push happens instantly (within 100ms of the webhook). The email goes to a background queue. If the email service is slow, it doesn't delay the creator seeing their earnings update.
The Full Flow
Here's the complete lifecycle of an asset on Oylkka Graphics:
- Creator uploads a file → presigned URL → direct S3 upload (2s)
- Upload-complete webhook → background job processes variants (4s)
- Asset appears in search index → Atlas Search makes it findable (<100ms)
- Buyer purchases → Stripe webhook → SSE notifies creator (<100ms)
- Creator sees earnings update in real time
The entire flow from upload to "asset is live and sellable" takes about 6 seconds. The creator sees it happen in real time through the SSE connection. No page refresh, no polling, no "check back later."
What I'd Change
The presigned URL flow works, but it requires the client to make three separate requests (get URL, upload, notify). A better UX would be a single drag-and-drop that handles all three internally. I'd wrap this in a custom hook that manages the upload state machine.
I'd also add retry logic for the image processing job. Currently, if Sharp fails (corrupted image, unsupported format), the asset stays in "processing" forever. A dead-letter queue with manual retry would be more robust.
Results
The marketplace handles 2000+ assets with sub-50ms search. Uploads complete reliably without timeout errors. Creators see earnings updates within 100ms of a sale. And the total infrastructure cost is under $20/month on Vercel + MongoDB Atlas + S3.
The presigned URL pattern is the biggest takeaway. If you're building anything that handles file uploads in a serverless environment, don't proxy through your API. Let the client upload directly to storage, and use your server only for authorization and post-processing.