Episode 3 — NodeJS MongoDB Backend Architecture / 3.7 — Handling Files with Express

3.7.f — Cloud Storage: Cloudinary and ImageKit

In one sentence: Production applications almost never store uploaded files on the same server that runs the app — instead they offload files to cloud services like Cloudinary or ImageKit that provide scalable storage, automatic CDN delivery, on-the-fly image transformations, and URL-based processing.


Table of Contents


1. Why Cloud Storage Over Local Storage

FactorLocal Disk StorageCloud Storage (Cloudinary/S3)
ScalabilityLimited by server disk sizeVirtually unlimited
CDNNo (files served from one location)Yes (files cached worldwide)
RedundancyServer dies = files lostMultiple copies across data centers
TransformationsManual (Sharp, ffmpeg)On-the-fly via URL parameters
DeploymentFiles tied to one serverFiles persist across deploys
Load balancingFiles only on one serverAll servers access same storage
CostIncluded in server costPay per storage + bandwidth
MaintenanceYou manage disk, backups, cleanupProvider handles everything
┌─────────────────────────────────────────────────────────────────┐
│            LOCAL STORAGE PROBLEM                                 │
│                                                                 │
│  Server A (uploads here)    Server B (no files!)                │
│  ┌─────────────┐            ┌─────────────┐                    │
│  │  app.js     │            │  app.js     │                    │
│  │  uploads/   │            │  uploads/   │  ← EMPTY!          │
│  │   cat.jpg   │            │             │                    │
│  └─────────────┘            └─────────────┘                    │
│                                                                 │
│  Load balancer sends request to Server B → 404 Not Found!       │
│                                                                 │
│            CLOUD STORAGE SOLUTION                                │
│                                                                 │
│  Server A       Server B       Cloudinary                       │
│  ┌──────┐       ┌──────┐       ┌───────────────┐               │
│  │app.js│       │app.js│       │  cat.jpg      │               │
│  └──┬───┘       └──┬───┘       │  dog.jpg      │               │
│     │              │           │  (CDN cached)  │               │
│     └──────────────┴───────────┤               │               │
│        Both servers reference   └───────────────┘               │
│        the same cloud URLs                                      │
└─────────────────────────────────────────────────────────────────┘

2. Cloudinary — Setup and Configuration

Step 1: Create a Cloudinary account

  1. Go to cloudinary.com and sign up (free tier available)
  2. From the Dashboard, note your:
    • Cloud Name (e.g., dxyz1234)
    • API Key (e.g., 123456789012345)
    • API Secret (e.g., abcdefghijklmnopqrstuvwx)

Step 2: Install the SDK

npm install cloudinary

Step 3: Configure in your app

const cloudinary = require('cloudinary').v2;

// Option 1: Configure with individual values
cloudinary.config({
  cloud_name: process.env.CLOUDINARY_CLOUD_NAME,
  api_key: process.env.CLOUDINARY_API_KEY,
  api_secret: process.env.CLOUDINARY_API_SECRET
});

// Option 2: Configure with URL string
// cloudinary.config({
//   url: process.env.CLOUDINARY_URL
//   // CLOUDINARY_URL=cloudinary://API_KEY:API_SECRET@CLOUD_NAME
// });

Step 4: Environment variables

# .env file (never commit this!)
CLOUDINARY_CLOUD_NAME=dxyz1234
CLOUDINARY_API_KEY=123456789012345
CLOUDINARY_API_SECRET=abcdefghijklmnopqrstuvwx
// Load env vars
require('dotenv').config();

3. Cloudinary Upload API

Upload from a local file path

async function uploadFromDisk(filePath) {
  try {
    const result = await cloudinary.uploader.upload(filePath, {
      folder: 'avatars',            // organize into folder
      public_id: 'user_123_avatar', // custom ID (optional)
      overwrite: true,              // replace if same public_id exists
      resource_type: 'image'        // 'image', 'video', 'raw', 'auto'
    });

    console.log('Upload result:', result);
    return result;
  } catch (error) {
    console.error('Upload failed:', error);
    throw error;
  }
}

What the upload result looks like

{
  public_id: 'avatars/user_123_avatar',
  version: 1699123456,
  signature: 'abc123def456',
  width: 800,
  height: 600,
  format: 'jpg',
  resource_type: 'image',
  created_at: '2025-04-10T12:00:00Z',
  bytes: 234567,
  type: 'upload',
  url: 'http://res.cloudinary.com/dxyz1234/image/upload/v1699123456/avatars/user_123_avatar.jpg',
  secure_url: 'https://res.cloudinary.com/dxyz1234/image/upload/v1699123456/avatars/user_123_avatar.jpg'
}

Key result properties

PropertyDescription
public_idUnique identifier (used for deletion and transformations)
secure_urlHTTPS URL to the uploaded file
urlHTTP URL to the uploaded file
formatFile format (jpg, png, pdf, etc.)
width / heightDimensions (for images and videos)
bytesFile size in bytes
resource_typeimage, video, or raw

4. Cloudinary Upload from Buffer (Memory Storage)

The most common pattern: use Multer's memory storage, then upload the buffer to Cloudinary.

const express = require('express');
const multer = require('multer');
const cloudinary = require('cloudinary').v2;
require('dotenv').config();

const app = express();

// Cloudinary config
cloudinary.config({
  cloud_name: process.env.CLOUDINARY_CLOUD_NAME,
  api_key: process.env.CLOUDINARY_API_KEY,
  api_secret: process.env.CLOUDINARY_API_SECRET
});

// Memory storage — file stays in RAM as buffer
const upload = multer({
  storage: multer.memoryStorage(),
  limits: { fileSize: 5 * 1024 * 1024 },
  fileFilter: (req, file, cb) => {
    if (file.mimetype.startsWith('image/')) cb(null, true);
    else cb(new Error('Only images allowed'), false);
  }
});

// Helper: upload buffer to Cloudinary
function uploadToCloudinary(buffer, options = {}) {
  return new Promise((resolve, reject) => {
    const stream = cloudinary.uploader.upload_stream(
      {
        folder: options.folder || 'uploads',
        resource_type: 'auto',
        ...options
      },
      (error, result) => {
        if (error) reject(error);
        else resolve(result);
      }
    );

    // Write buffer to the upload stream
    stream.end(buffer);
  });
}

// Route
app.post('/api/upload', upload.single('image'), async (req, res) => {
  try {
    // Upload buffer directly to Cloudinary
    const result = await uploadToCloudinary(req.file.buffer, {
      folder: 'avatars',
      transformation: [
        { width: 500, height: 500, crop: 'fill', gravity: 'face' }
      ]
    });

    res.json({
      message: 'Uploaded to Cloudinary',
      url: result.secure_url,
      publicId: result.public_id,
      width: result.width,
      height: result.height,
      format: result.format
    });
  } catch (error) {
    res.status(500).json({ error: 'Cloud upload failed: ' + error.message });
  }
});

app.listen(3000, () => console.log('Server running'));

Why memory storage + Cloudinary? The file never touches your server's disk. Buffer goes straight from RAM to Cloudinary. No temp files to clean up.


5. Cloudinary Transformations

Apply transformations during upload — Cloudinary processes the image before storing it.

During upload

const result = await cloudinary.uploader.upload(filePath, {
  transformation: [
    { width: 800, height: 600, crop: 'fill' },     // Resize and crop
    { quality: 'auto', fetch_format: 'auto' },       // Auto-optimize
    { effect: 'sharpen' }                             // Apply effect
  ]
});

Common transformations

TransformationCodeEffect
Resize (fit inside){ width: 800, height: 600, crop: 'fit' }Scale down, keep aspect ratio
Resize (fill and crop){ width: 200, height: 200, crop: 'fill' }Crop to exact size
Face-aware crop{ width: 200, height: 200, crop: 'fill', gravity: 'face' }Crop centered on face
Auto quality{ quality: 'auto' }Cloudinary picks optimal quality
Auto format{ fetch_format: 'auto' }Serve WebP to Chrome, JPEG to Safari
Blur{ effect: 'blur:300' }Gaussian blur
Grayscale{ effect: 'grayscale' }Black and white
Round corners{ radius: 20 }Rounded corners
Circle crop{ radius: 'max' }Circular crop
Overlay (watermark){ overlay: 'watermark', gravity: 'south_east' }Add watermark image

Generate a thumbnail

const result = await cloudinary.uploader.upload(filePath, {
  eager: [
    { width: 150, height: 150, crop: 'thumb', gravity: 'face' }, // Thumbnail
    { width: 800, crop: 'scale' }                                  // Medium size
  ],
  eager_async: true  // Process in background
});

// Access the thumbnail URL
const thumbnailUrl = result.eager[0].secure_url;

6. Cloudinary URL-Based Transformations

One of Cloudinary's most powerful features: transform images by editing the URL. No re-upload needed.

URL structure

https://res.cloudinary.com/{cloud_name}/image/upload/{transformations}/{public_id}.{format}

Examples

const publicId = 'avatars/user_123';

// Original image
const original = `https://res.cloudinary.com/dxyz1234/image/upload/avatars/user_123.jpg`;

// Resize to 200x200, face-aware crop
const thumb = `https://res.cloudinary.com/dxyz1234/image/upload/w_200,h_200,c_fill,g_face/avatars/user_123.jpg`;

// Auto quality + auto format (best optimization)
const optimized = `https://res.cloudinary.com/dxyz1234/image/upload/q_auto,f_auto/avatars/user_123.jpg`;

// Blur the image
const blurred = `https://res.cloudinary.com/dxyz1234/image/upload/e_blur:500/avatars/user_123.jpg`;

// Chain multiple transformations
const multi = `https://res.cloudinary.com/dxyz1234/image/upload/w_800,c_scale/q_auto,f_auto/avatars/user_123.jpg`;

Building URLs with the SDK

const url = cloudinary.url('avatars/user_123', {
  width: 200,
  height: 200,
  crop: 'fill',
  gravity: 'face',
  quality: 'auto',
  fetch_format: 'auto',
  secure: true
});
// https://res.cloudinary.com/dxyz1234/image/upload/c_fill,f_auto,g_face,h_200,q_auto,w_200/avatars/user_123

URL transformation cheat sheet

ParameterCodeEffect
Widthw_200200px wide
Heighth_200200px tall
Crop modec_fillFill dimensions, crop excess
Gravityg_faceFocus on detected face
Qualityq_autoAutomatic quality optimization
Formatf_autoAuto-select best format
DPRdpr_2.02x resolution for Retina
Radiusr_2020px rounded corners
Effecte_blur:300Blur effect

7. multer-storage-cloudinary — Direct Integration

Instead of memory storage + manual upload, you can use a storage engine that uploads directly to Cloudinary:

npm install multer-storage-cloudinary
const { CloudinaryStorage } = require('multer-storage-cloudinary');
const cloudinary = require('cloudinary').v2;
const multer = require('multer');

cloudinary.config({
  cloud_name: process.env.CLOUDINARY_CLOUD_NAME,
  api_key: process.env.CLOUDINARY_API_KEY,
  api_secret: process.env.CLOUDINARY_API_SECRET
});

const storage = new CloudinaryStorage({
  cloudinary: cloudinary,
  params: {
    folder: 'avatars',
    allowed_formats: ['jpg', 'jpeg', 'png', 'webp'],
    transformation: [{ width: 500, height: 500, crop: 'limit' }]
  }
});

const upload = multer({ storage });

app.post('/api/upload', upload.single('image'), (req, res) => {
  // req.file now contains Cloudinary info
  res.json({
    url: req.file.path,       // Cloudinary URL
    publicId: req.file.filename, // Cloudinary public_id
    // Note: no local file was saved!
  });
});

Dynamic params based on request

const storage = new CloudinaryStorage({
  cloudinary: cloudinary,
  params: async (req, file) => {
    let folder = 'uploads';
    let transformation = [];

    if (file.mimetype.startsWith('image/')) {
      folder = 'images';
      transformation = [{ width: 1200, crop: 'limit', quality: 'auto' }];
    } else if (file.mimetype === 'application/pdf') {
      folder = 'documents';
    }

    return {
      folder: folder,
      resource_type: 'auto',
      transformation: transformation,
      public_id: `${Date.now()}-${file.originalname.split('.')[0]}`
    };
  }
});

8. ImageKit — Alternative Cloud Service

ImageKit is a Cloudinary alternative with a generous free tier and similar features.

Setup

npm install imagekit
const ImageKit = require('imagekit');

const imagekit = new ImageKit({
  publicKey: process.env.IMAGEKIT_PUBLIC_KEY,
  privateKey: process.env.IMAGEKIT_PRIVATE_KEY,
  urlEndpoint: process.env.IMAGEKIT_URL_ENDPOINT
  // e.g., https://ik.imagekit.io/your_id
});

Upload from buffer

app.post('/api/upload', upload.single('image'), async (req, res) => {
  try {
    const result = await imagekit.upload({
      file: req.file.buffer,              // buffer or base64 string
      fileName: req.file.originalname,    // desired filename
      folder: '/avatars',
      tags: ['avatar', 'user-upload']
    });

    res.json({
      url: result.url,
      fileId: result.fileId,
      thumbnailUrl: result.thumbnailUrl
    });
  } catch (error) {
    res.status(500).json({ error: 'ImageKit upload failed' });
  }
});

ImageKit URL-based transformations

// Original
// https://ik.imagekit.io/your_id/avatars/user_123.jpg

// Resize to 200x200
// https://ik.imagekit.io/your_id/tr:w-200,h-200/avatars/user_123.jpg

// Auto format + quality
// https://ik.imagekit.io/your_id/tr:f-auto,q-auto/avatars/user_123.jpg

// Face crop
// https://ik.imagekit.io/your_id/tr:w-200,h-200,fo-face/avatars/user_123.jpg

Cloudinary vs ImageKit comparison

FeatureCloudinaryImageKit
Free tier storage25 GB20 GB
Free tier bandwidth25 GB/month20 GB/month
Free transformations25,000/monthUnlimited
CDNYes (Akamai)Yes (AWS CloudFront)
URL transformationsYesYes
Video supportYesYes
AI featuresBackground removal, taggingBackground removal
SDK languagesJS, Python, Ruby, PHP, etc.JS, Python, Ruby, PHP, etc.

9. Deleting Files from Cloud Storage

Cloudinary deletion

// Delete by public_id
async function deleteFromCloudinary(publicId) {
  try {
    const result = await cloudinary.uploader.destroy(publicId);
    console.log('Delete result:', result); // { result: 'ok' }
    return result;
  } catch (error) {
    console.error('Delete failed:', error);
    throw error;
  }
}

// Usage in a route
app.delete('/api/users/:id/avatar', async (req, res) => {
  try {
    const user = await User.findById(req.params.id);

    if (user.avatar?.cloudinaryId) {
      await deleteFromCloudinary(user.avatar.cloudinaryId);
    }

    user.avatar = null;
    await user.save();

    res.json({ message: 'Avatar deleted' });
  } catch (error) {
    res.status(500).json({ error: 'Failed to delete avatar' });
  }
});

// Delete multiple files
async function deleteMany(publicIds) {
  const result = await cloudinary.api.delete_resources(publicIds);
  return result;
}

ImageKit deletion

// Delete by fileId (returned from upload response)
async function deleteFromImageKit(fileId) {
  try {
    await imagekit.deleteFile(fileId);
    console.log('File deleted from ImageKit');
  } catch (error) {
    console.error('Delete failed:', error);
    throw error;
  }
}

10. Real-World Pattern: Upload, Process, Store URL

This is how production applications handle file uploads end-to-end:

┌─────────────────────────────────────────────────────────────────┐
│                PRODUCTION UPLOAD PIPELINE                         │
│                                                                 │
│  1. CLIENT sends file via FormData                              │
│       ↓                                                         │
│  2. MULTER (memory storage) receives buffer                     │
│       ↓                                                         │
│  3. VALIDATION (MIME check, size, magic bytes)                  │
│       ↓                                                         │
│  4. CLOUDINARY receives buffer, processes, stores               │
│       ↓                                                         │
│  5. Cloudinary returns secure_url + public_id                   │
│       ↓                                                         │
│  6. MONGODB stores URL + metadata (not the file!)               │
│       ↓                                                         │
│  7. API returns URL to client                                   │
│       ↓                                                         │
│  8. CLIENT displays image using Cloudinary URL                  │
└─────────────────────────────────────────────────────────────────┘

Complete production example

const express = require('express');
const multer = require('multer');
const cloudinary = require('cloudinary').v2;
const mongoose = require('mongoose');
require('dotenv').config();

const app = express();
app.use(express.json());

// --- Cloudinary Config ---
cloudinary.config({
  cloud_name: process.env.CLOUDINARY_CLOUD_NAME,
  api_key: process.env.CLOUDINARY_API_KEY,
  api_secret: process.env.CLOUDINARY_API_SECRET
});

// --- Mongoose Model ---
const UserSchema = new mongoose.Schema({
  name: { type: String, required: true },
  email: { type: String, required: true },
  avatar: {
    url: String,
    cloudinaryId: String,
    width: Number,
    height: Number
  }
}, { timestamps: true });

const User = mongoose.model('User', UserSchema);

// --- Multer Config (memory) ---
const upload = multer({
  storage: multer.memoryStorage(),
  limits: { fileSize: 5 * 1024 * 1024 },
  fileFilter: (req, file, cb) => {
    const allowed = ['image/jpeg', 'image/png', 'image/webp'];
    if (allowed.includes(file.mimetype)) cb(null, true);
    else cb(new Error('Only JPEG, PNG, and WebP images are allowed'), false);
  }
});

// --- Cloudinary Upload Helper ---
function uploadToCloudinary(buffer, folder) {
  return new Promise((resolve, reject) => {
    cloudinary.uploader.upload_stream(
      {
        folder,
        resource_type: 'image',
        transformation: [
          { width: 500, height: 500, crop: 'fill', gravity: 'face' },
          { quality: 'auto', fetch_format: 'auto' }
        ]
      },
      (error, result) => {
        if (error) reject(error);
        else resolve(result);
      }
    ).end(buffer);
  });
}

// --- Routes ---

// Upload avatar for a user
app.put('/api/users/:id/avatar', upload.single('avatar'), async (req, res) => {
  try {
    const user = await User.findById(req.params.id);
    if (!user) return res.status(404).json({ error: 'User not found' });

    // Delete old avatar from Cloudinary if exists
    if (user.avatar?.cloudinaryId) {
      await cloudinary.uploader.destroy(user.avatar.cloudinaryId);
    }

    // Upload new avatar to Cloudinary
    const result = await uploadToCloudinary(req.file.buffer, 'avatars');

    // Save Cloudinary URL in database (not the file itself!)
    user.avatar = {
      url: result.secure_url,
      cloudinaryId: result.public_id,
      width: result.width,
      height: result.height
    };
    await user.save();

    res.json({
      message: 'Avatar updated',
      avatar: user.avatar
    });
  } catch (error) {
    res.status(500).json({ error: 'Avatar upload failed: ' + error.message });
  }
});

// Delete avatar
app.delete('/api/users/:id/avatar', async (req, res) => {
  try {
    const user = await User.findById(req.params.id);
    if (!user) return res.status(404).json({ error: 'User not found' });

    if (user.avatar?.cloudinaryId) {
      await cloudinary.uploader.destroy(user.avatar.cloudinaryId);
    }

    user.avatar = undefined;
    await user.save();

    res.json({ message: 'Avatar deleted' });
  } catch (error) {
    res.status(500).json({ error: 'Failed to delete avatar' });
  }
});

// Get user (avatar URL comes from DB, served by Cloudinary CDN)
app.get('/api/users/:id', async (req, res) => {
  const user = await User.findById(req.params.id);
  if (!user) return res.status(404).json({ error: 'User not found' });
  res.json(user);
});

// --- Error Handler ---
app.use((err, req, res, next) => {
  if (err instanceof multer.MulterError) {
    return res.status(400).json({ error: err.message });
  }
  if (err) return res.status(400).json({ error: err.message });
  next();
});

mongoose.connect(process.env.MONGODB_URI)
  .then(() => app.listen(3000, () => console.log('Server running')));

11. Cost Considerations and Free Tier Limits

Cloudinary free tier (as of 2025)

ResourceFree Limit
Storage25 GB
Monthly bandwidth25 GB
Transformations25,000/month
Video processing500 seconds/month

ImageKit free tier (as of 2025)

ResourceFree Limit
Storage20 GB
Monthly bandwidth20 GB
TransformationsUnlimited
Video processingLimited

When free tier runs out

┌─────────────────────────────────────────────────────────┐
│  COST OPTIMIZATION STRATEGIES                            │
│                                                          │
│  1. Resize images BEFORE uploading (client-side canvas   │
│     or server-side Sharp) to reduce storage              │
│                                                          │
│  2. Use q_auto and f_auto to reduce bandwidth            │
│     (up to 70% smaller files)                            │
│                                                          │
│  3. Set eager transformations at upload time instead      │
│     of generating on first request                       │
│                                                          │
│  4. Delete unused files regularly                        │
│                                                          │
│  5. Use CDN caching headers (long max-age)               │
│                                                          │
│  6. Consider AWS S3 + CloudFront for very high           │
│     volume (often cheaper at scale)                      │
└─────────────────────────────────────────────────────────┘

AWS S3 alternative (for reference)

npm install @aws-sdk/client-s3
// S3 is more cost-effective at large scale but has no built-in transformations
// You need a separate service (CloudFront, Lambda@Edge, imgproxy) for processing
// Cloudinary/ImageKit are easier for small-to-medium apps

12. Key Takeaways

  1. Use cloud storage in production — local disk storage does not scale across multiple servers or survive redeployments.
  2. Memory storage + Cloudinary is the cleanest pattern: no temp files on disk, buffer goes directly to the cloud.
  3. Cloudinary and ImageKit provide CDN, transformations, and optimization out of the box.
  4. URL-based transformations let you resize, crop, and optimize images just by changing the URL — no re-upload needed.
  5. Store the URL and public_id in your database — never store the file itself in the database.
  6. Delete old files when updating or removing resources — cloud storage costs money.
  7. multer-storage-cloudinary is a convenient shortcut but the manual buffer approach gives you more control.
  8. Free tiers are generous — most small apps stay within limits. Monitor usage as you grow.
  9. The production pipeline is: Client -> Multer (memory) -> Validate -> Cloud Upload -> Save URL to DB -> Return URL to Client.

Explain-It Challenge

Can you explain to a friend: "Why do production apps use Cloudinary instead of saving files to the server's hard drive?" If you can name at least four advantages and sketch the upload pipeline from memory, you have mastered this topic.


Next → 3.7-Exercise-Questions