form-data-parser

form-data-parser

A streaming multipart/form-data parser that solves memory issues with file uploads in server environments. Built as an enhanced replacement for the native request.formData() API, it enables efficient handling of large file uploads by streaming directly to disk or cloud storage services like AWS S3 or Cloudflare R2, preventing server crashes from memory exhaustion.

Features

• Drop-in replacement for request.formData() with streaming file upload support
• Minimal buffering - processes file upload streams with minimal memory footprint
• Standards-based - built on the web Streams API and File API
• Smart fallback - parses application/x-www-form-urlencoded requests with the same total size and field count limits, and uses native request.formData() for other form requests
• Storage agnostic - works with any storage backend (local disk, S3, R2, etc.)

Why You Need This

The native request.formData() method has a few major flaws in server environments:

• It buffers all file uploads in memory
• It does not provide fine-grained control over file upload handling
• It does not prevent DoS attacks from malicious requests

In normal usage, this makes it difficult to process requests with large file uploads because they can exhaust your server's RAM and crash the application.

For attackers, this creates an attack vector where malicious actors can overwhelm your server's memory by sending large payloads with many files.

form-data-parser solves this by handling file uploads as they arrive in the request body stream, allowing you to safely store files and use either a) the File directly or b) a unique identifier for that file in the returned FormData object.

Installation

npm i remix

Usage

The parseFormData interface allows you to define an "upload handler" function for fine-grained control of handling file uploads.

import * as fsp from 'node:fs/promises'
import type { FileUpload } from 'remix/form-data-parser'
import { parseFormData } from 'remix/form-data-parser'

// Define how to handle incoming file uploads
async function uploadHandler(fileUpload: FileUpload) {
  // Is this file upload from the <input type="file" name="user-avatar"> field?
  if (fileUpload.fieldName === 'user-avatar') {
    let filename = `/uploads/user-${user.id}-avatar.bin`

    // Store the file safely on disk
    await fsp.writeFile(filename, fileUpload.bytes)

    // Return the file name to use in the FormData object so we don't
    // keep the file contents around in memory.
    return filename
  }

  // Ignore unrecognized fields
}

// Handle form submissions with file uploads
async function requestHandler(request: Request) {
  // Parse the form data from the request.body stream, passing any files
  // through your upload handler as they are parsed from the stream
  let formData = await parseFormData(request, uploadHandler)

  let avatarFilename = formData.get('user-avatar')

  if (avatarFilename != null) {
    console.log(`User avatar uploaded to ${avatarFilename}`)
  } else {
    console.log(`No user avatar file was uploaded`)
  }
}

To validate the resulting FormData object with remix/data-schema, use the remix/data-schema/form-data helpers.

To limit the overall shape of multipart requests, use the maxHeaderSize, maxFileSize, maxFiles, maxParts, and maxTotalSize options. For application/x-www-form-urlencoded requests, maxParts limits the number of form fields and maxTotalSize limits the raw request body size. By default, parseFormData() uses maxFiles = 20, maxParts = 1000, and maxTotalSize = maxFiles * maxFileSize + 1 MiB.

Known limit errors are thrown directly so you can handle them with instanceof checks. Other failures while parsing the request body are wrapped in FormDataParseError, with the original error available as error.cause. Errors thrown or rejected by your uploadHandler are not wrapped.

import {
  FormDataParseError,
  MaxFilesExceededError,
  MaxFileSizeExceededError,
  MaxHeaderSizeExceededError,
  MaxPartsExceededError,
  MaxTotalSizeExceededError,
} from 'remix/form-data-parser'

const oneKb = 1024
const oneMb = 1024 * oneKb

try {
  let formData = await parseFormData(request, {
    maxFiles: 5,
    maxFileSize: 10 * oneMb,
    maxParts: 25,
    maxTotalSize: 12 * oneMb,
  })
} catch (error) {
  if (error instanceof MaxFilesExceededError) {
    console.error(`Request may not contain more than 5 files`)
  } else if (error instanceof MaxHeaderSizeExceededError) {
    console.error(`Multipart headers may not exceed the configured size limit`)
  } else if (error instanceof MaxFileSizeExceededError) {
    console.error(`Files may not be larger than 10 MiB`)
  } else if (error instanceof MaxPartsExceededError) {
    console.error(`Request may not contain more than 25 form fields or multipart parts`)
  } else if (error instanceof MaxTotalSizeExceededError) {
    console.error(`Form data request may not exceed 12 MiB of total content`)
  } else if (error instanceof FormDataParseError) {
    console.error(`Could not parse form data:`, error.cause ?? error)
  } else {
    throw error
  }
}

If you're looking for a more flexible storage solution for FileUpload objects, this library pairs really well with the file-storage library for keeping files in various storage backends.

import { createFsFileStorage } from 'remix/file-storage/fs'
import type { FileUpload } from 'remix/form-data-parser'
import { parseFormData } from 'remix/form-data-parser'

// Set up storage for uploaded files
const fileStorage = createFsFileStorage('/uploads/user-avatars')

// Define how to handle incoming file uploads
async function uploadHandler(fileUpload: FileUpload) {
  // Is this file upload from the <input type="file" name="user-avatar"> field?
  if (fileUpload.fieldName === 'user-avatar') {
    let storageKey = `user-${user.id}-avatar`

    // Put the file in storage and return the stored LazyFile
    return fileStorage.put(storageKey, fileUpload)
  }

  // Ignore unrecognized fields
}

Demos

The demos directory contains working demos:

• demos/node - using form-data-parser with file-storage in Node.js

Related Packages

• data-schema - Tiny, standards-aligned validation with a form-data export for FormData and URLSearchParams
• file-storage - A simple key/value interface for storing FileUpload objects you get from the parser
• multipart-parser - The parser used internally for parsing multipart/form-data HTTP messages

License

See LICENSE