Backend Engineering · Cloud Architecture · Document Automation
Scaling Enterprise Reporting with Asynchronous Document Processing
Moving large PDF and Excel report generation off the request path and into queued background processing, with dedicated workers, private storage and a controlled download.
- Industry
- Enterprise applications
- Project type
- Reporting architecture
- Attribution
- Anonymized Engineering Experience
Simplified system overview
- Report request
- QueueStorage & queues
- PDF workerExcel worker
- Storage → download
01 / Overview
Project overview
Enterprise applications frequently need to generate complex PDF and Excel reports containing large datasets, formatted documents, images and attachments.
This work improved the reporting architecture through asynchronous cloud processing. The application accepts a report request quickly, and dedicated workers do the heavy lifting in the background.
Anonymized Engineering Experience. This case study draws on anonymized enterprise engineering experience. It is not presented as a PathWebSolutions client engagement, and no organization, system or data is identified.
02 / Challenge
The challenge
Reports rarely stay small. As datasets, formatting rules and attachments grow, a report generated inside a single web request begins to fail in ways users notice.
- Request and execution time limits
- Memory-intensive document generation
- Attachment and image processing
- Complex PDF layouts and Excel formatting
- Synchronous processing that is unreliable for large jobs
- Recovering from failed jobs
- Delivering completed files securely
03 / Solution
The solution
The request was separated from the work. An API records the report request and places a job on a queue. Background processing picks it up, hands it to a dedicated PDF or Excel worker, processes attachments and images, and assembles the final document. The file is written to private object storage, and the application tracks the job until the user can download it.
04 / Capabilities
Key capabilities
Report request API
Validates the request, creates a job and responds immediately with a reference to it.
Background job queue
Amazon SQS holds work until a processor is ready, separating demand from processing capacity.
Dedicated PDF processing
Layout-heavy documents render in a worker built for PDF output.
Dedicated Excel processing
Spreadsheet generation and formatting run in their own worker.
Attachment and image handling
Images and attachments are processed and sized for the document instead of being embedded at full resolution.
Document assembly
Rendered content and processed attachments are combined into the final report.
Private object storage
Completed files are kept in Amazon S3 and are not publicly exposed.
Job status and download
The application shows a job’s progress and provides the file through a controlled download.
05 / Architecture
Technical architecture
How the parts connect. The animated example follows real stages of the system; every step is also described in text.
- User / application
- Report request API
- Retry & failure handlingJob queueAmazon SQS
- Background processingAWS Lambda
- Split by output formatPDF workerExcel worker
- Attachment & image processing
- Final report assembly
- Private storageAmazon S3
- Status & download
- Data flow
- Processing
- Storage & queues
- Outcome
How it works
Accept the request
The API validates the report request, records a job and returns immediately, so nobody waits on a long-running HTTP request.
Queue the work
The job goes onto Amazon SQS, which absorbs bursts of requests and holds work until background processing is ready.
Render by format
Background processing sends the job to a dedicated PDF or Excel worker, each suited to its format’s memory and formatting needs.
Process attachments and assemble
Images and attachments are processed, then combined with the rendered output into the final report.
Store and deliver
The finished file is written to private Amazon S3 storage. The application reports the job’s status and provides a controlled download.
06 / Decisions
Engineering decisions
Separate HTTP requests from long-running work
A request should finish quickly. Generation moved behind an API that only accepts and tracks jobs, so request time limits no longer decide whether a report succeeds.
Queue-based asynchronous execution
A queue separates how fast requests arrive from how fast reports can be produced. Work waits safely instead of overwhelming processors, and failed messages can be retried.
Dedicated workers for each document format
PDF layout and Excel generation use different libraries, memory profiles and failure modes. Separate workers can be sized, tuned and changed independently.
Private storage, controlled downloads
Reports can contain sensitive business data. Files stay in private storage and are provided only through a controlled download step in the application.
Errors, retries and job status
Each report is a job with a status. Failures are recorded rather than lost, transient errors can be retried, and users see progress instead of a request that never returns.
Trade-offs
What asynchronous processing asks in return.
Moving work off the request path solves the timeouts, but it creates responsibilities the design has to cover.
- Asynchronous reports keep the application responsive but need a status and download experience.
- Parallel workers increase throughput but need concurrency limits and processing that is safe to repeat.
- Smaller images reduce memory and file size, but resolution still has to suit the document’s purpose.
- Large datasets are processed in chunks rather than loaded into memory at once.
07 / Stack
Technologies
08 / Outcomes
Business outcomes
No performance, volume or cost figures are published for this anonymized work.
- Resource-intensive document generation runs separately from user requests.
- Background processing can be managed and scaled independently of the application.
- A sounder foundation for reliable document delivery.
- Failures are visible and recoverable instead of appearing as timeouts.
- Room for larger and more complex reports.
09 / Takeaways
What this demonstrates
- Designing asynchronous systems on AWS for long-running work
- Separating workloads by their resource and formatting needs
- Treating long-running work as observable jobs
- Delivering sensitive files through controlled access
Discuss a similar project
Have a similar engineering challenge?
Tell us about the workflow, integration, or technical problem you’re trying to solve.