Payroll Batch Processor
A Spring Boot and Spring Batch application for processing large employee payroll datasets with chunk-based processing, database persistence, CSV export, and scheduled batch execution.
Project Overview
Payroll Batch Processor is a Spring Boot application designed to process employee payroll data using Spring Batch.
The application reads employee records from CSV files, applies payroll calculations, stores the processed results in MySQL, and provides APIs for triggering batch processing and exporting the results back to CSV. It also supports scheduled execution for automated daily processing.
Technical Challenges
One of the main challenges was processing large CSV datasets efficiently without treating every employee record as a separate database transaction.
The batch pipeline uses FlatFileItemReader to read employee records and processes them in chunks of 500 records. EmpProcessor applies the payroll business rules and transforms each employee into a processed payroll record, while JdbcBatchItemWriter writes the results to MySQL using batch operations.
The processing flow is designed around Spring Batch’s reader, processor, and writer model, with transaction boundaries applied at the chunk level. The project also includes fault-tolerant processing with exception retries to help handle transient failures without immediately failing the entire job.
Another part of the project was handling repeatable batch execution. Existing payroll records can be updated when duplicate employee IDs are encountered, while Flyway manages the required database migrations.
Project Gallery
Key Highlights
The project demonstrates several practical batch-processing and backend engineering concepts:
- Spring Boot and Spring Batch based batch processing
- CSV-based employee data ingestion
- Chunk-oriented processing with 500 records per transaction
- Payroll transformation through a dedicated processor
- Batch database writes using
JdbcBatchItemWriter - MySQL persistence with Flyway migrations
- CSV export of processed payroll data
- REST endpoints for manually triggering processing and exports
- Automatic daily batch execution using Spring Scheduler
- Retry handling for processing failures
- Spring Batch metadata for tracking job and step execution
- Swagger/OpenAPI documentation for the REST APIs
Payroll Batch ProcessorA practical Spring Batch application demonstrating chunk-based processing, database persistence, scheduling, and reliable handling of large data workflows.
The payroll workflow applies a 10% tax and a 5% performance bonus for salaries above ₹60,000, then calculates the final net salary before persisting the processed record.
The application also provides dedicated endpoints to manually trigger payroll processing and export the processed data to CSV, while scheduled execution can run the processing automatically each day.